A program to upgrade ioda files to a newer format. More...

#include <algorithm>
#include <cctype>
#include <cstdlib>
#include <exception>
#include <iostream>
#include <numeric>
#include <set>
#include <string>
#include <utility>
#include <vector>
#include "ioda/Engines/Factory.h"
#include "ioda/Engines/HH.h"
#include "ioda/Exception.h"
#include "ioda/Group.h"
#include "ioda/ObsGroup.h"
#include "ioda/Misc/DimensionScales.h"
#include "ioda/Misc/StringFuncs.h"

Include dependency graph for upgrade.cpp:

Go to the source code of this file.

Classes
struct	UpgradeParameters

Typedefs
typedef std::vector< ioda::Named_Variable >	Vec_Named_Variable

typedef std::map< ioda::Named_Variable, Vec_Named_Variable >	VarDimMap

Functions
bool	isPossiblyScale (const std::string &name)
	Convenience lambda to hint if a variable might be a scale. More...

std::list< std::string >	preferentialSortVariableNames (const std::vector< std::string > &allVars)
	Sort variable names in a preferential way so that likely scales end up first. More...

void	collectVarDimInfo (const ioda::Group &obsGroup, Vec_Named_Variable &varList, Vec_Named_Variable &dimVarList, VarDimMap &dimsAttachedToVars, ioda::Dimensions_t &maxVarSize0)
	Traverse file structure and determine dimension scales and regular variables. Also determine which dimensions are attached to which variables at which dimension numbers. More...

std::size_t	getChanSuffixPos (const std::string &name)
	Determine which variables may be grouped. More...

void	identifySimilarVariables (const Vec_Named_Variable &inVarList, VarDimMap &similarVariables, Vec_Named_Variable &dissimilarVariables)
	Determine which variables may be grouped. More...

void	copyData (const Vec_Named_Variable &old, ioda::Variable &newvar, const ioda::ObsGroup &base, const std::string &newVarName, const std::map< int, int > &chanNumToIndex)
	Copy data from oldvar into newvar. Offsets are supported for variable combination. More...

void	copyAttributes (const ioda::Has_Attributes &src, ioda::Has_Attributes &dest)
	Copy attributes from src to dest. Ignore duplicates and dimension scales. More...

bool	upgradeFile (const std::string &inputName, const std::string &outputName, const UpgradeParameters &params)

int	main (int argc, char **argv)

Detailed Description

A program to upgrade ioda files to a newer format.

Call program as: ioda-upgrade.x YAML_settings_file [input files] ... output_directory

Definition in file upgrade.cpp.

Typedef Documentation

◆ VarDimMap

typedef std::map<ioda::Named_Variable, Vec_Named_Variable> VarDimMap

Definition at line 86 of file upgrade.cpp.

◆ Vec_Named_Variable

typedef std::vector<ioda::Named_Variable> Vec_Named_Variable

Definition at line 85 of file upgrade.cpp.

Function Documentation

◆ collectVarDimInfo()

void collectVarDimInfo	(	const ioda::Group &	obsGroup,
		Vec_Named_Variable &	varList,
		Vec_Named_Variable &	dimVarList,
		VarDimMap &	dimsAttachedToVars,
		ioda::Dimensions_t &	maxVarSize0
	)

Traverse file structure and determine dimension scales and regular variables. Also determine which dimensions are attached to which variables at which dimension numbers.

Parameters

[in]	obsGroup	is the incoming group. Really any group works.
[out]	varList	is the list of variables (not dimension scales).
[out]	dimVarList	is the list of dimension scales.
[out]	dimsAttachedToVars	is the mapping of the scales attached to each variable.
[out]	maxVarSize0	is the max dimension length (nlocs). Unused here, but used in ioda.

Definition at line 95 of file upgrade.cpp.

Here is the call graph for this function:

◆ copyAttributes()

void copyAttributes	(	const ioda::Has_Attributes &	src,
		ioda::Has_Attributes &	dest
	)

Copy attributes from src to dest. Ignore duplicates and dimension scales.

Parameters

src	is the source.
dest	is the destination.

Definition at line 438 of file upgrade.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ copyData()

void copyData	(	const Vec_Named_Variable &	old,
		ioda::Variable &	newvar,
		const ioda::ObsGroup &	base,
		const std::string &	newVarName,
		const std::map< int, int > &	chanNumToIndex
	)

Copy data from oldvar into newvar. Offsets are supported for variable combination.

Parameters

oldvar	is the old variable.
newvar	is the new variable.
base	is the ObsGroup root object. Used in detecting ioda file versions.

Todo:: Add offset, oldvar_dims, newvar_dims.

Definition at line 337 of file upgrade.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getChanSuffixPos()

std::size_t getChanSuffixPos ( const std::string & name )

Determine which variables may be grouped.

Parameters

[in] inVarList is the list of all variables.

Definition at line 187 of file upgrade.cpp.

Here is the caller graph for this function:

◆ identifySimilarVariables()

void identifySimilarVariables	(	const Vec_Named_Variable &	inVarList,
		VarDimMap &	similarVariables,
		Vec_Named_Variable &	dissimilarVariables
	)

Determine which variables may be grouped.

Parameters

[in]	inVarList	is the list of all variables.
[out]	similarVariables	is the collection of similar variables, grouped by similarity and sorted numerically.
[out]	dissimilarVariables	are all variables that are not "similar".

Definition at line 214 of file upgrade.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ isPossiblyScale()

bool isPossiblyScale ( const std::string & name )

Convenience lambda to hint if a variable might be a scale.

This is not definitive, but has a high likelihood of being correct. The idea is that all variables will have either a "@" or "/" in their names, whereas dimension scales will not. This lambda returns true if the name has neither "@" nor "/" in its value.

Parameters

name	is the variable name

Returns: true if yes, false if no.

Definition at line 51 of file upgrade.cpp.

Here is the caller graph for this function:

◆ main()

int main	(	int	argc,
		char **	argv
	)

Definition at line 767 of file upgrade.cpp.

Here is the call graph for this function:

◆ preferentialSortVariableNames()

std::list<std::string> preferentialSortVariableNames ( const std::vector< std::string > & allVars )

Sort variable names in a preferential way so that likely scales end up first.

For speed.

Parameters

allVars is an unordered vector of all variables.

Returns: an ordered list. "nlocs" is first, then all potential scales, then all other variables.

Definition at line 61 of file upgrade.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ upgradeFile()

bool upgradeFile	(	const std::string &	inputName,
		const std::string &	outputName,
		const UpgradeParameters &	params
	)

Definition at line 492 of file upgrade.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

Classes

Typedefs

Functions

Detailed Description

Typedef Documentation

◆ VarDimMap

◆ Vec_Named_Variable

Function Documentation

◆ collectVarDimInfo()

◆ copyAttributes()

◆ copyData()

◆ getChanSuffixPos()

◆ identifySimilarVariables()

◆ isPossiblyScale()

◆ main()

◆ preferentialSortVariableNames()

◆ upgradeFile()