Met Office's implementation of the buddy check. More...

#include <MetOfficeBuddyCheck.h>

Inheritance diagram for ufo::MetOfficeBuddyCheck:

[legend]

Collaboration diagram for ufo::MetOfficeBuddyCheck:

[legend]

Classes
struct	MetaData
	Metadata of all observations processed by the filter. More...

Public Types
typedef MetOfficeBuddyCheckParameters	Parameters_

Public Member Functions
	MetOfficeBuddyCheck (ioda::ObsSpace &obsdb, const Parameters_ &parameters, std::shared_ptr< ioda::ObsDataVector< int > > flags, std::shared_ptr< ioda::ObsDataVector< float > > obserr)

	~MetOfficeBuddyCheck () override

Public Member Functions inherited from ufo::FilterBase
	FilterBase (ioda::ObsSpace &, const FilterParametersBaseWithAbstractAction &parameters, std::shared_ptr< ioda::ObsDataVector< int > >, std::shared_ptr< ioda::ObsDataVector< float > >)

	FilterBase (ioda::ObsSpace &, const eckit::Configuration &, std::shared_ptr< ioda::ObsDataVector< int > >, std::shared_ptr< ioda::ObsDataVector< float > >)

	~FilterBase ()

Public Member Functions inherited from ufo::ObsProcessorBase
	ObsProcessorBase (ioda::ObsSpace &, bool deferToPost, std::shared_ptr< ioda::ObsDataVector< int > >, std::shared_ptr< ioda::ObsDataVector< float > >)

	~ObsProcessorBase ()

void	preProcess () override

void	priorFilter (const GeoVaLs &) override

void	postFilter (const ioda::ObsVector &, const ioda::ObsVector &, const ObsDiagnostics &) override

oops::Variables	requiredVars () const override

oops::Variables	requiredHdiagnostics () const override

Static Public Member Functions
static const std::string	classname ()

Private Member Functions
void	print (std::ostream &) const override

void	applyFilter (const std::vector< bool > &, const Variables &, std::vector< std::vector< bool >> &) const override

int	qcFlag () const override

Variable	backgroundErrorVariable (const Variable &filterVariable) const
	Return the name of the variable containing the background error estimate of the specified filter variable. More...

std::vector< size_t >	getValidObservationIds (const std::vector< bool > &apply, const boost::optional< Eigen::ArrayXXi > &profileIndex) const
	Returns a vector of IDs of all observations that should be buddy-checked. More...

MetaData	collectMetaData (const boost::optional< Eigen::ArrayXXi > &profileIndex) const
	Collects and returns metadata of all observations. More...

std::vector< int >	getStationIds () const
	Returns a vector of integer-valued station IDs, obtained from the source indicated by the filter parameters. More...

std::vector< float >	calcBackgroundErrorHorizontalCorrelationScales (const std::vector< size_t > &validObsIds, const std::vector< float > &latitudes) const
	Calculates and returns background error correlation scales at observation locations. More...

std::vector< bool >	flagAndPrintVerboseObservations (const std::vector< size_t > &validObsIds, const std::vector< float > &latitudes, const std::vector< float > &longitudes, const std::vector< util::DateTime > &times, const std::vector< float > *pressures, const std::vector< int > &stationIds, const std::vector< float > &bgErrorHorizCorrScales) const
	Identifies observations whose buddy checks should be logged. More...

void	checkScalarData (const std::vector< MetOfficeBuddyPair > &pairs, const std::vector< int > &flags, const std::vector< bool > &verbose, const std::vector< float > &bgErrorHorizCorrScales, const std::vector< int > &stationIds, const std::vector< util::DateTime > &datetimes, const Eigen::ArrayXXf *pressures, const Eigen::ArrayXXf &obsValues, const Eigen::ArrayXXf &obsErrors, const Eigen::ArrayXXf &bgValues, const Eigen::ArrayXXf &bgErrors, Eigen::ArrayXXf &pges) const
	Buddy check for scalar quantities. More...

void	checkVectorData (const std::vector< MetOfficeBuddyPair > &pairs, const std::vector< int > &flags, const std::vector< bool > &verbose, const std::vector< float > &bgErrorHorizCorrScales, const std::vector< int > &stationIds, const std::vector< util::DateTime > &datetimes, const Eigen::ArrayXXf *pressures, const Eigen::ArrayXXf &uObsValues, const Eigen::ArrayXXf &vObsValues, const Eigen::ArrayXXf &obsErrors, const Eigen::ArrayXXf &uBgValues, const Eigen::ArrayXXf &vBgValues, const Eigen::ArrayXXf &bgErrors, Eigen::ArrayXXf &pges) const
	Buddy check for vector (two-dimensional) quantities. More...

void	flagRejectedObservations (const Variables &filtervars, const std::map< std::string, std::vector< float >> &grossErrProbsByVarName, std::vector< std::vector< bool >> &flagged) const

Private Attributes
Parameters_	options_

Additional Inherited Members
Protected Attributes inherited from ufo::FilterBase
const eckit::LocalConfiguration	config_

ufo::Variables	filtervars_

Protected Attributes inherited from ufo::ObsProcessorBase
ioda::ObsSpace &	obsdb_

std::shared_ptr< ioda::ObsDataVector< int > >	flags_

std::shared_ptr< ioda::ObsDataVector< float > >	obserr_

ufo::Variables	allvars_

ObsFilterData	data_

Detailed Description

Met Office's implementation of the buddy check.

The filter cross-checks observations taken at nearby locations against each other, updating their gross error probabilities (PGEs) and rejecting observations whose PGE exceeds a threshold specified in the filter parameters.

Variables to be checked should be specified using the "filter variables" YAML option, supporting surface (single-level) and multi-level variables. Variables can be either scalar or vector (with two Cartesian components, such as the eastward and northward wind components). In the latter case the two components need to specified one after the other in the "filter variables" list, with the first component having the first_component_of_two option set to true. Example:

filter variables:
- name: air_temperature
- name: eastward_wind
  options:
    first_component_of_two: true
- name: northward_wind

See MetOfficeBuddyCheckParameters for the documentation of the other available parameters.

This filter assumes background error estimates for each filter variable var can be retrieved from the var_background_error ObsDiagnostic. These diagnostics are typically produced using the BackgroundErrorVertInterp or BackgroundErrorIdentity operators. To make sure these operators are applied in addition to the "main" operator calculating model equivalents of observations, instruct the system to use a Composite operator with two or more components; for instance,

obs operator:
  name: Composite
  components:
  # operator used to evaluate H(x)
  - name: Identity
  # operator used to evaluate background errors
  - name: BackgroundErrorIdentity

Definition at line 82 of file src/ufo/filters/MetOfficeBuddyCheck.h.

Member Typedef Documentation

◆ Parameters_

typedef MetOfficeBuddyCheckParameters ufo::MetOfficeBuddyCheck::Parameters_

Definition at line 85 of file src/ufo/filters/MetOfficeBuddyCheck.h.

Constructor & Destructor Documentation

◆ MetOfficeBuddyCheck()

ufo::MetOfficeBuddyCheck::MetOfficeBuddyCheck	(	ioda::ObsSpace &	obsdb,
		const Parameters_ &	parameters,
		std::shared_ptr< ioda::ObsDataVector< int > >	flags,
		std::shared_ptr< ioda::ObsDataVector< float > >	obserr
	)

Definition at line 182 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

◆ ~MetOfficeBuddyCheck()

ufo::MetOfficeBuddyCheck::~MetOfficeBuddyCheck ( )

override

Definition at line 194 of file MetOfficeBuddyCheck.cc.

Member Function Documentation

◆ applyFilter()

void ufo::MetOfficeBuddyCheck::applyFilter	(	const std::vector< bool > &	apply,
		const Variables &	filtervars,
		std::vector< std::vector< bool >> &	flagged
	)		const

overrideprivatevirtual

Implements ufo::FilterBase.

Definition at line 199 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

◆ backgroundErrorVariable()

Variable ufo::MetOfficeBuddyCheck::backgroundErrorVariable ( const Variable & filterVariable ) const

private

Return the name of the variable containing the background error estimate of the specified filter variable.

Definition at line 334 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ calcBackgroundErrorHorizontalCorrelationScales()

std::vector< float > ufo::MetOfficeBuddyCheck::calcBackgroundErrorHorizontalCorrelationScales	(	const std::vector< size_t > &	validObsIds,
		const std::vector< float > &	latitudes
	)		const

private

Calculates and returns background error correlation scales at observation locations.

Only elements with indices corresponding to those of valid observations are filled in.

Definition at line 404 of file MetOfficeBuddyCheck.cc.

Here is the caller graph for this function:

◆ checkScalarData()

void ufo::MetOfficeBuddyCheck::checkScalarData	(	const std::vector< MetOfficeBuddyPair > &	pairs,
		const std::vector< int > &	flags,
		const std::vector< bool > &	verbose,
		const std::vector< float > &	bgErrorHorizCorrScales,
		const std::vector< int > &	stationIds,
		const std::vector< util::DateTime > &	datetimes,
		const Eigen::ArrayXXf *	pressures,
		const Eigen::ArrayXXf &	obsValues,
		const Eigen::ArrayXXf &	obsErrors,
		const Eigen::ArrayXXf &	bgValues,
		const Eigen::ArrayXXf &	bgErrors,
		Eigen::ArrayXXf &	pges
	)		const

private

Buddy check for scalar quantities.

Method: see the OPS Scientific Documentation Paper 2, sections 3.6 and 3.7

Parameters

	pairs	Buddy pairs.
	flags	Observation flags associated with the variable being checked.
	verbose	Whether to log buddy checks involving particular observations.
	bgErrorHorizCorrScales	Background error horizontal correlation scales (in km).
	stationIds	Station IDs ("call signs").
	datetimes	Observation times.
	pressures	Model average pressures can be null, representing surface data, single-level (num_levels parameter == 1) data or multi-level (num_levels > 1) data. In all cases except the single-level case, a vertical correlation of 1 is assumed. For single-level data, the estimate of the background error correlation depends upon the ratio of pressures between each pair of observations.
	obsValues	Observed values.
	obsErrors	Estimated errors of observed values.
	bgValues	Background values.
	bgErrors	Estimated errors of background values.
[in,out]	pges	Gross error probabilities. These values are updated by the buddy check.

Definition at line 462 of file MetOfficeBuddyCheck.cc.

Here is the caller graph for this function:

◆ checkVectorData()

void ufo::MetOfficeBuddyCheck::checkVectorData	(	const std::vector< MetOfficeBuddyPair > &	pairs,
		const std::vector< int > &	flags,
		const std::vector< bool > &	verbose,
		const std::vector< float > &	bgErrorHorizCorrScales,
		const std::vector< int > &	stationIds,
		const std::vector< util::DateTime > &	datetimes,
		const Eigen::ArrayXXf *	pressures,
		const Eigen::ArrayXXf &	uObsValues,
		const Eigen::ArrayXXf &	vObsValues,
		const Eigen::ArrayXXf &	obsErrors,
		const Eigen::ArrayXXf &	uBgValues,
		const Eigen::ArrayXXf &	vBgValues,
		const Eigen::ArrayXXf &	bgErrors,
		Eigen::ArrayXXf &	pges
	)		const

private

Buddy check for vector (two-dimensional) quantities.

Method: see the OPS Scientific Documentation Paper 2, sections 3.6 and 3.7

Parameters

	pairs	Buddy pairs.
	flags	Observation flags associated with the variable being checked.
	verbose	Whether to log buddy checks involving particular observations.
	bgErrorHorizCorrScales	Background error horizontal correlation scales (in km).
	stationIds	Station IDs ("call signs").
	datetimes	Observation times.
	pressures	Model average pressures can be null, represent surface data, single-level (num_levels parameter == 1) data or multi-level (num_levels > 1) data. In all cases except the single-level case, a vertical correlation of 1 is assumed. For single-level data, the estimate of the background error correlation depends upon the ratio of pressures between each pair of observations.
	uObsValues	Observed values of the first component, u.
	vObsValues	Observed values of the second component, v.
	obsErrors	Estimated errors of observed values (u or v).
	uBgValues	Background values of u.
	vBgValues	Background values of v.
	bgErrors	Estimated errors of background values (u or v).
[in,out]	pges	Probabilities of gross error in u or v. These values are updated by the buddy check.

Definition at line 565 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ classname()

static const std::string ufo::MetOfficeBuddyCheck::classname ( )

inlinestatic

Definition at line 86 of file src/ufo/filters/MetOfficeBuddyCheck.h.

◆ collectMetaData()

MetOfficeBuddyCheck::MetaData ufo::MetOfficeBuddyCheck::collectMetaData ( const boost::optional< Eigen::ArrayXXi > & profileIndex ) const

private

Collects and returns metadata of all observations.

Definition at line 338 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ flagAndPrintVerboseObservations()

std::vector< bool > ufo::MetOfficeBuddyCheck::flagAndPrintVerboseObservations	(	const std::vector< size_t > &	validObsIds,
		const std::vector< float > &	latitudes,
		const std::vector< float > &	longitudes,
		const std::vector< util::DateTime > &	times,
		const std::vector< float > *	pressures,
		const std::vector< int > &	stationIds,
		const std::vector< float > &	bgErrorHorizCorrScales
	)		const

private

Identifies observations whose buddy checks should be logged.

This function identifies observations located in one of the boxes specified in the tracedBoxes option, prints the metadata of these observations and sets the corresponding elements of the returned vector to true. Functions performing buddy checks are expected to log buddy checks involving these observations.

Parameters

pressures Optional – may be null.

Definition at line 424 of file MetOfficeBuddyCheck.cc.

Here is the caller graph for this function:

◆ flagRejectedObservations()

void ufo::MetOfficeBuddyCheck::flagRejectedObservations	(	const Variables &	filtervars,
		const std::map< std::string, std::vector< float >> &	grossErrProbsByVarName,
		std::vector< std::vector< bool >> &	flagged
	)		const

private

Marks observations whose gross error probability is >= options_->rejectionThreshold as rejected by the buddy check.

Definition at line 707 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getStationIds()

std::vector< int > ufo::MetOfficeBuddyCheck::getStationIds ( ) const

private

Returns a vector of integer-valued station IDs, obtained from the source indicated by the filter parameters.

Definition at line 370 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ getValidObservationIds()

std::vector< size_t > ufo::MetOfficeBuddyCheck::getValidObservationIds	(	const std::vector< bool > &	apply,
		const boost::optional< Eigen::ArrayXXi > &	profileIndex
	)		const

private

Returns a vector of IDs of all observations that should be buddy-checked.

Definition at line 692 of file MetOfficeBuddyCheck.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ print()

void ufo::MetOfficeBuddyCheck::print ( std::ostream & os ) const

overrideprivatevirtual

Implements ufo::FilterBase.

Definition at line 725 of file MetOfficeBuddyCheck.cc.

◆ qcFlag()

int ufo::MetOfficeBuddyCheck::qcFlag ( ) const

inlineoverrideprivatevirtual

Implements ufo::FilterBase.

Definition at line 100 of file src/ufo/filters/MetOfficeBuddyCheck.h.

Member Data Documentation

◆ options_

Parameters_ ufo::MetOfficeBuddyCheck::options_

private

Definition at line 245 of file src/ufo/filters/MetOfficeBuddyCheck.h.

The documentation for this class was generated from the following files:

Classes

Public Types

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ Parameters_

Constructor & Destructor Documentation

◆ MetOfficeBuddyCheck()

◆ ~MetOfficeBuddyCheck()

Member Function Documentation

◆ applyFilter()

◆ backgroundErrorVariable()

◆ calcBackgroundErrorHorizontalCorrelationScales()

◆ checkScalarData()

◆ checkVectorData()

◆ classname()

◆ collectMetaData()

◆ flagAndPrintVerboseObservations()

◆ flagRejectedObservations()

◆ getStationIds()

◆ getValidObservationIds()

◆ print()

◆ qcFlag()

Member Data Documentation

◆ options_