Metrics Reference

Continuous Cell-Level Metrics

These metrics are computed at every grid cell for every timestep.

Key	Name	Formula	Units
ne	Normalised Error	(test - source) / source * 100	percent
ane	Absolute Normalised Error	abs((test - source) / source) * 100	percent
rse	Root Squared Error	sqrt((test - source)^2)	same as variable
bias	Mean Error (Bias)	test - source	same as variable
mae	Mean Absolute Error	abs(test - source)	same as variable

Categorical Cell-Level Metrics

Require a threshold parameter. Events are defined as values exceeding the threshold.

Key	Name	Formula	Range
pod	Probability of Detection	Hits / (Hits + Misses)	0--1
far	False Alarm Ratio	False Alarms / (Hits + False Alarms)	0--1
csi	Critical Success Index	Hits / (Hits + False Alarms + Misses)	0--1
fbias	Frequency Bias	(Hits + False Alarms) / (Hits + Misses)	0--inf

Domain-Aggregated Metrics

Computed over the spatial domain for each timestep.

Key	Name	Description
ne	Normalised Error	NE computed from domain-summed values
ane	Absolute Normalised Error	ANE computed from domain-summed values
rmse	Root Mean Square Error	Spatially aggregated RMSE
bias	Mean Error	Spatially aggregated bias
pearson	Pearson Correlation	Spatial correlation coefficient (-1 to 1)
pod	Probability of Detection	From domain contingency table
far	False Alarm Ratio	From domain contingency table
csi	Critical Success Index	From domain contingency table
gss	Gilbert Skill Score	Equitable threat score from domain contingency table
fbias	Frequency Bias	From domain contingency table

Station Metrics

Per-station time series metrics.

Key	Name	Description
bias	Mean Bias	Mean difference (model - obs)
mae	Mean Absolute Error	Mean absolute difference
rmse	Root Mean Square Error	RMS of differences
ne	Normalised Error	Normalised by observed values
ane	Absolute Normalised Error	Absolute normalised difference
pearson	Pearson Correlation	Temporal correlation per station

Wind Metrics

Vector wind evaluation from U/V components.

Key	Name	Units	Description
vector_rmse	Vector RMSE	m/s	sqrt(mean(du^2 + dv^2))
speed_bias	Wind Speed Bias	m/s	mean(test_speed - source_speed)
direction_bias	Wind Direction Bias	degrees	Mean directional difference (-180 to 180)

Fractions Skill Score (FSS)

Spatial verification at multiple neighborhood scales (default: 1, 3, 5, 9, 17, 33, 65 grid cells).

Value	Interpretation
0.0	No skill
0.5	Useful skill threshold
1.0	Perfect

Requires a threshold parameter to binarize the fields before computing fractions.

Diurnal Metrics

Any of the following metrics grouped by hour-of-day (0--23):

bias, rmse, mae, pearson

Supports a utc_offset parameter for local time conversion.

API

Full signatures for every function in modverif.metrics, including compute_residual_skill_score (residuals rather than model/obs pairs, and an RMSE-ratio rather than the more common MSE-ratio skill score).

Standardized meteorological verification metrics.

ContingencyTable

A class representing a 2x2 contingency table for binary events. Matches MET Grid-Stat/Point-Stat output categories.

bias()

Frequency Bias. Bias = (Hits + False Alarms) / (Hits + Misses).

csi()

Critical Success Index (Threat Score). CSI = Hits / (Hits + False Alarms + Misses).

far()

False Alarm Ratio. FAR = False Alarms / (Hits + False Alarms).

from_data(source_data, test_data, threshold) classmethod

Create a contingency table from forecast and observation data using a threshold.

gss()

Gilbert Skill Score (Equitable Threat Score). GSS = (Hits - Hits_random) / (Hits + False Alarms + Misses - Hits_random) where Hits_random = (Hits + Misses) * (Hits + False Alarms) / Total

pod()

Probability of Detection (Hit Rate). POD = Hits / (Hits + Misses).

compute_ane(source_data, test_data, epsilon=1e-10)

Compute absolute normalised error between source and test data, returning int16. ANE = |((test - source) / source)| * 100

compute_ane_1d(model, obs, epsilon=1e-10)

Absolute normalised error for 1D paired arrays (as percentage).

compute_ane_domain(source_data, test_data, mask=None, epsilon=1e-10)

Compute domain-aggregated absolute normalised error for each timestep.

compute_bias(source_data, test_data)

Compute Mean Error (Bias) between source and test data. Bias = test - source

compute_bias_domain(source_data, test_data, mask=None)

Compute domain-aggregated Mean Error (Bias) for each timestep.

compute_diurnal_stats(times, model, obs, metric='bias', utc_offset=0.0)

Compute a metric grouped by hour-of-day.

Parameters:

Name	Type	Description	Default
times	ndarray	Array of datetime64 values.	required
model	ndarray	1D model values (n_times,).	required
obs	ndarray	1D observation values (n_times,).	required
metric	str	One of 'bias', 'rmse', 'mae', 'pearson'.	'bias'
utc_offset	float	Hours to add to UTC to get local time.	0.0

Returns:

Name	Type	Description
hours	ndarray	Integer array of shape (24,).
values	ndarray	Float64 array of shape (24,).

compute_fraction_field(binary_field, neighborhood_size)

Compute fraction of True cells within a square neighborhood.

Parameters:

Name	Type	Description	Default
binary_field	ndarray	2D bool array (y, x).	required
neighborhood_size	int	Odd integer for neighborhood window size.	required

Returns:

Type	Description
ndarray	2D float array of fractions.

compute_fss(source_data, test_data, threshold, neighborhood_size, mask=None)

Compute Fractions Skill Score for a single 2D field.

Parameters:

Name	Type	Description	Default
source_data	ndarray	2D reference field (y, x).	required
test_data	ndarray	2D forecast/test field (y, x).	required
threshold	float	Binary event threshold.	required
neighborhood_size	int	Odd integer for neighborhood window size.	required
mask	ndarray	2D boolean mask. Only masked cells contribute.	None

Returns:

Type	Description
float	FSS value in [0, 1].

compute_fss_multi_scale(source_data, test_data, threshold, neighborhood_sizes=None, mask=None)

Compute FSS across multiple neighborhood sizes.

Parameters:

Name	Type	Description	Default
source_data	ndarray	2D reference field (y, x).	required
test_data	ndarray	2D forecast/test field (y, x).	required
threshold	float	Binary event threshold.	required
neighborhood_sizes	list[int]	Neighborhood sizes. Default: [1, 3, 5, 9, 17, 33, 65].	None
mask	ndarray	2D boolean mask.	None

Returns:

Type	Description
dict[int, float]	Mapping neighborhood_size -> FSS.

compute_lagged_correlation(model, obs, max_lag=None)

Cross-correlation between model and observation time series at multiple lags.

A positive optimal lag means the model leads (event arrives early); a negative optimal lag means the model lags (event arrives late). See compute_xcorr_best_lag, whose convention is the OPPOSITE of this one.

WARNING: non-finite pairs are dropped before lagging, so the series are compacted and the lag is counted in surviving-sample steps rather than in time. On a gappy series -- an interrupted rain gauge, say -- the returned "lag" is therefore not a lag in hours. For displaced-timing work on gappy observations use compute_xcorr_best_lag, which re-masks per lag and so keeps the lag axis in real time.

Parameters:

Name	Type	Description	Default
model	ndarray	1D model time series (n_times,).	required
obs	ndarray	1D observation time series (n_times,).	required
max_lag	int	Maximum lag (in timesteps) to evaluate in both directions. Default is n_times // 4.	None

Returns:

Name	Type	Description
lags	ndarray	Integer lag values from -max_lag to +max_lag.
correlations	ndarray	Pearson correlation at each lag.

compute_mae(source_data, test_data)

Compute Mean Absolute Error between source and test data. MAE = |test - source|

compute_mae_1d(model, obs)

Mean absolute error averaged over time.

compute_mean_bias(model, obs)

Mean error averaged over time. model and obs are 1D (n_times,).

compute_ne(source_data, test_data, epsilon=1e-10)

Compute normalised error between source and test data, returning int16. NE = ((test - source) / source) * 100

compute_ne_1d(model, obs, epsilon=1e-10)

Normalised error for 1D paired arrays (as percentage).

compute_ne_domain(source_data, test_data, mask=None, epsilon=1e-10)

Compute domain-aggregated normalised error for each timestep.

compute_pearson_correlation(source_data, test_data)

Compute Pearson correlation coefficient between two arrays.

compute_pearson_domain(source_data, test_data, mask=None)

Pearson correlation over the spatial domain per timestep.

compute_residual_skill_score(resid, resid_base)

Skill of one set of residuals against a baseline's, as a fraction of RMSE removed.

1 - rmse(resid) / rmse(resid_base). Positive means the model beat the baseline; zero means it matched it; negative means the baseline was better, which is the outcome worth reporting rather than hiding.

Two things to note before comparing this with a number from elsewhere:

• It takes residuals, not (model, obs) pairs, unlike the rest of this module. The baseline is often something with no per-point prediction to subtract -- a leave-one-out mean, a climatology -- so residuals are the only common currency.
• It is an RMSE ratio, not the more common MSE-ratio skill score (MSESS). The two are not interchangeable: an MSE ratio of 0.5 is an RMSE ratio of about 0.29.

Parameters:

Name	Type	Description	Default
resid	ndarray	Residuals of the method under test.	required
resid_base	ndarray	Residuals of the reference method.	required

Returns:

Type	Description
float	Skill score, or NaN if the baseline has zero RMSE (nothing to improve on).

compute_rmse_1d(model, obs)

RMSE over a 1D time series.

compute_rmse_domain(source_data, test_data, mask=None)

Compute domain-aggregated root mean square error for each timestep.

compute_rse(source_data, test_data)

Compute root squared error between source and test data. RSE = sqrt((test - source)^2)

compute_vector_rmse(source_u, source_v, test_u, test_v)

Vector RMSE: sqrt(mean((du)^2 + (dv)^2)).

Parameters:

Name	Type	Description	Default
source_u	ndarray	Reference U/V wind components (2D or 1D).	required
source_v	ndarray	Reference U/V wind components (2D or 1D).	required
test_u	ndarray	Test U/V wind components (2D or 1D).	required
test_v	ndarray	Test U/V wind components (2D or 1D).	required

Returns:

Type	Description
float	Scalar vector RMSE.

compute_wind_direction_bias(source_u, source_v, test_u, test_v)

Mean angular difference in wind direction (degrees). Uses circular statistics to handle wraparound.

Returns:

Type	Description
float	Mean directional bias in degrees [-180, 180].

compute_wind_speed_bias(source_u, source_v, test_u, test_v)

Wind speed bias: mean(test_speed - source_speed).

Returns:

Type	Description
float	Mean wind speed bias.

compute_xcorr_best_lag(model, obs, max_lag, min_pairs=48)

Best-correlating lag between an observation series and a model series, masked per lag.

Sits beside compute_lagged_correlation deliberately, because the two compute the same physical quantity and disagree about its sign. Keeping them apart would let a reader use either without noticing.

WARNING: the sign convention here is the OPPOSITE of compute_lagged_correlation.

• Here: a positive lag means the model is later than the observations -- matching a model_start - obs_start timing difference, which is how a displacement is usually reported.
• There: a positive lag means the model leads.

Neither is wrong; they were written for different questions. Do not "fix" one to match the other -- published results depend on both.

Because of that, swapping the two series is invisible: it returns the negated lag with a bitwise-identical correlation, and no output reveals the mistake. The argument order here is therefore (model, obs), matching every other function in this module, so habit gives the right answer.

Differs from compute_lagged_correlation in one other way that matters more than it looks: the finite mask is recomputed at each lag rather than once up front. Dropping non-finite pairs before lagging compacts the series, so the lag axis stops being time. Re-masking keeps it honest on gappy observations, at the cost of comparing slightly different samples across lags -- which is why min_pairs exists.

Parameters:

Name	Type	Description	Default
obs	ndarray	Observation series, evenly spaced in time. Non-finite entries mark gaps.	required
model	ndarray	Model series on the same time axis.	required
max_lag	int	Largest lag to test, in timesteps, in both directions.	required
min_pairs	int	A lag is skipped unless this many finite pairs overlap. Guards against a lag winning on a handful of coincidentally-agreeing points at the edge of the overlap.	48

Returns:

Name	Type	Description
best_lag	float	Lag with the highest correlation -- integer-valued, but typed float because it is NaN when no lag had enough pairs. Exact ties resolve to the most negative lag, since the scan runs from -max_lag upward -- relevant for periodic series where max_lag reaches a full period.
r_best	float	Correlation at that lag.
r_zero	float	Correlation at zero lag, for comparison -- how much the alignment actually bought.
n_pairs	int	Finite pairs contributing at the best lag.