Evaluation

Evaluation runs on the host and reads self-contained prediction artifacts. Framework evaluators remain independent of application packages. Application evaluators handle physical units, coordinate weights, gauge freedoms, domain metrics, and plots.

Contracts and outputs

EvaluationResult [source] `dataclass`

EvaluationResult(
    prediction_path: Path,
    output_dir: Path,
    metrics: dict[str, Any],
    per_frame: tuple[dict[str, float | int], ...] = (),
)

Collect metrics and files produced by one prediction evaluator.

Attributes:

Name	Type	Description
`prediction_path`	`Path`	Prediction artifact used as evaluator input.
`output_dir`	`Path`	Directory containing generated evaluation outputs.
`metrics`	`dict[str, Any]`	JSON-serializable global and summary metrics.
`per_frame`	`tuple[dict[str, float \| int], ...]`	Optional frame-wise metric rows written to CSV.

PredictionEvaluator [source]

Bases: Protocol

Define a host-only evaluator selected through Hydra configuration.

evaluate [source]

evaluate() -> EvaluationResult

Evaluate one saved prediction artifact and write configured outputs.

Returns:

Type	Description
`EvaluationResult`	Completed evaluation result.

write_evaluation_outputs [source]

write_evaluation_outputs(
    result: EvaluationResult,
) -> tuple[Path, Path | None]

Write metrics JSON and optional per-frame CSV files.

Parameters:

Name	Type	Description	Default
`result`	`EvaluationResult`	Completed evaluation result.	required

Returns:

Type	Description
`tuple[Path, Path \| None]`	Metrics JSON path and optional per-frame CSV path.

resolve_evaluation_output_dir [source]

resolve_evaluation_output_dir(
    prediction_path: str | Path,
    output_dir: str | Path | None = None,
) -> Path

Resolve an explicit destination or a results directory beside the prediction artifact.

Parameters:

Name	Type	Description	Default
`prediction_path`	`str \| Path`	Prediction artifact path.	required
`output_dir`	`str \| Path \| None`	Optional explicit output directory.	`None`

Returns:

Type	Description
`Path`	Resolved explicit directory or `<prediction_directory>/results`.

json_ready [source]

json_ready(value: Any) -> Any

Convert NumPy and non-finite values into strict JSON-compatible objects.

Parameters:

Name	Type	Description	Default
`value`	`Any`	Arbitrary value from an evaluation metrics tree.	required

Returns:

Type	Description
`Any`	Recursively converted JSON-compatible value.

Generic regression

RegressionEvaluator [source]

RegressionEvaluator(
    prediction_path: str | Path,
    output_dir: str | Path | None = None,
)

Evaluate aggregate errors in a self-contained PhiJAX prediction artifact.

Parameters:

Name	Type	Description	Default
`prediction_path`	`str \| Path`	Prediction `.npz` artifact path.	required
`output_dir`	`str \| Path \| None`	Optional directory receiving `metrics.json`. Defaults to `results` beside the artifact.	`None`

Store resolved input and output paths.

Parameters:

Name	Type	Description	Default
`prediction_path`	`str \| Path`	Prediction `.npz` artifact path.	required
`output_dir`	`str \| Path \| None`	Optional directory receiving `metrics.json`.	`None`

evaluate [source]

evaluate() -> EvaluationResult

Compute aggregate metrics and persist them as JSON.

Returns:

Type	Description
`EvaluationResult`	Completed aggregate evaluation result.

evaluate_prediction_artifact [source]

evaluate_prediction_artifact(
    prediction_path: str | Path,
) -> dict[str, float]

Evaluate flat prediction and target arrays from a PhiJAX artifact.

Parameters:

Name	Type	Description	Default
`prediction_path`	`str \| Path`	`.npz` artifact containing `flat_prediction` and `flat_target`.	required

Returns:

Type	Description
`dict[str, float]`	Aggregate regression metrics.

Raises:

Type	Description
`FileNotFoundError`	If the artifact does not exist.
`KeyError`	If required arrays are absent.
`ValueError`	If artifact arrays cannot be compared.

regression_metrics [source]

regression_metrics(
    prediction: ArrayLike, target: ArrayLike
) -> dict[str, float]

Compute aggregate regression errors for prediction and reference arrays.

Parameters:

Name	Type	Description	Default
`prediction`	`ArrayLike`	Finite predicted values with any non-empty shape.	required
`target`	`ArrayLike`	Finite reference values with the same shape as `prediction`.	required

Returns:

Type	Description
`dict[str, float]`	Relative L2 error, root mean squared error, mean absolute error, and maximum absolute error.

Raises:

Type	Description
`ValueError`	If arrays are empty, mismatched, non-finite, or the reference has zero L2 norm.

Metric primitives

finite_mask [source]

finite_mask(*arrays: ArrayLike) -> NDArray[np.bool_]

Select entries that are finite in every broadcast-compatible array.

Parameters:

Name	Type	Description	Default
`arrays`	`ArrayLike`	One or more arrays with broadcast-compatible shapes.	`()`

Returns:

Type	Description
`NDArray[bool_]`	Boolean array selecting jointly finite entries.

Raises:

Type	Description
`ValueError`	If no arrays are supplied.

normalized_rmse [source]

normalized_rmse(
    prediction: ArrayLike,
    reference: ArrayLike,
    *,
    denominator: float,
    mask: ArrayLike | None = None,
    weights: ArrayLike | None = None,
) -> float

Compute percent normalized RMSE over finite masked entries.

Parameters:

Name	Type	Description	Default
`prediction`	`ArrayLike`	Predicted scalar field.	required
`reference`	`ArrayLike`	Reference scalar field.	required
`denominator`	`float`	Positive global normalization value.	required
`mask`	`ArrayLike \| None`	Optional Boolean validity mask.	`None`
`weights`	`ArrayLike \| None`	Optional non-negative mean-square weights.	`None`

Returns:

Type	Description
`float`	Normalized RMSE in percent, or `nan` when it is undefined.

vector_normalized_rmse [source]

vector_normalized_rmse(
    prediction_components: Sequence[ArrayLike],
    reference_components: Sequence[ArrayLike],
    *,
    denominator: float,
    mask: ArrayLike | None = None,
    weights: ArrayLike | None = None,
) -> float

Compute percent normalized RMSE for vector-valued fields.

Parameters:

Name	Type	Description	Default
`prediction_components`	`Sequence[ArrayLike]`	Predicted scalar components in a common basis.	required
`reference_components`	`Sequence[ArrayLike]`	Reference scalar components in matching order.	required
`denominator`	`float`	Positive global vector-magnitude normalization.	required
`mask`	`ArrayLike \| None`	Optional Boolean validity mask.	`None`
`weights`	`ArrayLike \| None`	Optional non-negative mean-square weights.	`None`

Returns:

Type	Description
`float`	Vector normalized RMSE in percent, or `nan` when it is undefined.

Raises:

Type	Description
`ValueError`	If component counts differ or no components are supplied.

squared_correlation [source]

squared_correlation(
    prediction: ArrayLike,
    reference: ArrayLike,
    *,
    mask: ArrayLike | None = None,
) -> float

Compute squared Pearson correlation over finite masked entries.

Parameters:

Name	Type	Description	Default
`prediction`	`ArrayLike`	Predicted scalar field.	required
`reference`	`ArrayLike`	Reference scalar field.	required
`mask`	`ArrayLike \| None`	Optional Boolean validity mask.	`None`

Returns:

Type	Description
`float`	Squared correlation, or `nan` when fewer than two varying values exist.

max_abs [source]

max_abs(
    values: ArrayLike, *, mask: ArrayLike | None = None
) -> float

Return the finite masked maximum absolute value.

Parameters:

Name	Type	Description	Default
`values`	`ArrayLike`	Scalar values to reduce.	required
`mask`	`ArrayLike \| None`	Optional Boolean validity mask.	`None`

Returns:

Type	Description
`float`	Maximum absolute value, or `nan` when no valid value exists.

vector_max_magnitude [source]

vector_max_magnitude(
    components: Sequence[ArrayLike],
    *,
    mask: ArrayLike | None = None,
) -> float

Return the finite masked maximum vector magnitude.

Parameters:

Name	Type	Description	Default
`components`	`Sequence[ArrayLike]`	Non-empty scalar components in a shared basis.	required
`mask`	`ArrayLike \| None`	Optional Boolean validity mask.	`None`

Returns:

Type	Description
`float`	Maximum magnitude, or `nan` when no valid value exists.

Raises:

Type	Description
`ValueError`	If no components are supplied.

robust_summary [source]

robust_summary(values: Sequence[float]) -> dict[str, float]

Summarize a metric series using its range, median, and MAD scale.

Parameters:

Name	Type	Description	Default
`values`	`Sequence[float]`	Numeric metric values, possibly including non-finite entries.	required

Returns:

Type	Description
`dict[str, float]`	Mapping containing `min`, `max`, `median`, and `robust_sigma`.

subtract_weighted_frame_means [source]

subtract_weighted_frame_means(
    values: ArrayLike,
    *,
    weights: ArrayLike,
    mask: ArrayLike,
) -> tuple[NDArray[np.float64], NDArray[np.float64]]

Subtract a weighted spatial mean independently from every final-axis frame.

Parameters:

Name	Type	Description	Default
`values`	`ArrayLike`	Scalar field with at least one spatial axis followed by a frame axis.	required
`weights`	`ArrayLike`	Non-negative weights broadcast-compatible with `values`.	required
`mask`	`ArrayLike`	Boolean validity mask shaped like `values`.	required

Returns:

Type	Description
`tuple[NDArray[float64], NDArray[float64]]`	Gauge-aligned values and the subtracted mean for every frame.

Raises:

Type	Description
`ValueError`	If `values` has fewer than two dimensions or other arrays are not broadcast-compatible.

Evaluation

Contracts and outputs

EvaluationResult [source] dataclass

PredictionEvaluator [source]

evaluate [source]

write_evaluation_outputs [source]

resolve_evaluation_output_dir [source]

json_ready [source]

Generic regression

RegressionEvaluator [source]

evaluate [source]

evaluate_prediction_artifact [source]

regression_metrics [source]

Metric primitives

finite_mask [source]

normalized_rmse [source]

vector_normalized_rmse [source]

squared_correlation [source]

max_abs [source]

vector_max_magnitude [source]

robust_summary [source]

subtract_weighted_frame_means [source]

EvaluationResult [source] `dataclass`