Prepare weather-model data for machine-learning training

Why use this

1. Weather-training data can become inconsistent when fields use different units, grids, forecast lead times or missing-value conventions. A visually plausible preview does not show whether channels are aligned correctly or whether a derived variable was calculated under the intended assumptions.

2. A channel-specification packet could make those dependencies explicit before fetching or transforming data. Researchers could review the intended arrays and lineage while keeping schema completeness separate from physical validity or suitability for model training.

Why an agent / simpler alternative

Useful judgment

Reconcile dataset metadata and identify channel definitions that require specialist clarification.

Simpler alternative

Use an established, validated dataset preparation pipeline when its field selection and grid already match the research design.

Unsuitable when

The task lacks authoritative variable definitions or expects data previews to establish scientific correctness.

Proposed inputs

1. Name

   Training-channel specification

   Owner

   Research lead

   Minimum scope

   Variable names, units, levels, times and array axes

   Quality check

   Identify derived variables and required source fields.

2. Name

   Dataset metadata sample

   Owner

   Authorized data owner

   Minimum scope

   Fictional field inventories, grid descriptions and missing-value conventions

   Quality check

   Check units, coordinate order and temporal alignment.

Access and setup — proposed or unknown

Proposed minimum

1. Offline metadata descriptions and a proposed channel manifest without data retrieval.

Compatibility unknowns

1. Unknown: current runtime, archive permissions, GRIB support and transformation implementation.

External actions requiring approval

1. Dataset access, credentials, downloads, compute and storage budgets, and export destinations.

Expected deliverable and review checks

Proposed artifact

Channel-to-source mapping with transformation assumptions, array layout and researcher validation questions.

Critical checks

1. Every derived channel lists its required inputs and unresolved assumptions.

2. Spatial axes and time semantics are explicit rather than inferred from filenames.

Missing output evidence

No new workflow run or output inspection was performed for this overlay; use the canonical source-outcome references.

Failure signals and proposed responses

1. Signal

   Two channels use different forecast lead times under one timestamp.

   Why it matters

   The resulting sample could mix incompatible temporal states.

   Proposed response

   Flag alignment and ask the researcher to specify the intended time relationship.

2. Signal

   A variable label omits its unit or vertical level.

   Why it matters

   An apparently matched field may represent a different quantity.

   Proposed response

   Keep the mapping unresolved until authoritative metadata is supplied.

Effort and cost — unknown

Setup work

1. Define channel contracts and data-lineage requirements.

Ongoing review

1. Inspect metadata exceptions and later authorized scientific checks.

Unknowns

1. Unknown: archive access costs, storage volume and transformation validity.

First check — proposed, unexecuted

Status

proposed_unexecuted

Synthetic fixture

An invented field manifest mixes forecast lead times and omits a unit and vertical level.

Acceptance checks

1. Identify unresolved channel mappings.

2. Reject a training-ready label without retrieving or transforming data.

Stop conditions

1. Network retrieval, array generation, compute allocation or scientific validation is needed.

Limits

1. A manifest review is not a meteorological validation, tensor inspection or executed preprocessing test.