API Reference¶
The public API consists of three ingest classes. WrfIngest and Era5Ingest are the
concrete converters; both share the convert() and resolve_variables() interface from
the H5Ingest base class.
File format conversions to cfdb
H5Ingest
¶
Abstract base class for converting HDF5/netCDF4 files to cfdb.
Subclasses must implement the abstract methods to provide source-specific parsing of CRS, time, spatial coordinates, variable mappings, and data reading.
Initialization derives all metadata from the input files and exposes it as attributes for inspection before calling convert().
Parameters¶
input_paths : str, Path, or list thereof One or more source HDF5/netCDF4 file paths.
file_glob_pattern = '*'
class-attribute
instance-attribute
¶
Glob pattern for finding source files in directories. Override in subclasses.
resolve_variables(variables)
¶
Resolve user-provided variable names to mapping keys.
Accepts mapping keys, source variable names, or cfdb short names. Returns a list of mapping keys. If variables is None, returns all available mapping keys.
When a cfdb_name maps to multiple keys (e.g., both surface and level-interpolated variants of air_temp), all matching keys are returned.
convert(cfdb_path, variables=None, start_date=None, end_date=None, bbox=None, target_levels=None, vertical_coord='height', max_mem=2 ** 27, chunk_shape=None, dataset_type='grid', **cfdb_kwargs)
¶
Convert source files to a cfdb dataset.
Variables are stored with coordinates appropriate to their type:
- Surface variables (height is a float): (time, y, x)
- Level-interpolated variables (height='levels'): (time,
Parameters¶
cfdb_path : str or Path Output cfdb file path. variables : list of str or None Variable names to convert (mapping keys, source names, or cfdb names). None converts all available mapped variables. start_date, end_date : str, np.datetime64, or None Optional time range filter. bbox : tuple of 4 floats or None Bounding box as (min_lon, min_lat, max_lon, max_lat) in WGS84. target_levels : list of float or None Target levels for level-interpolated variables. Interpretation depends on vertical_coord: height in meters or pressure in Pa. vertical_coord : str Name of the vertical coordinate for level-interpolated variables. 'height' (default) or 'pressure'. max_mem : int Memory budget in bytes for rechunkit read buffers. chunk_shape : tuple of ints or None Output chunk shape. For 4D variables: (time, z, y, x). For 3D surface variables: (1, ny, nx) is used automatically. Defaults to (1, 1, ny, nx) for 4D. dataset_type : str Passed to cfdb.open_dataset. **cfdb_kwargs Extra kwargs for cfdb.open_dataset (e.g., compression).
WrfIngest
¶
Bases: H5Ingest
Convert WRF output files to cfdb.
Handles WRF-specific features including CRS extraction from MAP_PROJ attributes, wind rotation from grid-relative to earth-relative using COSALPHA/SINALPHA, precipitation increment computation from accumulated fields, and 3D variable level interpolation from eta to height coordinates.
Parameters¶
input_paths : str, Path, or list thereof One or more wrfout file paths.
resolve_variables(variables)
¶
Resolve user-provided variable names to mapping keys.
Accepts mapping keys, source variable names, or cfdb short names. Returns a list of mapping keys. If variables is None, returns all available mapping keys.
When a cfdb_name maps to multiple keys (e.g., both surface and level-interpolated variants of air_temp), all matching keys are returned.
convert(cfdb_path, variables=None, start_date=None, end_date=None, bbox=None, target_levels=None, vertical_coord='height', max_mem=2 ** 27, chunk_shape=None, dataset_type='grid', **cfdb_kwargs)
¶
Convert source files to a cfdb dataset.
Variables are stored with coordinates appropriate to their type:
- Surface variables (height is a float): (time, y, x)
- Level-interpolated variables (height='levels'): (time,
Parameters¶
cfdb_path : str or Path Output cfdb file path. variables : list of str or None Variable names to convert (mapping keys, source names, or cfdb names). None converts all available mapped variables. start_date, end_date : str, np.datetime64, or None Optional time range filter. bbox : tuple of 4 floats or None Bounding box as (min_lon, min_lat, max_lon, max_lat) in WGS84. target_levels : list of float or None Target levels for level-interpolated variables. Interpretation depends on vertical_coord: height in meters or pressure in Pa. vertical_coord : str Name of the vertical coordinate for level-interpolated variables. 'height' (default) or 'pressure'. max_mem : int Memory budget in bytes for rechunkit read buffers. chunk_shape : tuple of ints or None Output chunk shape. For 4D variables: (time, z, y, x). For 3D surface variables: (1, ny, nx) is used automatically. Defaults to (1, 1, ny, nx) for 4D. dataset_type : str Passed to cfdb.open_dataset. **cfdb_kwargs Extra kwargs for cfdb.open_dataset (e.g., compression).
Era5Ingest
¶
Bases: H5Ingest
Convert ERA5 NetCDF files to cfdb.
Handles ERA5's one-variable-per-file structure. Input files are scanned to detect which variable each contains, and a file index is built for efficient access during conversion.
Parameters¶
input_paths : str, Path, or list thereof One or more ERA5 NetCDF file paths, or a directory containing them.
resolve_variables(variables)
¶
Resolve user-provided variable names to mapping keys.
Accepts mapping keys, source variable names, or cfdb short names. Returns a list of mapping keys. If variables is None, returns all available mapping keys.
When a cfdb_name maps to multiple keys (e.g., both surface and level-interpolated variants of air_temp), all matching keys are returned.
convert(cfdb_path, variables=None, start_date=None, end_date=None, bbox=None, target_levels=None, vertical_coord='pressure', max_mem=2 ** 27, chunk_shape=None, split=False, dataset_type='grid', **cfdb_kwargs)
¶
Convert ERA5 files to cfdb.
Parameters¶
cfdb_path : str or Path Output path. For combined mode, a .cfdb file path. For split mode, a directory where individual .cfdb files are created. variables : list of str or None Variable names to convert. None converts all available. start_date, end_date : str or None Optional time range filter. bbox : tuple of 4 floats or None Bounding box (min_lon, min_lat, max_lon, max_lat) in WGS84. target_levels : list of float or None Pressure levels in Pa for pressure-level variables. If None and pressure-level variables are requested, levels are read from the source files. vertical_coord : str Vertical coordinate name. Default 'pressure' for ERA5. max_mem : int Memory budget in bytes for read buffers. chunk_shape : tuple or None Output chunk shape for 4D variables. split : bool If True, create one cfdb file per variable. dataset_type : str Passed to cfdb.open_dataset. **cfdb_kwargs Extra kwargs for cfdb.open_dataset (e.g., compression).