pelagos_py.steps.input_output.load_data#

class pelagos_py.steps.input_output.load_data.LoadOG1(name, parameters=None, diagnostics=False, context=None)[source]#

Bases: pelagos_py.steps.base_step.BaseStep

Loads NetCDF files from file_path.

If filter_bad_times is set then measurements made outside of the time range specified by data_start and data_end are removed before further processing.

Time values are expected to be stored under the “TIME” variable name in the NetCDF file (corresponding to OG1 format). If “TIME” is not monotonically increasing, or has missing values (NaT) then this will raise an error.

Parameters:

  • filter_bad_time (bool, optional) – If True (default), removes all timestamps outside the expected time window.

  • data_start (str or np.datetime64, optional) – The minimum valid timestamp for the data. If not provided, the filter defaults to the DEPLOYMENT_TIME found in the dataset, or 1990-01-01T00:00:00 if no deployment time is found.

  • data_end (str or np.datetime64, optional) – The maximum valid timestamp for the data. If not provided, it defaults to the current system time when the pipeline is run.

Examples

Example usage in a pipeline configuration:

steps:
  - name: Load OG1
    parameters:
      file_path: "/path/to/your/dataset.nc"
      filter_bad_time: false
      data_start: "2023-05-01T00:00:00"
      data_end: "2024-05-01T00:00:00"
generate_diagnostics()[source]#

Print a structural summary of the loaded dataset.

Called automatically at the end of run() when diagnostics is enabled. Delegates to pelagos_py.utils.diagnostics.generate_info(), which prints the dataset’s dimensions, variables and global attributes (via xarray.Dataset.info()) to stdout — a quick check that the data was loaded as expected.