pelagos_py.pipeline

pelagos_py.pipeline.REPORT_STEP_NAME = 'Write Data Report (Python)'

Name of the report step that triggers background diagnostic capture.

class pelagos_py.pipeline.Pipeline(config_path=None, config=None)

Bases: pelagos_py.utils.config_mirror.ConfigMirrorMixin

Pipeline that manages a sequence of processing steps.

Config-aware pipeline that can:

• Load config YAML into private self._parameters

• Keep global_parameters mirrored to _parameters[‘pipeline’]

• Build, run, and export steps as before

Parameters:

ConfigMirrorMixin (Class) – Class to handle configuration

classmethod load_config(config_path)

Create a pipeline from a YAML config file (same as Pipeline(config_path=...)).

Pipeline.load_config("my_pipeline.yaml").run()

classmethod make_config(file_path, ask=False, config_path=None)

Build a config for an OG1 file from the default template and create a pipeline from it.

The template’s steps are adapted to what the file holds (e.g. the PAR steps are dropped if it has no PAR) and the config is saved to config_path (default: <file name>.yaml next to the file), so the run can be edited and repeated with load_config(). With ask=True each choice is asked in the terminal; otherwise the defaults are used and logged.

Pipeline.make_config("glider.nc").run()
Pipeline.make_config("glider.nc", ask=True)  # build and save only

build_steps(steps_config)

Build steps from configuration.

Individual steps, including parameters and diagnostics, are saved to self.steps using add_step() for other functions.

Parameters:

steps_config (list of dict) – List of step configurations.

add_step(step_name, parameters=None, diagnostics=False, run_immediately=False)

Dynamically adds a step and optionally runs it immediately.

Parameters:

• step_name (str) – Name of the step to add.

• parameters (dict, optional) – Parameters for the step.

• diagnostics (bool, optional) – Whether to enable diagnostics for this step.

• run_immediately (bool, optional) – Whether to run the step immediately after adding it.

Raises:

ValueError – If the step name is not recognized.

execute_step(step_config, _context)

Executes a single step.

Parameters:

• step_config (dict) – Configuration for the step to execute.

• _context (dict) – Current context to pass to the step.

run_last_step()

Runs only the most recently added step based on the index in self.steps.

run()

Runs the entire pipeline.

generate_config()

Generate a configuration dictionary from the current pipeline setup.

Returns:

Configuration dictionary of the current pipeline.

Return type:

dict

export_config(output_path='generated_pipeline.yaml')

Write current config to file (respects private _parameters)

Parameters:

output_path (str) – Path to save the exported configuration YAML file.

Returns:

Configuration dictionary of the current pipeline.

Return type:

dict

save_config(path='pipeline_config.yaml')

Save the canonical private config (same as manager.save_config).

Parameters:

path (str) – Path to save the exported configuration YAML file.

get_data()

Returns data from the current pipeline context.