toolbox.pipeline

Pipeline Class

Classes

Pipeline

PipelineManager

A class enabling the execution of multiple pipelines in sequence.

Module Contents

class toolbox.pipeline.Pipeline(config_path=None)[source]
steps = [][source]
graph[source]
build_steps(steps_config, parent_name=None)[source]

Recursively build steps from configuration

add_step(step_name, parameters=None, diagnostics=False, parent_name=None, run_immediately=False)[source]

Dynamically adds a step and optionally runs it immediately

execute_step(step_config, _context)[source]

Executes a single step

run_last_step()[source]

Runs only the most recently added step

run()[source]

Runs the entire pipeline

visualise_pipeline()[source]

Generates a visualiation of the pipeline execution

generate_config()[source]

Generate a configuration dictionary from the current pipeline setup

export_config(output_path='generated_pipeline.yaml')[source]
class toolbox.pipeline.PipelineManager[source]

A class enabling the execution of multiple pipelines in sequence.

pipelines[source]
alignment_map[source]
settings[source]
load_mission_control(config_path)[source]

Load pipeline and alignment configuration from a mission control YAML file.

add_pipeline(name, config_path)[source]

Add a single pipeline with a unique name.

run_all()[source]

Run all registered pipelines and cache the resulting contexts.

get_contexts()[source]

Retrieve the context dictionary from each pipeline.

get_cached_contexts()[source]

Return previously cached contexts after run_all().

summarise_all_profiles() → pandas.DataFrame[source]

For all pipelines, summarise profiles and plot glider-to-glider distance time series. This includes:

  • Computing median TIME, LATITUDE, LONGITUDE per profile

  • Matching each profile to its closest in time from another source

  • Plotting a distance grid comparing all gliders

Returns:

Concatenated summary of all glider profiles, with closest match info appended.

Return type:

pd.DataFrame