Settings and presets

Every pipeline stage past Stage 0 has knobs — noise smoothing widths, edge-coherence widths, line shape, decay-time bounds, rescue SNR cutoffs. ftmwpipeline exposes those knobs through a single layered resolution chain, at the level of detail a given workflow needs:

• one-off experiments: pass keyword arguments to a stage function;

• recurring instrument workflows: load a preset YAML by name and override one or two knobs from the CLI;

• programmatic sweeps: build a settings dataclass in Python and pass it as settings=.

All three surfaces resolve to the same settings, so the same parameters produce the same result whichever drives the analysis.

How a stage’s parameters resolve

A stage’s parameters come from layers that are merged per field. Highest precedence first:

1. explicit — a settings dataclass passed as settings= (plus the handful of genuine non-knob keyword arguments a stage accepts, such as shape on the fit)

2. persisted — what the previous run of this stage on this .ftmw file used

3. preset — a .yml preset loaded by name or path with preset=

4. recommended — an upstream-stage hint (e.g., Stage 2b’s recommended_shape for Stage 5); empty for stages without an upstream feeder

5. hard defaults — the library’s stock values

Each knob walks this chain independently. A value you set in an explicit settings= bundle wins; everything else falls through one layer at a time until it hits a concrete value, and the hard defaults are guaranteed to fill any remaining gap so the resolved settings instance is always complete.

A .yml preset never overrides a value persisted in the .ftmw file. The preset layer sits below the persisted layer: a preset only seeds fields the file has not already fixed. This is what keeps a shared .ftmw reproducible — the file reproduces the same result on its own, regardless of any preset a recipient happens to have loaded. To force a value regardless of what the file holds, pass it explicitly through settings=.

The same template applies to every stage that exposes knobs. The five settings dataclasses, in pipeline order:

Stage	Settings dataclass	Sub-blocks	Persisted at
1	FTSettings	flat (no sub-blocks)	processing_parameters/ft_processing
2	NoiseSettings	flat (no sub-blocks)	processing_parameters/stage2_noise
2b	TauCalibrationSettings	stft, polish, aggregation, band, gaussian, recommendation	processing_parameters/stage2b_tau
3	PeakDetectionSettings	promotion, savgol, primary_pass, gap_pass	processing_parameters/stage3_peaks
4	WindowPlanningSettings	coherence, clustering, contributor, leakage	processing_parameters/stage4_windows
5	StageFitSettings	shape, tau, seeder, conservative, penalties, rescue, thaw, spur, baseline, doublet_alternative, peak_survival	processing_parameters/stage5_fit

Each row is independent: you can tune Stage 2 noise smoothing without touching Stage 5, override Stage 3 SNR cutoffs without re-running the τ calibration, and so on. The persisted layer for one stage is unrelated to the persisted layer for another.

Three ways to drive a stage

The three input surfaces work identically across every stage. The examples below use Stage 5 (the fit step) because it has the most visible knobs; substitute estimate_noise, calibrate_tau, detect_peaks, assign_windows, or fit_peaks and the same patterns apply.

Stock defaults

The simplest call uses every hard default — Lorentzian shape, max_decay_factor=5, rescue.max_rounds=5, and the rest of the documented stock values:

import ftmwpipeline.api as ftmw
ftmw.fit_peaks("exp.ftmw")

Equivalent at the CLI:

ftmwpipeline fit run exp.ftmw

One-off overrides

At the command line, each stage exposes a flag per knob. A flag beats every other layer for its field; unspecified knobs flow through the chain unchanged:

ftmwpipeline fit run exp.ftmw --shape gaussian --max-decay-factor 3.0

Each flag is reconstructed into the stage’s settings object before the stage runs, so a flag and the corresponding settings= field are the same explicit override reached by two routes.

In Python, a few knobs survive as first-class convenience keyword arguments (the fit’s shape and its τ-override pair), part of the explicit layer as well:

ftmw.fit_peaks("exp.ftmw", shape="gaussian")

Every other knob is set through a settings= dataclass (below) or a preset YAML.

Presets

A preset is a named bundle of knob values that brings an experiment or instrument’s recipe under version control. A preset can cover one stage or several stages at once: a single YAML file can carry a stage2: block, a stage2b: block, a stage3: block, a stage4: block, and a stage5: block side-by-side. Each stage’s loader reads only its own block and ignores the rest, so one preset can drive a complete instrument-specific recipe.

ftmwpipeline ships exactly one packaged preset, defaults: every pipeline knob written at its package-wide default value, and nothing else. Applying it is a no-op — --preset defaults reproduces exactly what the pipeline does with no preset at all — so its purpose is documentation. It is the canonical, copy-and-edit starting point: open it, delete the blocks you do not care about, and change the few values you want to pin.

Use it by bare name (mostly to read it; as a recipe you would copy and edit it first):

ftmwpipeline fit run exp.ftmw --preset defaults

Or, the usual workflow, load a YAML file you wrote yourself by path:

ftmwpipeline fit run exp.ftmw --preset ./my_lab_recipe.yaml

The same preset name (or path) passed to any stage CLI subcommand loads only that stage’s block:

ftmwpipeline noise run    exp.ftmw --preset ./my_lab_recipe.yaml
ftmwpipeline tau run      exp.ftmw --preset ./my_lab_recipe.yaml
ftmwpipeline peaks run    exp.ftmw --preset ./my_lab_recipe.yaml
ftmwpipeline windows run  exp.ftmw --preset ./my_lab_recipe.yaml
ftmwpipeline fit run      exp.ftmw --preset ./my_lab_recipe.yaml

A stage whose block is missing from the preset loads an empty XxxSettings and falls through to the next layer of the resolver — nothing breaks.

A preset and explicit overrides compose: the explicit value wins per field, so you can adopt a preset’s recipe and tweak one knob in the same call. At the command line, that is a preset plus a per-knob flag:

ftmwpipeline fit run exp.ftmw \
  --preset ./my_lab_recipe.yaml \
  --max-residual-rescue-rounds 3

The flag lands in the explicit layer; the preset seeds the preset layer beneath it (and beneath anything the file has already persisted).

Preset YAML format

A preset is a YAML mapping with up to five top-level stage blocks — stage2 (noise), stage2b (τ calibration), stage3 (peaks), stage4 (windows), stage5 (the fit) — plus optional name and description metadata that the loader carries but ignores. Within a stage block, knobs are grouped into the same named sub-blocks the settings dataclass uses (for example stage5 has tau, seeder, conservative, penalties, rescue, thaw, spur, baseline, doublet_alternative, peak_survival; stage2 is flat, with no sub-blocks). Every field is optional: omit a knob and it falls through the resolver; write it and it is pinned at the preset layer. An unknown stage block, sub-block, or field name is rejected with an error so typos surface loudly rather than silently doing nothing.

name: my_lab_recipe
description: my instrument, tuned

stage2:
  window_mhz: 120.0            # flat — Stage 2 has no sub-blocks
stage3:
  promotion:
    min_snr: 4.0
stage5:
  shape: gaussian              # see the note below before pinning this
  tau:
    max_decay_factor: 4.0

For the meaning, type, and suggested values of every knob, read the shipped defaults preset (it lists them all at their defaults) or run ftmwpipeline settings show for a file’s resolved values and ftmwpipeline scan list for the tunable knobs and their sweep grids.

Line shape and the Stage 2b recommendation

The defaults preset leaves stage5.shape unset for a reason. Stage 2b runs a three-way lineshape vote (stage2b.recommendation.auto_recommend, default true) and stamps the winning shape on the .ftmw as the recommended layer. Stage 5 consumes that recommendation only when ``stage5.shape`` is unset — because a pinned shape: sits in the higher-precedence preset layer and silently overrides the recommendation. So:

• leave stage5.shape out to let Stage 2b choose the shape (the out-of-the-box behavior);

• set stage5.shape: lorentzian or gaussian to force it, knowingly bypassing the vote.

Declaring the instrument clock tree

The stage5.spur.clocks block declares an instrument’s clock-source fundamentals so the spur detector can use the locked intermod lattice instead of the generic integer-MHz anchor (and add a drift lane for any unlocked source). The defaults preset ships this commented out as a template; the Blackchirp 2638 instrument is shown as a worked example:

stage5:
  spur:
    clocks:
      - {freq_mhz: 5120, locked: true,  label: synth-downconv}
      - {freq_mhz: 5760, locked: true,  label: synth-upconv}
      - {freq_mhz: 16000, locked: true, label: awg}
      - {freq_mhz: 8000, locked: true,  label: awg-half}
      - {freq_mhz: 6250, locked: false, label: scope-adc}

Declare chain fundamentals only — harmonics and products derive. The same declaration can be supplied at import from a clocks.csv sidecar instead; see Declaring Instrument Clocks.

Constructing settings in Python

For programmatic sweeps (comparing several knob variants from a notebook, or building a recipe at runtime), construct the relevant settings dataclass directly:

from ftmwpipeline.core.stage_fit_settings import (
    StageFitSettings, ShapeSpec
)
from ftmwpipeline.core.peak_shape import PeakShape

s = StageFitSettings(shape=ShapeSpec(kind=PeakShape.GAUSSIAN))
s.tau.max_decay_factor = 3.0
s.rescue.max_rounds = 3
ftmw.fit_peaks("exp.ftmw", settings=s)

Same shape for any stage:

from ftmwpipeline.core.noise_settings import NoiseSettings

s = NoiseSettings()
s.window_mhz = 120.0
s.smoothing_mhz = 600.0
ftmw.estimate_noise("exp.ftmw", settings=s)

from ftmwpipeline.core.peak_detection_settings import PeakDetectionSettings

s = PeakDetectionSettings()
s.promotion.min_snr = 4.0
s.savgol.sg_window = 13
ftmw.detect_peaks("exp.ftmw", settings=s)

settings= and preset= populate different layers, so you can pass both in one call. A settings= bundle is the explicit layer and outranks the persisted record; a preset= YAML is the preset layer and is outranked by it. Reach for settings= to force values regardless of what the file holds, and for preset= to supply a recipe that defers to anything already chosen on the file. When both are given, the settings= fields win per field and the preset seeds the rest:

s = StageFitSettings()
s.rescue.max_rounds = 3
ftmw.fit_peaks("exp.ftmw", preset="./my_lab_recipe.yaml", settings=s)

Persistence and auto-inheritance

Every time a stage runs, its resolved settings are stamped into the .ftmw file under the canonical record for that stage (see the Persisted at column in the table above). The next call to the same stage on that file inherits those settings unless you override them, so a sequence like:

ftmwpipeline fit run exp.ftmw --preset ./my_lab_recipe.yaml
ftmwpipeline fit show exp.ftmw
# ... look at the fit, decide to tighten rescue ...
ftmwpipeline fit run exp.ftmw --max-residual-rescue-rounds 3

behaves as expected: the second fit keeps the settings from the first call’s preset and only tightens the rescue; the prior recipe need not be re-supplied.

The persisted block is structured to be inspectable on disk:

$ h5dump -A exp.ftmw | head -40
/processing_parameters/stage5_fit
  @creation_time = "2026-05-27T17:42:11..."
  @preset_name = "my_lab_recipe"
  shape/
    @kind = "gaussian"
  tau/
    @max_decay_factor = 5.0
    @per_band_tau = TRUE
    @tau_penalty_lambda = 500.0
    ...
  conservative/
    @significance = 0.05
    @max_peaks = 8
    ...
  rescue/
    @max_rounds = 5
    ...

Every sub-block of every stage is its own HDF5 group so you can grep one block in isolation. Unset fields use the __None__ sentinel string (the same convention Stage 1’s FTSettings uses).

The preset_name attribute records the bare name (or path) you supplied to --preset for that run. The reproducibility recipe is straightforward: the resolved values plus that name describe the run exactly.

Writing your own preset

A preset is a small YAML file. Each stage’s settings sit under a top-level per-stage block (stage2:, stage2b:, stage3:, stage4:, stage5:); blocks compose freely so one YAML can drive the whole pipeline:

name: my_lab_recipe
description: |
  Whatever your lab calls this recipe. Multi-line markdown ok.

stage2:
  window_mhz: 120.0
  smoothing_mhz: 600.0

stage2b:
  stft:
    n_seg: 10
  polish:
    polish_snr_cap: 9.0
  gaussian:
    snr_min: 20.0

stage3:
  promotion:
    min_snr: 4.0
    weak_medium_snr: 12.0
  savgol:
    sg_window: 13
  primary_pass:
    primary_window: blackmanharris

stage4:
  coherence:
    edge_m: 64
    edge_threshold: 8.0
  clustering:
    max_window_width_mhz: 40.0
  contributor:
    min_freeze_snr: 50.0

stage5:
  shape: gaussian
  tau:
    max_decay_factor: 3.0
    per_band_tau: true
  conservative:
    max_peaks: 6
  rescue:
    max_rounds: 3
    snr_threshold: 3.0

A few rules:

• Each per-stage block is independent — drop the ones you don’t need. A preset with only a stage5: block leaves Stages 2, 2b, 3, and 4 on hard defaults; a preset with only a stage2: block leaves Stage 5 alone, and so on.

• Inside a stage block, each sub-block (e.g., stage5.tau, stage3.promotion) holds a flat map of field_name: value. The fields available are listed in the Sub-blocks column of the table above and in each dataclass’s docstring.

• Stage 5’s shape accepts the short form (shape: gaussian); the long form (shape: {kind: gaussian}) also works and leaves room for future shape-specific parameter blocks (e.g., Voigt).

• Only set fields you care about. Anything omitted stays None so the resolver falls through to the next layer (probably the hard defaults).

• Unknown keys raise ValueError at load time, so typos surface immediately rather than silently doing the wrong thing.

• name and description at the top level are documentation — the parser preserves them but the stages don’t use them.

To use a YAML you wrote, pass its path:

ftmwpipeline fit run exp.ftmw --preset ./my_lab_recipe.yaml

To ship one alongside the package, drop it into src/ftmwpipeline/presets/ and refer to it by bare name.

Cross-stage recommendations

The recommended layer of the resolution chain is where one stage hands a hint to a later stage. The Stage 2b → Stage 5 path is the canonical example: Stage 2b’s τ calibration writes a recommended_shape attribute on its output group (lorentzian, gaussian, or voigt), and Stage 5’s resolver reads it as the recommended layer of the shape field. The attribute carries the __None__ sentinel until Stage 2b’s 3-way L/G/V discriminator runs; once it does, Stage 5 picks the recommendation up automatically — one step weaker than the file’s persisted value, two steps weaker than an explicit argument or preset. The pipeline recommends a line shape, but an explicit or persisted choice always overrides it.

If the file has persisted shape: gaussian and Stage 2b later recommends Lorentzian, the persisted value wins. An explicit --shape lorentzian likewise always wins, regardless of the recommendation.

The recommended layer of the other four stages (2, 2b, 3, 4) is reserved but currently empty — no upstream feeder produces a hint for those stages yet. The layer is kept in every resolver’s signature so a future cross-stage recommender (e.g., a Stage 1 T_active-driven Stage 2 smoothing-window suggestion, or a Stage 2b τ_maj feeder into Stage 4’s leakage.tau_us) can land without API churn.

Where to look in the codebase

Per-stage settings modules (all share the same architectural template):

• ftmwpipeline.core.noise_settings — NoiseSettings, with resolve() and load_preset().

• ftmwpipeline.core.tau_calibration_settings — TauCalibrationSettings.

• ftmwpipeline.core.peak_detection_settings — PeakDetectionSettings.

• ftmwpipeline.core.window_planning_settings — WindowPlanningSettings.

• ftmwpipeline.core.stage_fit_settings — StageFitSettings.

The matching HDF5 persistence groups are the Persisted at column of the settings table above.

Other useful references:

• src/ftmwpipeline/presets/ — the packaged preset YAML files.

• ftmwpipeline.core.settings — the Stage 1 FTSettings precedent the per-stage pattern extends.