Inside the .ftmw file

Every experiment is one self-contained, portable file with a .ftmw extension. The file is an HDF5 container that holds the raw measurement, every stage’s result, the settings each stage used, and the provenance of the source data. Moving an analysis between machines, archiving it, or handing it to a collaborator is a matter of copying this one file; reproducing the analysis requires nothing else.

What the file holds

A .ftmw file accumulates content as the pipeline runs. At any point it contains some prefix of:

• The raw FID — the time-series and its acquisition metadata (sample spacing, probe frequency, sideband, shot count, point count, duration), stored losslessly, together with any source-format recommended processing parameters.

• The canonical FT settings — the data-selection and display parameters that define the spectrum every later stage works on (see What is persisted, and what is recomputed below).

• The noise estimate — the per-bin noise from Stage 2.

• The decay-time calibrations — the exponential and Gaussian τ twins and the recommended line shape from Stage 2b, and the optional scope-timebase self-calibration.

• The detected peaks, the window plan, and the fitted model — from Stages 3, 4, and 5, the last including the per-window fitted-parameter covariance.

• The review record and final products — the consolidated, calibrated line list and the human-review decisions from Stage 6.

• The settings each stage used and the source provenance record.

The exact HDF5 group and attribute names are an implementation detail; the file is meant to be inspected through the Command-Line Reference (ftmwpipeline info) and the info() method rather than by reaching into fixed paths. Because the container is plain HDF5, it is also readable with any HDF5 tool (h5dump, h5py) for ad hoc inspection.

Stage tracking and dependencies

The pipeline is a sequence of stages with declared dependencies. The file records which stages have completed, and each stage checks that its inputs are present before it runs. The dependency graph is:

Stage	Requires
Stage 0 — import	nothing (it creates the file)
Stage 1 — FT	Stage 0
Stage 2 — noise	Stage 1
Stage 2b — τ calibration	Stages 0, 1, 2
Stage 3 — peak detection	Stages 1, 2 (Stage 2b recommended)
Stage 4 — window assignment	Stage 3
Stage 5 — fitting	Stages 0, 4 (Stage 2b recommended)
Stage 6 — review	Stage 5

Running a stage whose required predecessor has not completed fails with an error naming the missing dependency, rather than producing a result from incomplete inputs. Where Stage 2b is recommended but not required, the consuming stage runs without it and falls back to a documented default (Stage 3’s gap-pass decay basis, Stage 5’s per-window starting τ); supplying the calibration improves the result without being a precondition. The scope-timebase self-calibration depends only on the raw FID and feeds the report’s frequency uncertainty rather than any fit.

Re-running a stage with new parameters is safe. Re-running an earlier stage (or changing the canonical FT settings, or re-importing the source) invalidates the downstream results that depended on it, so the file is never left in a silently inconsistent state: the invalidated stages must be re-run.

What is persisted, and what is recomputed

The file stores the expensive, hard-to-reproduce products and recomputes the cheap ones on demand. The most visible case is the spectrum itself: the canonical FT is not stored. Only the processing parameters that define it are persisted, and the frequency-domain spectrum is recomputed from the raw FID plus those parameters whenever a stage needs it. A completed Stage 1 is therefore proven by the presence of its persisted parameters, not by a stored spectrum array.

The canonical FT is unconditionally unapodized, un-windowed, and native-length. The persisted Stage 1 parameters are the data selection (start time, end time, and the frequency trim range) plus display scaling; there are no apodization or zero-padding settings (apodization trades resolution and biases the line shape, and zero-padding corrupts the noise and χ² statistics that later stages depend on). Every later stage operates on the single spectrum these parameters define. How those parameters are resolved across explicit overrides, presets, and the persisted values is described in Settings and presets.

The noise estimate, peaks, window plan, fitted parameters (with covariance), and the finalized line list are all persisted, because they are expensive to recompute and are the scientific record of the analysis.

Provenance and safe re-import

When the file is created, it records the provenance of the data it was built from: the source path, the source modification time, a content hash, the import timestamp, the format name, and the loader parameters. This record is what makes re-import safe and reproducible.

Re-importing the same source onto an existing file is detected by the recorded provenance and is non-destructive — the existing analysis is reused rather than reprocessed, so re-running an import cell in a notebook does not discard downstream work. Importing a different source onto an existing file is refused unless overwriting is requested explicitly, so a finished analysis is never silently replaced.

Error conditions

The file model surfaces problems explicitly rather than guessing:

• Opening a file that does not exist raises FileNotFoundError with guidance on how to create one.

• Opening a file that is present but not a readable pipeline file raises PipelineCorruptionError.

• Creating over a file built from a different source raises PipelineExistsError with the options for proceeding.

• Running a stage whose dependency is missing raises StageDependencyError naming the unmet stage.

All of these are subclasses of a common PipelineFileError, so a script can catch the whole family at once.