sage.data.noise.lowfar_noise

Start-time noise dataset and mining reader for GW detection training.

The standard training strategy randomly samples noise windows, so the model rarely sees the extreme-tail background that dominates searches at very low false alarm rates (FARs). “Hard” noise mining targets those windows — ones that fool a trained model into high ranking statistics — and persists only their per-detector start times (not raw strain) for later replay during training.

This module provides the two shared primitives for that workflow:

StartTimeDataset — a persisted set of per-detector (start, segment) indices

for hard windows, with .save()/.load() (npz + json sidecar).

_MiningReader — reads noise windows at given per-detector start times from

the memmap noise sampler, for (re)scoring by the model.

The mining algorithm that produces these datasets lives in sage.data.noise.cma_mae_mining (CMA-MAE, via pyribs).

Classes

StartTimeDataset

Persisted record of per-detector noise window start times and their scores.

Module Contents

class StartTimeDataset(detectors, start_indices, segment_indices, gps_times, scores, bin_files, sample_rate, seq_len)

Persisted record of per-detector noise window start times and their scores.

Each of the N samples is identified by D per-detector absolute memmap indices (start_indices) and the corresponding D segment IDs (segment_indices). The segment IDs match the segment_index field from the sidecar JSON metadata and are needed to look up PSDs during postprocessing.

Saved as a compressed .npz archive; string metadata (detector names, file paths) is written to a companion .json file next to the .npz.

Parameters:

• detectors (list[str]) – Ordered detector names, e.g. ["H1", "L1"].

• start_indices (np.ndarray, shape (N, D), dtype int64) – Absolute memmap start index for each sample × detector.

• segment_indices (np.ndarray, shape (N, D), dtype int64) – Segment ID for each sample × detector.

• gps_times (np.ndarray, shape (N,), dtype float64) – GPS start time of the H1 window (detector index 0), for reference.

• scores (np.ndarray, shape (N,), dtype float32) – Model ranking statistic for each sample.

• bin_files (list[str]) – Absolute paths to the original .bin noise files, one per detector.

• sample_rate (float) – Sample rate in Hz.

• seq_len (int) – Window length in samples (including padding).

detectors

start_indices

segment_indices

gps_times

scores

bin_files

sample_rate

seq_len

save(path)

Save to path (.npz + companion .json for string metadata).

classmethod load(path)

Load from path (.npz + companion .json).

filter(min_score)

Return a new dataset keeping only samples with score >= min_score.

merge(other)

Concatenate two compatible datasets (same detectors and bin_files).

dedup()

Return a new dataset with unique per-detector start-time rows.

A sample’s identity is its start_indices row — two samples with the same per-detector start indices are the same physical window. The hard-noise miner re-finds already-saved windows every epoch (replayed hard noise sits in the score tail by construction and re-clears the keep threshold), so without deduplication the accumulated dataset would grow with exact duplicates of the same few windows. The highest-scoring occurrence of each unique row is kept.