sage.data.primer.retry

Utilities for retrying failed or missing GWOSC segment downloads.

Three public functions are provided:

get_expected_segments

Re-query GWOSC and split into mini-segments exactly as the original download did, returning the full list of expected (b0, b1) GPS pairs.

find_missing_segments

Compare that expected list against an already-downloaded segments.json and return records for every segment that is absent.

retry_detector

End-to-end retry for one detector: discover missing/failed segments (from a failed_segments.json if present, otherwise via TimelineQuery) and append them to the existing .bin and segments.json files.

Functions

get_expected_segments(detector, run[, sample_rate, ...])

Reconstruct the full list of expected mini-segments for a detector/run

find_missing_segments(detector, run, existing_meta[, ...])

Find mini-segments expected by the GWOSC timeline but absent from

retry_detector(detector, run, data_dir[, num_workers, ...])

Discover and re-download all missing or failed segments for one detector,

Module Contents

get_expected_segments(detector, run, sample_rate=2048.0, corrupt_trim_fraction=0.2, mini_segment_duration=512.0, split_minimum_duration=16.0, prune_minimum_duration=22.0, rm_window_length=30.0)[source]

Reconstruct the full list of expected mini-segments for a detector/run via TimelineQuery, using the same parameters as the original download.

Parameters:

  • detector (str) – Detector prefix, e.g. "H1".

  • run (str) – Observing run label, e.g. "O3a" or "O3b".

  • sample_rate (float) – Target sample rate in Hz (default 2048).

  • corrupt_trim_fraction (float) – Fraction of a second trimmed from each edge to remove downsampling artefacts. Rounded up to the nearest integer sample count.

  • mini_segment_duration (float) – Body length of each mini-segment in seconds, before the buffer is added to each edge (default 512.0).

  • split_minimum_duration (float) – Minimum duration passed to split_into_mini_segments (default 16.0).

  • prune_minimum_duration (float) – Minimum segment duration passed to prune_segments (default 22.0).

  • rm_window_length (float) – Half-window in seconds around known events to excise (default 30.0).

Returns:

Pre-trim GPS (b0, b1) pairs for every expected mini-segment.

Return type:

list of (float, float)

find_missing_segments(detector, run, existing_meta, sample_rate=2048.0, corrupt_trim_fraction=0.2, mini_segment_duration=512.0, split_minimum_duration=16.0, prune_minimum_duration=22.0, rm_window_length=30.0)[source]

Find mini-segments expected by the GWOSC timeline but absent from existing_meta (the loaded *_segments.json content).

The comparison un-trims the stored GPS times by corrupt_trim_fraction before matching against the pre-trim values returned by GWOSC, then rounds to one decimal place to absorb floating-point drift.

Parameters:

  • detector (str)

  • run (str)

  • existing_meta (list[dict]) – Contents of the data_{det}_{run}_segments.json sidecar.

  • sample_rate

  • corrupt_trim_fraction

  • mini_segment_duration

:param : :param split_minimum_duration: Same meaning as in get_expected_segments(). :param prune_minimum_duration: Same meaning as in get_expected_segments(). :param rm_window_length: Same meaning as in get_expected_segments().

Returns:

One record per missing segment with keys segment_index, detector, observing_run, gps_start, gps_end, reason. Ready to pass directly to retry_detector().

Return type:

list[dict]

retry_detector(detector, run, data_dir, num_workers=8, sample_rate=2048.0, corrupt_trim_fraction=0.2, noise_low_freq_cutoff=15.0, max_download_retries=15, retry_delay=5.0, proxy_reset_every=50, proxy_reset_sleep=90.0, mini_segment_duration=512.0, split_minimum_duration=16.0, prune_minimum_duration=22.0, rm_window_length=30.0)[source]

Discover and re-download all missing or failed segments for one detector, appending results to the existing .bin and segments.json files.

Missing segments are located in one of two ways:

  1. If data_{det}_{run}_failed_segments.json exists in data_dir, those GPS ranges are used directly (fast path — no GWOSC query).

  2. Otherwise the GWOSC timeline is reconstructed via find_missing_segments() and diffed against segments.json.

On completion, segments.json is updated in-place and a *_retry_failed.json is written for any segments that still could not be fetched after max_download_retries attempts.

Parameters:

  • detector (str) – Detector prefix, e.g. "H1".

  • run (str) – Observing run label, e.g. "O3a" or "O3b".

  • data_dir (str or Path) – Directory containing data_{det}_{run}.bin and data_{det}_{run}_segments.json.

  • num_workers (int) – Number of parallel download threads (default 8).

  • sample_rate (float) – Target sample rate in Hz (default 2048).

  • corrupt_trim_fraction (float)

  • noise_low_freq_cutoff (float)

  • max_download_retries (int)

  • retry_delay (float) – Seconds to wait between retry attempts.

  • proxy_reset_every (int)

  • proxy_reset_sleep (float)

  • mini_segment_duration (float)

  • split_minimum_duration (float)

  • prune_minimum_duration (float)

  • rm_window_length (float)