sage.benchmark.mlgwsc1.evaluator

Functions

find_injection_times(fgfiles, injfile[, ...])

Determine injections which are contained in the file.

find_closest_index(array, value[, assume_sorted])

Find the index of the closest element in the array for the given

mchirp(mass1, mass2)

Return chirp mass (M☉) from component masses mass1 and mass2.

figure([title, size_x, size_y])

Create a single-panel matplotlib figure with serif font and a centred title.

param_vs_param(output_dir, injparams, found_injections)

Plotting param vs param plots similar to MLGWSC-1 paper

found_param_plots(noise_stats, output_dir, injparams, ...)

Generate parameter histograms and scatter plots for found injections at each FAR threshold.

network_output(found_injections, noise_stats, ...[, ...])

Plot overlaid histograms of network scores for found injections and noise triggers.

parameter_learning(injparams, noise_stats, ...)

Plot network output score versus each source parameter for found injections.

read_data(args, idxs)

Load injection parameters and foreground/background events for both teams.

compare_plot_1(team_1, team_2, save_dir)

Plot overlaid injection-parameter histograms for both pipelines at each FAR threshold.

compare_plot_2(team_1, team_2, save_dir)

Scatter param-vs-param plots colour-coded by unique and shared detections.

compare_plot_3(team_1, team_2, save_dir)

Colour-strip plots showing the per-bin ratio of unique detections (team_1 / team_2).

compare_plot_4(team_1, team_2, save_dir)

Plot SNR-binned efficiency curves (True Alarm Probability vs optimal SNR).

compare_groups(team_1, team_2, output_dir)

Comparing the found injections by different (any 2) groups

get_stats(args, idxs[, duration, output_dir, snrs])

Calculate the false-alarm rate and sensitivity of a search

main([raw_args, cfg_far_scaling_factor, dataset])

CLI entry point for the MLGWSC-1 testing-phase evaluator.

Module Contents

find_injection_times(fgfiles, injfile, padding_start=0, padding_end=0)[source]

Determine injections which are contained in the file.

Parameters:

  • fgfiles (list of str) – Paths to the files containing the foreground data (noise + injections).

  • injfile (str) – Path to the file containing information on the injections in the foreground files.

  • padding_start ({float, 0}) – The amount of time (in seconds) at the start of each segment where no injections are present.

  • padding_end ({float, 0}) – The amount of time (in seconds) at the end of each segment where no injections are present.

Returns:

  • duration – A float representing the total duration (in seconds) of all foreground files.

  • bool-indices – A 1D array containing bools that specify which injections are contained in the provided foreground files.

find_closest_index(array, value, assume_sorted=False)[source]

Find the index of the closest element in the array for the given value(s).

Parameters:

  • array (np.array) – 1D numpy array.

  • value (number or np.array) – The value(s) of which the closest array element should be found.

  • assume_sorted ({bool, False}) – Assume that the array is sorted. May improve evaluation speed.

Returns:

Array of indices. The length is determined by the length of value. Each index specifies the element in array that is closest to the value at the same position.

Return type:

indices

mchirp(mass1, mass2)[source]

Return chirp mass (M☉) from component masses mass1 and mass2.

figure(title='', size_x=16.0, size_y=14.0)[source]

Create a single-panel matplotlib figure with serif font and a centred title.

Parameters:

  • title (str, optional) – Figure super-title.

  • size_x (float, optional) – Figure width in inches (default 16.0).

  • size_y (float, optional) – Figure height in inches (default 14.0).

Returns:

  • axs (matplotlib.axes.Axes)

  • fig (matplotlib.figure.Figure)

param_vs_param(output_dir, injparams, found_injections)[source]

Plotting param vs param plots similar to MLGWSC-1 paper

found_param_plots(noise_stats, output_dir, injparams, found_injections)[source]

Generate parameter histograms and scatter plots for found injections at each FAR threshold.

Saves per-parameter injection histograms under output_dir/FOUND_INJECTIONS/ and param-vs-param scatter plots for high-SNR bad-FAR events under output_dir/HIGH_SNR_BAD/.

Parameters:

  • noise_stats (numpy.ndarray, shape (N_bg,)) – Sorted background network scores used to derive FAR thresholds.

  • output_dir (str) – Root output directory.

  • injparams (dict) – Injection parameter arrays (masses, distance, snr, …).

  • found_injections (numpy.ndarray, shape (2, N_found)) – Row 0: injection indices; row 1: network scores for found injections.

network_output(found_injections, noise_stats, output_dir, team_name, lower_threshold=0.0)[source]

Plot overlaid histograms of network scores for found injections and noise triggers.

Parameters:

  • found_injections (numpy.ndarray, shape (2, N_found)) – Row 0: injection indices; row 1: network scores for found injections.

  • noise_stats (numpy.ndarray, shape (N_bg,)) – Network scores from background (noise-only) triggers.

  • output_dir (str) – Directory where the PNG is saved.

  • team_name (str) – Used to construct the output filename.

  • lower_threshold (float, optional) – Minimum score for display; scores below are excluded (default 0.0).

parameter_learning(injparams, noise_stats, found_injections, output_dir)[source]

Plot network output score versus each source parameter for found injections.

Scatter plots of (parameter value, network score) are saved under output_dir/LEARNING/, with FAR threshold lines overlaid.

Parameters:

  • injparams (dict) – Injection parameter arrays keyed by parameter name.

  • noise_stats (numpy.ndarray, shape (N_bg,)) – Sorted background network scores used to derive FAR thresholds.

  • found_injections (numpy.ndarray, shape (2, N_found)) – Row 0: injection indices; row 1: network scores.

  • output_dir (str) – Root directory for output plots.

read_data(args, idxs)[source]

Load injection parameters and foreground/background events for both teams.

Parameters:

  • args (argparse.Namespace) – Parsed CLI arguments; must contain injection_file, foreground_events, background_events, team1, team2, and orchid_results.

  • idxs (numpy.ndarray of bool) – Boolean mask selecting which injections lie inside the analysed segments.

Returns:

  • team_1 (dict) – Event arrays and metadata for the first team (Sage).

  • team_2 (dict) – Event arrays and metadata for the second team (e.g. PyCBC).

  • injparams (dict) – Injection parameters (masses, distance, tc, …) filtered by idxs.

  • use_chirp_distance (bool) – True if chirp-distance weighting should be used for sensitivity.

compare_plot_1(team_1, team_2, save_dir)[source]

Plot overlaid injection-parameter histograms for both pipelines at each FAR threshold.

Saves one multi-panel PNG per FAR threshold under save_dir.

Parameters:

  • team_1 (dict) – Must contain params (list of parameter names), found_idx, found_stats, far_thresholds, and per-parameter arrays.

  • team_2 (dict) – Same structure as team_1 for the comparison pipeline.

  • save_dir (str) – Directory where plots are saved (created if absent).

compare_plot_2(team_1, team_2, save_dir)[source]

Scatter param-vs-param plots colour-coded by unique and shared detections.

Blue = unique to team_1, red = unique to team_2, grey = found by both. One multi-panel PNG is saved per FAR threshold.

Parameters:

  • team_1 (dict) – Pipeline dict with found_idx, found_stats, far_thresholds, and per-parameter arrays (including mchirp for duration calculation).

  • team_2 (dict) – Same structure as team_1.

  • save_dir (str) – Output directory (created if absent).

compare_plot_3(team_1, team_2, save_dir)[source]

Colour-strip plots showing the per-bin ratio of unique detections (team_1 / team_2).

Uses a diverging colormap centred at ratio=1 so bins where team_1 outperforms team_2 appear red and vice-versa blue.

Parameters:

  • team_1 (dict) – Pipeline dict with params, found_idx, found_stats, far_thresholds, and per-parameter arrays.

  • team_2 (dict) – Same structure as team_1.

  • save_dir (str) – Output directory (created if absent).

compare_plot_4(team_1, team_2, save_dir)[source]

Plot SNR-binned efficiency curves (True Alarm Probability vs optimal SNR).

One PNG per FAR threshold, overlaying both pipelines.

Parameters:

  • team_1 (dict) – Pipeline dict with snr, found_idx, found_stats, and far_thresholds.

  • team_2 (dict) – Same structure as team_1.

  • save_dir (str) – Output directory (created if absent).

compare_groups(team_1, team_2, output_dir)[source]

Comparing the found injections by different (any 2) groups In this module we make 3 plots for comparison:

  1. Histogram of all injections with found injections of both pipelines

  2. Scatter plot of param vs param (unique finds from both teams are coloured)

  3. Colour strip plot quantifying np.log10(Nnn/Nmf) found in each bin

  4. Efficiency curves made for each of the two groups

Each of these plots for all params are made for different FAR thresholds

get_stats(args, idxs, duration=None, output_dir=None, snrs=None)[source]

Calculate the false-alarm rate and sensitivity of a search algorithm.

Parameters:

  • fgevents (np.array) – A numpy array with three rows. The first row has to contain the times returned by the search algorithm where it believes to have found a true signal. The second row contains a ranking statistic like quantity for each time. The third row contains the maxmimum distance to an injection for the given event to be counted as true positive. The values have to be determined on the foreground data, i.e. noise plus additive signals.

  • bgevents (np.array) – A numpy array with three rows. The first row has to contain the times returned by the search algorithm where it believes to have found a true signal. The second row contains a ranking statistic like quantity for each time. The third row contains the maxmimum distance to an injection for the given event to be counted as true positive. The values have to be determined on the background data, i.e. pure noise.

  • injparams (dict) – A dictionary containing at least two entries with keys tc and distance. Both entries have to be numpy arrays of the same length. The entry tc contains the times at which injections were made in the foreground. The entry distance contains the according luminosity distances of these injections.

  • duration ({None or float, None}) – The duration of the analyzed background. If None the injections are used to infer the duration.

Returns:

Returns a dictionary, where each key-value pair specifies some statistic. The most important are the keys far and sensitive-distance.

Return type:

dict

main(raw_args=None, cfg_far_scaling_factor=None, dataset=None)[source]

CLI entry point for the MLGWSC-1 testing-phase evaluator.

Parses arguments, loads foreground/background events and injection parameters, computes FAR and sensitive distance via get_stats(), writes results to an HDF5 file, and saves sensitivity-vs-FAR plots comparing Sage against MLGWSC-1 competition teams.

Parameters:

  • raw_args (list of str, optional) – Argument list for programmatic invocation; defaults to sys.argv.

  • cfg_far_scaling_factor (float, optional) – FAR scaling factor from config (overrides --far-scaling-factor CLI arg).

  • dataset (int, optional) – Dataset index (overrides --dataset CLI arg).