sage.plotting.pp_calibration

P-P (probability-integral-transform) calibration plot for the heteroscedastic parameter heads.

For a well-calibrated Gaussian prediction N(mu, sigma) the PIT value z = Phi((y - mu) / sigma) of the true target y is uniformly distributed on [0, 1]. Plotting the empirical CDF of the PIT values against the diagonal therefore reveals miscalibration of the predicted uncertainties:

  • curve on the diagonal -> calibrated

  • curve shallower than diagonal -> over-confident (sigma too small)

  • curve steeper than diagonal -> under-confident (sigma too large)

This validates the sigma mechanism that the multi-detector consistency statistic relies on (it is uncertainty-weighted, so trustworthy sigmas matter), and works equally for the merged heteroscedastic heads. It complements plot_calibration_curve (which calibrates the classifier).

Functions

plot_pp_calibration(mu, sigma, y[, param_names, ...])

P-P calibration plot of heteroscedastic predictions.

Module Contents

plot_pp_calibration(mu, sigma, y, param_names=None, epoch=None, export_dir=None, save=True, title=None)[source]

P-P calibration plot of heteroscedastic predictions.

Parameters:

  • mu (array-like, shape (N,) or (N, P)) – Predicted means, predicted standard deviations (NOT log-variances), and the true targets, for N samples and optionally P parameters. Pass only the supervised samples (e.g. signals); mask out noise first.

  • sigma (array-like, shape (N,) or (N, P)) – Predicted means, predicted standard deviations (NOT log-variances), and the true targets, for N samples and optionally P parameters. Pass only the supervised samples (e.g. signals); mask out noise first.

  • y (array-like, shape (N,) or (N, P)) – Predicted means, predicted standard deviations (NOT log-variances), and the true targets, for N samples and optionally P parameters. Pass only the supervised samples (e.g. signals); mask out noise first.

  • param_names (list[str] or None) – Names for the P parameters (used in the legend).

  • epoch (int or str or None) – Epoch identifier for the title / filename.

  • export_dir (str or None) – If given (and save), writes calibration/pp_calibration_{epoch}.png.

  • save (bool) – Save to disk if True, else show interactively.

  • title (str or None) – Override the default title.

Returns:

Per-parameter calibration metrics: {name: {"ks": float, "cov1sigma": float, "cov2sigma": float}} where ks is the Kolmogorov-Smirnov distance of the PIT from uniform (0 = perfect).

Return type:

dict