Calibration Scoreboard

Aggregated calibration track record across every claim_type in the forecast ledger. Raw forecast Brier vs calibrated-output Brier, per-claim-type cohort breakdown, 10×10 probability-bucket grid, and the miss list (most-recent wrong calls). Sample sizes are disclosed everywhere; insufficient cohorts (N < 30) are flagged as preliminary and never publish a synthetic Brier number.

Calibrating · the forecast-ledger has no resolved claims yet. Once the lane-16 resolution-scheduler cron fires (daily 08:00 UTC), composite Brier and per-claim-type cohort metrics will populate here. Sample size will always be disclosed.

Raw vs Calibrated Brier preliminary

Raw forecast Brier is the model output before calibration patches. Calibrated Brier is the post-calibration-applier output (when CalibrationApplier is wired). Honest split — we never collapse these into a single headline number.

Raw Brier (uncalibrated forecast)
N=0
sample_size=0 · awaiting first resolution batch
Calibrated Brier (post-applier)
calibrated: pending
calibrator wires post N≥100 · isotonic patches refresh weekly

Per-Claim-Type Cohort Breakdown

One row per claim_type in the ledger. N resolved = denominator (sample_size). Insufficient cohorts (N<30) display "calibrating" instead of a Brier number — synthetic-Brier suppression is permanent until N crosses the threshold.

claim_type N resolved N pending empirical hit rate Brier status last update

Probability Bucket Grid (claim_type × probability_bucket)

For each (claim_type, 10-pt probability bucket) cell: empirical hit rate, sample size N, status. Cells with N<30 show "N/30" progress and suppress synthetic Brier values. Color: green = sufficient, amber = preliminary, grey = no_resolved.

Miss List most recent wrong_call

Top 10 most-recently-resolved claims where the model was wrong (realized_outcome ≠ predicted_outcome). This is the missed_prediction drilldown — click a row for full claim provenance, inputs, and methodology hash.

resolved_at claim_id claim_type predicted actual abs_error methodology_hash
No missed_prediction rows yet — ledger has no resolved claims, so the miss_list is empty by definition. The wrong_call drilldown will populate after the resolve-forecasts cron fires.

Per-Horizon Brier (5-horizon set)

Each horizon scored independently — never aggregated. Q10 Option J floor ratchets with N_eff: 0.22 → 0.20 → 0.18 → 0.16 → 0.14 at N_eff = 30 / 100 / 300 / 1000 / 3000. Cells with N_eff < 30 show "calibrating" instead of a number. Cohort: trading_quant (Q6 lock).

Preliminary · synthetic Brier suppressed. Cohorts with N<30 are flagged as insufficient; we never publish a synthetic or preliminary number that could mislead. When sample size crosses the threshold (n < 30 → n ≥ 30), the cell switches from "calibrating · N/30" to a real numeric Brier value with the denominator disclosed. Methodology hashes are propagated honestly — when a cohort spans multiple methodologies, the row shows mixed:<count> rather than silently picking one.

Verifiability: every numeric Brier on this page is computed live from forecast-ledger.summarize() (lane 16) and CalibrationTrackRecord.getTrackRecord() (lane F). No values are fabricated, cached longer than 5 minutes, or sourced from synthetic baselines.