Aggregated calibration track record across every claim_type in the forecast ledger. Raw forecast Brier vs calibrated-output Brier, per-claim-type cohort breakdown, 10×10 probability-bucket grid, and the miss list (most-recent wrong calls). Sample sizes are disclosed everywhere; insufficient cohorts (N < 30) are flagged as preliminary and never publish a synthetic Brier number.
Brier of the served probability over resolved live ledger claims only; the simulated backfill is never read. Skill is judged against an out-of-sample climatology: for each claim, the up-rate of same-horizon claims whose outcomes were known before it was made. BSS > 0 means better than that base rate. ECE uses 10 probability bins. Level bias is the mean P(up) minus the realized up-rate. Calendar-rule claims (asof + N calendar days, before 2026-09-28) and session-rule claims (the N-th NYSE session, from 2026-09-28) are shown separately.
Loading live calibration…
| Class · horizon | N resolved | Brier | Climatology Brier (OOS) | BSS vs OOS climatology | ECE (10 bins) | Level bias | Calendar-rule claims | Session-rule claims |
|---|
Raw forecast Brier is the model output before calibration patches. Calibrated Brier is the post-calibration-applier output (when CalibrationApplier is wired). Honest split — we never collapse these into a single headline number.
One row per claim_type in the ledger. N resolved = denominator (sample_size). Insufficient cohorts (N<30) display "calibrating" instead of a Brier number — synthetic-Brier suppression is permanent until N crosses the threshold.
| claim_type | N resolved | N pending | empirical hit rate | Brier | status | last update |
|---|---|---|---|---|---|---|
For each (claim_type, 10-pt probability bucket) cell: empirical hit rate, sample size N, status. Cells with N<30 show "N/30" progress and suppress synthetic Brier values. Color: green = sufficient, amber = preliminary, grey = no_resolved.
Top 10 most-recently-resolved claims where the model was wrong (realized_outcome ≠ predicted_outcome). This is the missed_prediction drilldown — click a row for full claim provenance, inputs, and methodology hash.
Each horizon scored independently — never aggregated. Q10 Option J floor ratchets with N_eff: 0.22 → 0.20 → 0.18 → 0.16 → 0.14 at N_eff = 30 / 100 / 300 / 1000 / 3000. Cells with N_eff < 30 show "calibrating" instead of a number. Cohort: trading_quant (Q6 lock).
simulated backfill, not live: the hold-out Brier figures in this panel come from the d18 simulated backfill. Live calibration (resolved live claims) →
Brier = REL − RES + UNC (Murphy 1973). A Brier number alone confounds a fixable term with an unfixable one:
RELiability is what recalibration (Platt / isotonic) fixes; RESolution is separation the ranking either
has or hasn't — any monotone recalibration preserves forecast ordering, so it cannot manufacture separation. Bars are shares of
UNCertainty (the score of always predicting the base rate; a perfect forecast has RES = UNC). Slices with BSS < 0
say so in words — a bar that reads as partial success would be dishonest there.
Per-cohort Platt / isotonic calibrators from /api/calibration/scoreboard. Only cohorts the endpoint tags with live provenance are summarised; cohorts fitted on the simulated backfill are counted, never shown as numbers.
Each time you record a view on the cockpit's decision card (Up / Down / No edge), it is matched to the model's ledger claim for that ticker, horizon and date. When the claim resolves, the same outcome scores both: Brier for your view (a hard call counts as 1, 0 or 0.5) and Brier for the model's recorded P(up). Views are accepted only on the forecast date and never after the claim resolves. No comparison is drawn below 30 resolved views.
Every published decision carries a predicted hit rate. As outcomes resolve, a Beta-Binomial posterior (α = 1 + hits, β = 1 + misses) accumulates per tier × horizon cohort. A cohort is calibrated when its predicted rate sits inside the posterior's 95% credible interval; outside it, Kelly sizing on that cohort should freeze until the gap closes. The predicted column is the tier-aware hit rate (Φ of predicted drift over horizon sd); hover it to see which probe a row uses.
| Tier | Horizon | n_obs | α / β | μ̂ (posterior) | 95% CI | Predicted | Δ | Calibrated? |
|---|---|---|---|---|---|---|---|---|
| Loading… | ||||||||
state hash — · api hash — · last cron run — · source /api/d82-calibration (table thv2_d82_posterior_state)