Accuracy Report | SignalTuned

Backtest generated 2026-06-07T07:08:20.040696 on engine v1.6.111 · 3 most recent reference years (2022, 2023, 2024).

Earned by math. Verified by outcomes.

Every season the engine reruns as a walk-forward backtest — train on year Y, grade the preseason projections against what actually happened in year Y+1. The numbers below are the result. They don't get adjusted toward a target. Outliers don't get excluded. The cohort doesn't get curated.

An honest note on leakage. Engine parameters and rookie baselines are not frozen per reference year, so these columns are next-season-out-of-sample for the predictions but not a fully leakage-free holdout. For a clean out-of-sample read on the rookie model specifically, our leave-one-class-out estimate is MAE ≈ 3.46 (vs 3.27 in-sample — a +0.19 optimism gap), still well below the 4.71 of the prior approach. The 2026 season is reserved as a pre-registered, untouched holdout for prospective scoring.

How to read the table. r is correlation between projection and actual. Closer to 1.0 is better. MAE is mean absolute error in fantasy points per game. Lower is better. bias is the average over/under by position. Positive means the engine projected high. The volume-weighted variants downweight small-sample players, and that's the version that matters for projection trust.

Per-position accuracy

Position	2022	2023	2024
QB	31	0.518	5.40	2.94	34	0.657	4.13	1.86	31	0.660	4.12	2.70
Volume-weighted: 2022: MAE_w 5.09 · bias_w 3.34 · 2023: MAE_w 4.34 · bias_w 1.50 · 2024: MAE_w 3.95 · bias_w 3.20
RB	72	0.680	2.92	1.30	74	0.743	2.92	1.07	72	0.744	2.90	0.04
Volume-weighted: 2022: MAE_w 3.12 · bias_w -0.06 · 2023: MAE_w 2.61 · bias_w -0.44 · 2024: MAE_w 3.18 · bias_w -1.70
WR	126	0.659	2.72	1.43	113	0.737	2.59	1.70	128	0.795	1.97	0.52
Volume-weighted: 2022: MAE_w 2.61 · bias_w 0.33 · 2023: MAE_w 2.26 · bias_w 0.64 · 2024: MAE_w 2.26 · bias_w -0.71
TE	59	0.720	2.25	1.22	64	0.710	2.29	1.10	74	0.800	2.01	0.85
Volume-weighted: 2022: MAE_w 2.13 · bias_w 0.27 · 2023: MAE_w 2.15 · bias_w 0.12 · 2024: MAE_w 2.12 · bias_w -0.45

Known calibration notes

Publishing accuracy receipts means publishing the misses too. These are the calibration gaps we know about, what causes them, and what's planned.

QB projections run hot (volume-weighted bias +3.3 / +1.5 / +3.2 across the reference years above). Two compounding causes: the shrinkage layer pulls young, already-producing QBs toward a conservative positional prior, and veteran pocket passers carry a longevity modifier the outcome data says is too generous. On SuperFlex boards this can also place QBs above where market consensus slots them. The fix — a QB-specific shrinkage skip for demonstrated producers — is built and staged behind a flag, deliberately not live: three earlier QB recalibrations validated on proxy data and failed, so this one ships only after the 2026 season provides a clean held-out cohort and a full-engine backtest clears it.
WOPR regression flags don't apply to consensus-elite receivers. Elite target earners exceed the opportunity model's prediction every year — that's elite target command the model's inputs can't fully see, not a sell signal. Their flag renders as a neutral "WOPR WATCH" instead of a regression warning.

What changed in v1.6.111

Released 2026-06-06.

DYNASTY H3b buried-depth-chart discount SHIPPED (ports the redraft H3b gate; audit P1 'port the three redraft-only gates to dynasty').

SNAPSHOT-GUARDED (returns early when FF_SNAPSHOT_AS_OF_SEASON is set): depth_chart_order is a point-in-time live signal, so it never runs in backtests/walk-forward and the caches are byte-identical content (re-stamped to v1.6.111, not…
VALIDATION (live-board OFF/ON, scripts/validate_buried_depth_dynasty.py; NOT ab_harness-gateable since snapshot-guarded): cohort = 185 deep backups (highest rank 122, Parker Washington), 0 discounted players in the top 60 (no startable…
SCOPE NOTE: the other two 'redraft-only gates' from the backlog were NOT ported — H7 missed-year-0 is already covered (and stronger, 0.50 vs 0.20) by _apply_stale_player_decay; DISC-R4 young-rushing-QB is a QB lever deferred to the Sept…

This page is server-rendered from backtest_results.json and engine_config.json at request time. Backtest harness: scripts/ab_harness.py. Full per-version change log: Model Card. Production engine version: v1.6.111.

Position	2022				2023				2024
Position	n	r	MAE	bias	n	r	MAE	bias	n	r	MAE	bias
QB	31	0.518	5.40	2.94	34	0.657	4.13	1.86	31	0.660	4.12	2.70
Volume-weighted: 2022: MAE_w 5.09 · bias_w 3.34 · 2023: MAE_w 4.34 · bias_w 1.50 · 2024: MAE_w 3.95 · bias_w 3.20
RB	72	0.680	2.92	1.30	74	0.743	2.92	1.07	72	0.744	2.90	0.04
Volume-weighted: 2022: MAE_w 3.12 · bias_w -0.06 · 2023: MAE_w 2.61 · bias_w -0.44 · 2024: MAE_w 3.18 · bias_w -1.70
WR	126	0.659	2.72	1.43	113	0.737	2.59	1.70	128	0.795	1.97	0.52
Volume-weighted: 2022: MAE_w 2.61 · bias_w 0.33 · 2023: MAE_w 2.26 · bias_w 0.64 · 2024: MAE_w 2.26 · bias_w -0.71
TE	59	0.720	2.25	1.22	64	0.710	2.29	1.10	74	0.800	2.01	0.85
Volume-weighted: 2022: MAE_w 2.13 · bias_w 0.27 · 2023: MAE_w 2.15 · bias_w 0.12 · 2024: MAE_w 2.12 · bias_w -0.45