Pokémon TCG — Empirical Ablation Monograph

Evidence role

This page preserves the Level 3 empirical monograph as a distinct analytical source. It does not replace the current database ledger or the code-validated runtime contract. The monograph combines a 420-game curriculum matrix, a deck-conditioned “Pilot versus Vehicle” interpretation, a historical FP16 failure analysis and an Abelian correction proposal. Each result family has its own denominator and evidence status.

Cross-stage matrix

The source monograph reports the following FP32 matrix, with 420 games per curriculum stage:

Stage	Training interpretation in source	Wins	Losses	Win rate
Stage 1	Raw BC, all Elo	60	360	14.3%
Stage 2	Elo-filtered BC	64	356	15.2%
Stage 3	Loss-corrupted top-100 pool	58	362	13.8%
Stage 4	Loss-corrected top-100 pool	72	348	17.1%

The direct SQLite tournament ledger identifies these rows as tournaments 119–122 and records the same rates to two decimals. Stage 4 is the best aggregate member of this cohort, but its 17.14% remains below the handoff target. The matrix supports a cohort-specific observation, not a general leaderboard claim.

Validation accuracy and game-theoretic win rate

The monograph's central analytical claim is that behavior-cloning validation accuracy and arena win rate are different observables. Behavioral cloning minimizes a divergence from the replay action distribution:

$$\mathcal{L}_{\mathrm{BC}}(\theta) = D_{\mathrm{KL}}(\pi_{\mathrm{human}} \Vert \pi_{\theta}).$$

Tournament performance instead depends on the trajectory distribution induced by the learned policy against a declared opponent:

$$\mathrm{WR}(\pi_{\theta}, \pi^*) = \mathbb{E}_{\tau \sim (\pi_{\theta},\pi^*)}[\mathbb{I}(R_T=+1)].$$

The practical conclusion is sound as an evaluation rule: a high in-pool accuracy does not establish arena strength. The stronger statement that behavioral cloning must asymptotically reproduce a single “human ceiling” is a model of the problem, not a theorem about every dataset or policy class. Wikifita therefore keeps the monograph's equation and marks causal generalization as a hypothesis.

Pilot versus Vehicle

The source monograph separates the neural policy from the 60-card list it pilots. Under the source's Stage 4 deck analysis, the same weights produced:

Deck	Source label	Result in the source cohort
#633	Yan, fast energy acceleration	27.9%, 39 W / 101 L
#21	Oshbocker, high-HP setup	20.0%, 28 W / 112 L
#251	Submission default	12.9%, 18 W / 122 L

The current tournament reconciliation independently confirms the #633 and #251 140-game slices for tournament 122. The #21 row belongs to the monograph's deck-analysis artifact and is not silently merged into the current database slice. This is exactly the kind of distinction that a future vehicle-draft experiment must preserve.

The interpretation is therefore:

policy quality + vehicle capability + opponent/deck cohort

The deck-conditioned difference motivates the future autoregressive vehicle draft. It does not prove that deck choice alone caused every difference, because pairing, seed and matchup composition remain possible contributors.

Historical FP16 collapse

The monograph records a drop from an approximately 45.0% MLX result to 3.3% in a standalone PyTorch CPU evaluation during an older FP16 migration. It attributes the failure to reduced numerical range in attention and softmax. The linked Antigravity report preserves the same historical narrative.

The current code boundary is independently different: the live MLX trainer and PyTorch converter enforce strict FP32. That verifies the present dtype contract, not every low-level causal detail in the historical report. The safe synthesis is:

• FP16 collapse is a historical empirical incident;
• strict FP32 is the current implementation contract;
• the old failure should not be used as a current benchmark number;
• future mixed-precision work needs a new controlled experiment.

Sampling bias and invariant Elo

The monograph reports that only a small fraction of daily competition matches appeared in the exported replay archives and that registered-team coverage was incomplete. Those percentages are source-reported from the historical sampling analysis. They explain why a raw local Elo can be badly calibrated against the official ladder, but they are not a current database count.

The proposed correction uses an additive translation over an overlapping set of agents:

$$\Delta R_{\mathrm{Abeliano}} = \sum_{k \in \mathcal{C}} w_k\left(R_k^{\mathrm{remote}} - \hat{R}_{k,\infty}^{\mathrm{local}}\right), \qquad w_k = \frac{\exp(N_k/20)}{\sum_{j \in \mathcal{C}}\exp(N_j/20)}.$$

The current local rating code and database provide an empirical surface for this analysis. The PageRank comparison remains a mathematical bridge and research proposal, not a code-validated isomorphism.

What this monograph supports

• Stage 4 was the strongest aggregate checkpoint in the declared 420-game FP32 matrix.
• Deck selection materially changes the observed result for the same checkpoint.
• Validation metrics and tournament outcomes must be reported separately.
• Historical FP16 failures justify the current FP32 boundary but do not establish a universal numerical law.
• Sparse public replay sampling makes rating calibration a provenance and measurement problem, not only an Elo-formula problem.

What it does not support

• It does not prove that the Transformer architecture is fully validated.
• It does not prove that Stage 3's loss incident has one isolated causal mechanism.
• It does not merge the 420-game matrix with the separate 3,600-game tournament analysis or the 10-run curriculum suite.
• It does not establish that RoPEND, MoE or vehicle drafting exists in the current source tree.
• It does not turn the PageRank/Abelian analogy into an implemented theorem.

Primary source and provenance

• docs/empirical_ablation_monograph.md — source monograph dated August 14, 2026.
• Commit cec2eab — empirical ablation specification added to the handoff wave.
• Commit d9c67a7 — integrated monograph and nine manuscript chapters entered the repository.
• Commit 86aa528 — Stage 3 tournament changes and Abelian Elo formulation entered the project history.
• Commit 459f9d6 — later strict-FP32 and validation-related changes in the architecture lineage.

Cross-references

• pokemon_tcg_aug14_ablations_tournaments — stored tournament rows and denominator-separated cohorts.
• pokemon_tcg_stage3_training_failure_postmortem — training-loss and validation incident.
• pokemon_tcg_data_augmentation_vehicle_draft — future vehicle generation and synthetic-data contract.
• pokemon_tcg_elo_pagerank_invariance — rating and graph-invariance research bridge.
• pokemon_tcg_manuscript_and_empirical_synthesis — nine-chapter narrative map.
• pokemon_tcg_current_state_reconciliation — current code and database boundary.