Pokémon TCG AI Battle - 2026-08-14 Ablations and Tournament Ledger

Why the cohorts stay separate

The repository contains several evaluation cohorts with different checkpoints, opponent sets, sweep settings and game counts. They must not be collapsed into one ranking. This page records the numbers and their denominators so later ETL work can join them safely.

FP32 Stage 1-4 matrix

SQLite tournament records 119-122 are the comparable 420-game matrix for the four FP32 curriculum packages. Each row-model used three local decks and faced two baselines plus five first_sub_kaggle_2707 deck labels, with 20 games per matchup.

DB tournament	Package	Wins	Losses	Win rate
119	`stage1_fp32.tar.gz`	60	360	14.29%
120	`stage2_fp32.tar.gz`	64	356	15.24%
121	`stage3_fp32.tar.gz`	58	362	13.81%
122	`stage4_fp32.tar.gz`	72	348	17.14%

Within tournament 122, the deck choice changes the result materially. The Yan vehicle, identified in the matchup labels as Deck #633, recorded 39 wins and 101 losses, or 27.9%, across its 140 games. The starter/default vehicle identified as Deck #251 recorded 18 wins and 122 losses, or 12.9%, across its 140 games. The 27.9% figure is therefore a deck-conditioned Stage 4 result, not the Stage 4 overall rate.

This is a useful saliency observation. It is not a causal proof that the deck alone explains the difference, because the tournament also contains matchup and random-seed variation.

Earlier full round-robin and benchmark records

The same SQLite ledger contains earlier cohorts:

DB IDs	Cohort	Observed results
98-102	13-game full round-robin over curriculum stages and teacher	Stage 1 28.47%, Stage 2 28.93%, Stage 3 epoch 31 30.42%, Stage 3 epoch 32 30.20%, teacher 67.16%
105-109	20-game multi-agent benchmark	Stage 1 14.17%, Stage 2 14.31%, Stage 3 14.03%, Stage 4 14.72%, teacher 43.89%
119-122	420-game FP32 Stage 1-4 matrix	Stage 4 is the best overall at 17.14%
123-128	800-game and later teacher sweeps	Stage 1 12.52%, Stage 2 12.38%, Stage 3 12.75%, Stage 4 12.50%, teacher 51.00% and 64.35% in the latest 1,760-game row

The 13-game and 20-game groups are not interchangeable with the 420-game group. They use different opponent and deck configurations and should be retained as separate observations.

BC curriculum suite artifacts

The August 6-7 suite remains the 10-config experiment over (1d, 3d, 5d) × (1 epoch, 10 epochs) × (top-Elo OFF, ON). The existing pokemon_tcg_bc_curriculum_ablation page records the per-run public-agent and intra-suite peer results. The live checkpoint logs add a validation ledger:

Run	Kept rows	Parameters	Best validation accuracy
`1d_10ep_OFF`	30,031	1,302,151	56.16%
`1d_10ep_ON`	30,024	1,302,151	58.03%
`3d_1ep_OFF`	90,232	1,302,151	53.20%
`3d_1ep_ON`	90,145	1,302,151	51.34%
`3d_10ep_OFF`	90,232	1,302,151	55.88%
`3d_10ep_ON`	90,145	1,302,151	56.27%
`5d_1ep_OFF`	150,425	1,302,151	56.72%
`5d_1ep_ON`	150,409	1,302,151	52.65%
`5d_10ep_OFF`	150,425	1,302,151	60.70%
`5d_10ep_ON`	150,409	1,302,151	57.37%

All ten logs report TBPTT chunks of 16 and the split Muon/AdamW optimizer. Their training-log validation figures are not a replacement for tournament win rate. The current source tree also contains the corresponding checkpoints and TensorBoard event directories, so these values are artifact-backed, not only conversation claims.

What the ablations support

The Stage 1-4 FP32 matrix supports Stage 4 as the best overall member of that 420-game cohort, but the absolute rate remains below the August 14 target.
Deck choice is a major observed axis: the same Stage 4 weights produce 27.9% on the Yan vehicle and 12.9% on the starter/default vehicle in the same tournament record.
The 10-config suite supports a data and regime sensitivity story. The best validation result is 5d_10ep_OFF at 60.70%, while the public-agent tournament result recorded in the older ablation page is 21.0%.
Validation accuracy, public-agent win rate and peer round-robin win rate are different observables. None alone proves a causal architectural improvement.
The current research docs therefore keep the RoPE-ND and MoE phases as later experiments rather than claiming that they already produced these results.

Database persistence and reset uncertainty

The current tournaments table contains 128 rows with IDs 1 through 128, including the Stage 1-4 and teacher cohorts above. This demonstrates persistence of tournament summaries in the current database snapshot. It does not prove that every local replay from every earlier tournament was retained in matches: the current matches table contains 1,760 local rows associated with the latest local observation set, while earlier tournament summaries are not all linked to local replay rows.

Therefore the correct statement is: the current database preserves a tournament ledger, but the available snapshot does not establish whether the underlying match/replay tables were reset, partially rebuilt or selectively retained between tournament runs. No result here depends on assuming one of those histories.

Stage 3 training incident

The ablation table must be read together with pokemon_tcg_stage3_training_failure_postmortem. Stage 3's in-pool validation was not a deployment-aligned guardrail: the run labeled as a one-day top-100 curriculum loaded two dates and split 255 validation episodes from the same filtered pool. Its validation accuracy reached roughly 59.4–59.7%, while the comparable 420-game tournament cohort recorded 13.81%.

The trainer history also changed sparse auxiliary-loss normalization between a942373 and 0dec40f. Stage 4 logs exposed training auxiliary values around 1,000 beside validation values around 0.2–1.5. This makes the Stage 4 tournament improvement an observation under a changed objective and FP32/runtime context, not a clean isolated architecture ablation. The later pokemon_tcg_ropend_moe_blueprint should inherit this denominator and split discipline before adding more heads or synthetic vehicle data.

Cross-references

pokemon_tcg_bc_curriculum_ablation - full 10-config public and peer tables
pokemon_tcg_aug14_data_etl_database_audit - current SQLite and Parquet evidence
pokemon_tcg_aug14_architecture_and_handoff_audit - implementation versus blueprint status
pokemon_tcg_evaluation_results - evaluation provenance rules
pokemon_tcg_tournament_system - tournament CLI and source-separated sweep behavior
pokemon_tcg_deck_strategy - local deck-selection boundary