--- type: reference title: "Pokémon TCG AI Battle — Temporal Learning and Data Contract" description: "Trajectory-preserving data model, autoregressive action semantics and minimal recurrent register design." tags: [pokemon-tcg, temporal-learning, tbptt, recurrence, multi-select, data-quality] timestamp: "2026-08-15T16:31:30-03:00" --- # Pokémon TCG AI Battle — Temporal Learning and Data Contract ## Current source reconciliation This page preserves the original temporal-contract analysis. The current source has crossed two boundaries that were still described as pending here: `agent/main.py` now performs autoregressive multi-select, and its tracker state carries `memory_out` into the next decision. The exact current trainer/inference contract is [[pokemon_tcg_current_state_reconciliation]]. ## Historical loss of trajectory The replay builder was temporally aware while it read a match, but the original saved corpus and trainer treated emitted rows as independent samples. This was corrected in [[pokemon_tcg_training_overhaul_2026_07_29]]. The missing boundary metadata is the difference between a temporal dataset and a bag of observations. The minimum sidecar schema is: ```text episode_id side step_id decision_id substep new_episode/reset terminal reward ``` The sequence key is `(episode_id, side)`. Validation splits must happen at episode level, never at a raw row suffix. Daily ingestion must deduplicate episodes and retain source date, deck and submission metadata when available. ## Autoregressive multi-select Compound action decisions are already represented as substeps in the data: \[ \pi(a_1\mid s), \quad \pi(a_2\mid s,a_1), \quad \pi(\mathrm{SUBMIT}\mid s,a_{ learned initial scratch state decision t -> model(state_t, memory_t) decision t -> return scratch output as memory_{t+1} next action -> model(state_{t+1}, memory_{t+1}) match end -> reset memory ``` Memory is isolated by match and side. Concurrent matches must not share register state. Multi-select rows from one engine decision read the same incoming memory, and only the final substep commits memory to the next decision. ## TBPTT contract Build ordered chunks for each `(episode_id, side)` with length measured in engine decisions. Historical smoke runs used 32 decisions per chunk. Current configurations vary by run and must be read from the checkpoint/run manifest. Carry memory across chunks but stop gradients at chunk boundaries: \[ (\ell_t,J_{t+1})=F_\theta(x_t,J_t), \qquad J_{\mathrm{next}}=\operatorname{stop\_gradient}(J_{\mathrm{out}}). \] Mask padded timesteps and normalize the loss by real decisions. Gradient accumulation operates over chunks. The order inside an episode is never shuffled; the order of episodes or lanes may be shuffled. ## Counterfactual sufficiency tests The first purpose of counterfactual pairs is diagnosis, not data augmentation. Find histories with similar local observations but different relevant histories: ```text same local board, different revealed opponent cards same board, different resources already consumed same visible Alakazam state, different prior search path same counts, different hidden/deck posterior ``` The current autoregressive scorer can still collapse these cases if the external tracker emits the same representation at each substep. The minimal recurrence should be tested for: \[ J_t^{(1)}\neq J_t^{(2)} \] when the histories should produce different strategic continuations. A memory difference alone is not enough; it must improve action selection or long-horizon behavior without introducing leakage. ## Data hygiene The encoder already performs numerical normalization for many fields, including HP, energy, counts, prizes, flags and masks. This is different from corpus curation. Daily replay ingestion needs: - episode deduplication; - provenance date and source; - label legality checks against action masks; - NaN/Inf and range checks; - episode-level train/validation splits; - explicit deck and matchup distribution tracking; - rare-action and rare-matchup retention; - a mixture of recent and historical data rather than blind overwrite. A useful manifest entry contains `episode_id`, source day, submission IDs when available, both deck hashes, result, rating context, action family and policy/version provenance. This enables later Elo-oriented evaluation without confusing duplicated rows for independent evidence. ## Tests ### Dataset - every label is legal under its action mask; - episode and step metadata are monotone and consistent; - no chunk crosses an episode boundary; - reset and terminal flags occur at exact boundaries; - validation shares no episode with training; - multi-select substeps preserve order. ### Multi-select - selected options disappear from the mask; - minimum and maximum counts are enforced; - `SUBMIT` is accepted only when legal; - the number of forward passes follows the number of substeps. ### Recurrence - initial memory is deterministic and resettable; - memory persists between decisions; - side and match memories are isolated; - `stop_gradient` appears only at TBPTT boundaries; - a new match never inherits old memory. ## Related pages - [[pokemon_tcg_agent_architecture]] — why the current policy loses temporal state. - [[pokemon_tcg_mlx_migration]] — where the sequence work enters the MLX plan. - [[pokemon_tcg_tbptt_training_contract]] — exact decision-chunk, row-budget, accumulation, scheduler, checkpoint and progress accounting contract. - [[pokemon_tcg_training_overhaul_2026_07_29]] — implemented recurrence and real smoke evidence. - [[pokemon_tcg_would_ko_prospective_search]] — causal past versus prospective branches. - [[pokemon_tcg_ladder_and_research]] — how temporal evidence feeds Elo-oriented analysis.