WikifitaGitHub live67e8de5
pesquisa · kaggle/pokemon_tcg_temporal_learning

Pokémon TCG AI Battle — Temporal Learning and Data Contract

Trajectory-preserving data model, autoregressive action semantics and minimal recurrent register design.

Baixar raw

Pokémon TCG AI Battle — Temporal Learning and Data Contract

Current source reconciliation

This page preserves the original temporal-contract analysis. The current source has crossed two boundaries that were still described as pending here: agent/main.py now performs autoregressive multi-select, and its tracker state carries memory_out into the next decision. The exact current trainer/inference contract is pokemon_tcg_current_state_reconciliation.

Historical loss of trajectory

The replay builder was temporally aware while it read a match, but the original saved corpus and trainer treated emitted rows as independent samples. This was corrected in pokemon_tcg_training_overhaul_2026_07_29.

The missing boundary metadata is the difference between a temporal dataset and a bag of observations. The minimum sidecar schema is:

episode_id
side
step_id
decision_id
substep
new_episode/reset
terminal
reward

The sequence key is (episode_id, side). Validation splits must happen at episode level, never at a raw row suffix. Daily ingestion must deduplicate episodes and retain source date, deck and submission metadata when available.

Autoregressive multi-select

Compound action decisions are already represented as substeps in the data:

[ \pi(a_1\mid s), \quad \pi(a_2\mid s,a_1), \quad \pi(\mathrm{SUBMIT}\mid s,a_{<k}). ]

The earlier topk(count) implementation collapsed this into one pass. The current agent/main.py recomputes the encoded observation and policy after each selected option, updates the picked feature and action mask, prevents duplicate selection and stops on legal submission conditions.

This is a semantic correction independent of future Mamba, world-model or reinforcement-learning work.

Minimal persistent registers

The existing scratch tokens provide a ready-made memory interface. No new recurrent cell is required for the first experiment.

match start -> learned initial scratch state
decision t  -> model(state_t, memory_t)
decision t  -> return scratch output as memory_{t+1}
next action -> model(state_{t+1}, memory_{t+1})
match end   -> reset memory

Memory is isolated by match and side. Concurrent matches must not share register state. Multi-select rows from one engine decision read the same incoming memory, and only the final substep commits memory to the next decision.

TBPTT contract

Build ordered chunks for each (episode_id, side) with length measured in engine decisions. Historical smoke runs used 32 decisions per chunk. Current configurations vary by run and must be read from the checkpoint/run manifest. Carry memory across chunks but stop gradients at chunk boundaries:

[ (\ell_t,J_{t+1})=F_\theta(x_t,J_t), \qquad J_{\mathrm{next}}=\operatorname{stop_gradient}(J_{\mathrm{out}}). ]

Mask padded timesteps and normalize the loss by real decisions. Gradient accumulation operates over chunks. The order inside an episode is never shuffled; the order of episodes or lanes may be shuffled.

Counterfactual sufficiency tests

The first purpose of counterfactual pairs is diagnosis, not data augmentation. Find histories with similar local observations but different relevant histories:

same local board, different revealed opponent cards
same board, different resources already consumed
same visible Alakazam state, different prior search path
same counts, different hidden/deck posterior

The current autoregressive scorer can still collapse these cases if the external tracker emits the same representation at each substep. The minimal recurrence should be tested for:

[ J_t^{(1)}\neq J_t^{(2)} ]

when the histories should produce different strategic continuations. A memory difference alone is not enough; it must improve action selection or long-horizon behavior without introducing leakage.

Data hygiene

The encoder already performs numerical normalization for many fields, including HP, energy, counts, prizes, flags and masks. This is different from corpus curation.

Daily replay ingestion needs:

  • episode deduplication;
  • provenance date and source;
  • label legality checks against action masks;
  • NaN/Inf and range checks;
  • episode-level train/validation splits;
  • explicit deck and matchup distribution tracking;
  • rare-action and rare-matchup retention;
  • a mixture of recent and historical data rather than blind overwrite.

A useful manifest entry contains episode_id, source day, submission IDs when available, both deck hashes, result, rating context, action family and policy/version provenance. This enables later Elo-oriented evaluation without confusing duplicated rows for independent evidence.

Tests

Dataset

  • every label is legal under its action mask;
  • episode and step metadata are monotone and consistent;
  • no chunk crosses an episode boundary;
  • reset and terminal flags occur at exact boundaries;
  • validation shares no episode with training;
  • multi-select substeps preserve order.

Multi-select

  • selected options disappear from the mask;
  • minimum and maximum counts are enforced;
  • SUBMIT is accepted only when legal;
  • the number of forward passes follows the number of substeps.

Recurrence

  • initial memory is deterministic and resettable;
  • memory persists between decisions;
  • side and match memories are isolated;
  • stop_gradient appears only at TBPTT boundaries;
  • a new match never inherits old memory.

Related pages