---
type: reference
title: "Pokémon TCG AI Battle — Current Agent Architecture"
description: "Evidence-based description of the entity/action Transformer running today: sizes, token schema, embedding zones, aux heads, scratch registers, positional strategy (no RoPE/RoPE-ND), meta-buckets, split heads."
tags: [pokemon-tcg, transformer, entity-action, action-masking, belief-state, behavioral-cloning, aux-heads, scratch-registers, meta-buckets]
timestamp: "2026-08-15T16:31:30-03:00"
---

# Pokémon TCG AI Battle — Current Agent Architecture

## August 14 boundary update

This page still describes the live entity/action Transformer. The August 14 handoff adds a documented future architecture, not a silent replacement of this one. A direct source-tree check found no tracked `rl/ropend`, `rl/moe`, `rl/policy_moe_torch.py`, `rl/policy_moe_mlx.py` or `rl/deck/vehicle_draft.py` implementation. The corresponding RoPE-ND, four-expert MoE, vehicle-draft and Apex documents remain preserved as planned or experimental work in [[pokemon_tcg_aug14_architecture_and_handoff_audit]].

The current source-backed model still uses the existing scratch and auxiliary-head path. The current FP32 inference contract and the August 14 database, ablation and tournament evidence are cataloged in [[pokemon_tcg_aug14_data_etl_database_audit]] and [[pokemon_tcg_aug14_ablations_tournaments]].

## Architecture identity

Entity/action Transformer trained by behavioral cloning over a structured, externally reconstructed belief state. Not a causal language decoder. It exposes a recurrent memory interface through scratch registers and TBPTT, but it has no dedicated recurrent cell. The exact register count belongs to the run configuration: the default is 16 and the session configuration used 32.

Current build (config default → session baseline in `configs/train_config.json`):

```text
d_model              128
attention heads      4
Transformer layers   3        (session runs 4)
FFN width            512      (4 × d_model)
scratch registers    16       (session runs 32)
static card features enabled
split policy/value   enabled  (dedicated VALUE_TOK and SUBMIT_TOK heads)
structured head      disabled for current BC baseline
options              up to MAX_OPTIONS=192 plus SUBMIT_ACTION=192
total trainable      ~1.30M params  (~845k Muon-routed + ~456k AdamW-routed)
```

Positional information does **not** use RoPE, sinusoidal or RoPE-ND. Positions are carried by zone-typed token embeddings (`type_emb`, `sel_type_emb`, `sel_ctx_emb`) plus explicit gather from `opt_src_pos` / `opt_tgt_pos` — each option token points at the exact state token of the card/unit it references, so the option's src/tgt embedding IS that state token, not a rebuilt copy. RoPE-ND is deferred backlog, not shipped ([[pokemon_tcg_ladder_and_research]]).

The token sequence has approximately 337 positions before options:

```text
[CLS] [SELECT_TYPE] [SELECT_CONTEXT]
[SELF_DECK ×60] [OPP_DECK ×60]
[SELF_PRIZE ×6] [OPP_PRIZE ×6]
[SELF_HAND ×30] [OPP_HAND ×30]
[SELF_DISCARD ×60] [OPP_DISCARD ×60]
[STADIUM ×2] [EFFECT ×2]
[SELF_UNITS ×(1+8)] [OPP_UNITS ×(1+8)]
[SCRATCH ×N_SCRATCH]
[OPTIONS ×(up to 192)]
```

Padding capacity is part of the storage format; not every position holds a real entity. `padding_idx=0` semantics apply — card and attack ID zero embed to a zero vector, not a learned "absence" embedding. The option surface holds up to 192 tokens plus one reserved `SUBMIT_ACTION` index at position 192.

## What the model actually receives

The encoder combines several kinds of information:

| Input | Role |
|---|---|
| Card and attack IDs | Categorical identity embeddings |
| Static card features | Deterministic domain attributes projected into model space |
| Unit attributes | Dynamic HP, energy, effects, status and other board state |
| Zone/type embeddings | Ownership and semantic role of deck, hand, discard, active, bench, stadium and effects |
| Selection context | Current action phase and selection type |
| Source/target positions | Explicit references from an option to state tokens |
| Action mask | Exact legality boundary supplied by the engine/encoder |
| Tracker flags | Information about visibility, drawability and certainty |

The option token is a structured candidate, not a fixed semantic class tied to an array index. It combines source and target representations, structural attributes, verb identity and attack identity. The output head scores those candidates and applies the legal-action mask.

Mathematically, the current policy is close to:

\[
\pi_\theta(a_t\mid\hat b_t),
\qquad
\hat b_t=\mathcal T(o_t,\text{logs}_t,H_{t-1}),
\]

where `GameTracker` and `AbilityTracker` construct the working belief state before the Transformer is called.

## External state is a strength and a boundary

Keeping the exact rules transition outside the neural network is appropriate. The engine should remain the sound authority for legal actions and state transitions:

\[
s_{t+1}=P(s_t,a_t),
\qquad
m(s_t,a_t)\in\{0,1\}.
\]

The current limitation is that the external system also supplies most of the agent's temporal belief compression. If two histories produce the same tracker output, the current policy cannot distinguish them:

\[
\mathcal T(H_1)=\mathcal T(H_2)
\Rightarrow
\pi(\cdot\mid H_1)=\pi(\cdot\mid H_2).
\]

This is not a claim that the tracker is wrong. It is a statement about what
information is available to the neural policy. The current model is therefore
best described as an autoregressive action scorer over each externally
reconstructed belief-state substep, with bounded learned memory rather than a
full internal game-state model.

## Scratch/register tokens

Scratch registers are learned workspace vectors inserted between state and option tokens. In the current implementation they are initialized from learned parameters at match start, then the final substep's output becomes the next decision's `memory_in`:

\[
J_0^{\mathrm{in}}=J_{\mathrm{init}},
\qquad
J_{t+1}^{\mathrm{in}}=J_t^{\mathrm{out}}.
\]

They support intrapass integration and inter-decision memory. `agent/main.py` stores the memory per tracked side, and the inference model carries it through each autoregressive selection. This memory interface does not prove that the learned state is strategically sufficient.

## Action semantics

The dataset expands multi-select actions into sequential substeps:

```text
picked=[]       -> option_1
picked=[1]      -> option_2
picked=[1,2]    -> SUBMIT
```

The earlier live path used one forward pass and `topk(count)`. The current `agent/main.py` path recomputes the encoded observation and logits after each selected option, masks already-picked options, and terminates on legal `SUBMIT_ACTION`. The old `topk(count)` behavior remains a historical defect recorded in the migration pages, not the current action contract.

The corrected factorization is:

\[
\pi(a_1\mid s),
\quad
\pi(a_2\mid s,a_1),
\quad
\ldots,
\quad
\pi(\mathrm{SUBMIT}\mid s,a_{<k}).
\]

Each substep must update `picked`, the option mask and the option tokens, then run the next forward pass.

## Evidence and current ceiling

The repository's local evaluation artifacts show strong performance against the included public opponents and a smaller set of difficult matchups. The recorded tournament results are useful regression evidence, but they are not a causal attribution of which architectural component produced them. In particular, local win rate is not interchangeable with the competition's proprietary ladder rating.

The current performance can be explained without assuming a hidden breakthrough:

1. the rules engine removes illegal actions;
2. the encoder provides typed entities and action references;
3. static card attributes reduce the burden of learning domain facts;
4. replay labels provide a strong behavioral prior;
5. interchangeable actions can be grouped or deduplicated;
6. the model only needs to rank a reduced candidate surface.

The architecture's likely ceiling is temporal rather than purely computational. It preserves a bounded learned memory, but the rules engine and tracker still perform most belief reconstruction, and the train/inference feature contract must remain aligned. The current code passes the observation logs into the tracker:

\[
\hat b_{\mathrm{train}}=\mathcal T(o_t,\mathrm{logs}_t,H_{t-1}),
\qquad
\hat b_{\mathrm{live}}=\mathcal T(o_t,\mathrm{logs}_t,H_{t-1}).
\]

The immediate repair is to pass the complete observation through the same tracker and encoder path used during dataset construction.

## Embeddings by zone

The token vocabulary is grouped into families, each with its own learned embedding table. All embed to `d_model` unless otherwise noted:

| family | table | notes |
|---|---|---|
| card identity | `card_emb: Embedding(vocab+1, d_model)` | padding_idx=0 |
| card static features | `static_proj: Linear(24, d_model)` | fed by `EN_Card_Data.csv` buffer |
| zone type | `type_emb: Embedding(N_TTYPES, d_model)` | which zone this token belongs to (`SELF_DECK`, `OPP_HAND`, ...) |
| select type | `sel_type_emb: Embedding(N_SELECT_TYPES=16, d_model)` | wire `select.type` ordinal |
| select ctx | `sel_ctx_emb: Embedding(N_SELECT_CTX=64, d_model)` | wire `select.context` ordinal |
| unit attributes | `unit_attr_proj: Linear(24, d_model)` | dynamic HP, energy, status, next-turn dmg-reduce buff |
| visibility flags | `drawable_emb / opp_drawable_emb / hand_certain_emb` | scalar mx.array, added when the row's visibility flag is set |
| option verb | `opt_verb_emb: Embedding(17, d_model)` | option-type ordinal (0..16 incl SPECIAL_CONDITION) |
| option src/tgt | `opt_src_proj / opt_tgt_proj: Linear(d_model, d_model)` | gathered from resolved state token, not rebuilt |
| option structural | `opt_attr_proj: Linear(36, d_model)` | 36 float feats/option (would_ko trio, structural, already-picked flag) |
| attack identity | `attack_emb: Embedding(2048, d_model)` | attack-id (padding_idx=0) |
| CLS scalars | `scalar_proj: Linear(19, d_model)` | 13 board/turn + 5 select-dynamics + 1 our-turn offensive buff |
| meta buckets | `meta_bucket_emb / agent_bucket_emb / deck_bucket_emb: Embedding(11, d_model)` | 0..9 Elo decile + UNKNOWN |
| day scalar | `day_proj: Linear(1, d_model)` | normalized competition-day integer |
| meta context base | `meta_ctx_base: mx.array(d_model)` | learned bias mixed into the meta context |
| scratch | `scratch: mx.array(N_SCRATCH, d_model)` + `learned_init: mx.array(N_SCRATCH, d_model)` | see below |

`N_META_BUCKETS = 11` — Elo deciles 0–9 (0 = strongest) plus slot 10 (`UNKNOWN_BUCKET`) for domain-legitimate "not observed on this day yet" cases (newly released cards, first-game opponents, etc.). Cards / agents / decks all share this schema.

## Scratch registers

A small block of learned workspace tokens inserted between the state stream and the option stream:

```text
J_0_in     = learned_init             (seed at match/decision start)
J_{t+1}    = scratch_slice(model_output_t)
memory_in  is the previous chunk's scratch output
memory_out is the current chunk's scratch output
```

`--scratch-registers N` controls the count. Range 0–64. Config default: 16. Current session config: 32.

When TBPTT is on (see [[pokemon_tcg_tbptt_training_contract]]), memory is carried per `(episode_id, side)` lane across chunks, with `stop_gradient` only at the chunk boundary. Reset on `new_episode`, isolated per side. Not shared across matches or processes.

Inspired by "ViTs Need Registers": the scratch tokens are non-semantic workspace positions the model can use as scratchpad without stealing capacity from state or option tokens.

## Split heads (dedicated VALUE and SUBMIT)

`split_heads=true` reserves two token slots at the head of the sequence with dedicated output heads:

- `VALUE_TOK` — feeds the value head (scalar or categorical). Short residual keeps prize-count signal clean.
- `SUBMIT_TOK` — feeds the `submit_head` (single logit that is spliced into the option scoring at index `SUBMIT_ACTION`).

Off (`split_heads=false`): a single CLS token feeds both value and action heads (`2 × d_model` input to the value head instead of `d_model`).

## Auxiliary heads

Four `Linear(d_model, 1)` heads applied to the CLS output (or `VALUE_TOK` when `split_heads=true`):

| head | target column | loss | default weight |
|---|---|---|---|
| `ko_head_aux` | `aux_ko` (int8) | BCE | 0.5 |
| `prize_head_aux` | `aux_prize_delta` (float) | MSE | 0.5 |
| `terminal_head_aux` | `aux_terminal` (int8) | BCE | 0.5 |
| `return_head_aux` | `aux_return` (float) | MSE | 1.0 |

All four masked by `aux_valid` per row. Fixed loss weights via `--aux-*-weight` flags — automatic uncertainty weighting (Kendall & Gal) is deferred backlog.

## Attention mechanics

- **Additive mask** — the padded-key mask is added to attention scores. The shared finite negative sentinel is `-65504.0`; `0` marks valid keys. The current trainer/inference model is strict FP32 even though this sentinel is also finite in FP16.
- **MHA bias** — attention projections carry a bias term (`bias=True`), matching the PyTorch reference. The MLX side instantiates this explicitly to avoid inheriting a different default.
- **FP32 current path** — the trainer sets model parameters and forward activations to FP32, and the PyTorch converter validates FP32 floating tensors. The historical FP16 path remains documented in [[pokemon_tcg_torch_inference]].

## Static card features

`EN_Card_Data.csv` is loaded once as an `np.ndarray` and stored as a buffer (`_card_feat_np`), not a trainable parameter. A learned `Linear(24, d_model)` projects it into model space when `--static true`. The buffer is checkpointed by SHA256 in the payload so a resume with a changed CSV raises.

## Structured verb head (disabled in baseline)

`--structured true` adds per-`OptionType` query and bias embeddings (`type_query: Embedding(17, d_model)`, `type_bias: Embedding(17, 1)`) that gate the shared `opt_head` scoring. A `structured_weight_decay` (default 0.1, higher than the shared AdamW decay of 0.01) collapses rare verbs toward the shared fallback. Off by default in the current BC baseline.

## Categorical value head (opt-in)

When instantiated with `value_categorical=True`, the value head is `Linear(d_model, value_atoms)` producing atom logits over support `[-value_vmax, +value_vmax]`. The runtime output is the expected scalar `E[value] = softmax(atom_logits) · atom_support` — atom logits are never returned as-is. Off in the current baseline (scalar value).

## What the model actually receives

The encoder combines several kinds of information:

| Input | Role |
|---|---|
| Card and attack IDs | Categorical identity embeddings |
| Static card features | Deterministic domain attributes projected into model space |
| Unit attributes | Dynamic HP, energy, effects, status and other board state |
| Zone/type embeddings | Ownership and semantic role of deck, hand, discard, active, bench, stadium and effects |
| Selection context | Current action phase and selection type |
| Source/target positions | Explicit references from an option to state tokens (gathered, not rebuilt) |
| Action mask | Exact legality boundary supplied by the engine/encoder |
| Tracker flags | Visibility, drawability and certainty per token |
| Meta buckets | Elo-decile bucket per card / agent / deck |
| Day scalar | Normalized competition-day integer |
| Scratch registers | Persistent inter-decision memory when TBPTT is on |

The option token is a structured candidate, not a fixed semantic class tied to an array index. It combines source and target representations, structural attributes, verb identity and attack identity. The output head scores those candidates and applies the legal-action mask.

Mathematically, the current policy is close to:

\[
\pi_\theta(a_t\mid\hat b_t),
\qquad
\hat b_t=\mathcal T(o_t,\text{logs}_t,H_{t-1}),
\]

where `GameTracker` and `AbilityTracker` construct the working belief state before the Transformer is called. `H_{t-1}` is the previous decision's scratch memory (or `learned_init` at match start).

## External state is a strength and a boundary

Keeping the exact rules transition outside the neural network is appropriate. The engine should remain the sound authority for legal actions and state transitions:

\[
s_{t+1}=P(s_t,a_t),
\]

## Architecture boundaries

### Current and authorized

- Entity/action Transformer at 128/4/3-4/512.
- Exact additive action masks and structured option tokens.
- External rules engine and trackers (`GameTracker` + `AbilityTracker`).
- Behavioral cloning with four aux heads.
- 16–32 scratch tokens made persistent across decisions when TBPTT is on.
- Sequential metadata and truncated backpropagation through time.
- MLX training on M3 Pro, strict-FP32 PyTorch arena inference (see [[pokemon_tcg_current_state_reconciliation]] and [[pokemon_tcg_torch_inference]]).

### Deferred

Mamba/Mamba-2, Hope/Nested Learning, multidimensional RoPE / RoPE-ND, Energy-Based Transformer, TRM, J-Lens, strategic MoE, PPO/GRPO/GSPO, PSRO, world models in submitted inference, symbolic regression, custom Metal and distributed Mac training remain research backlog items in [[pokemon_tcg_ladder_and_research]]. They must not be smuggled into the current port under the label of optimization.

## Related pages

- [[pokemon_tcg_ai_battle]] — project hub and evidence boundary
- [[pokemon_tcg_training_pipeline]] — how this architecture is trained today
- [[pokemon_tcg_torch_inference]] — historical FP16 delivery path and current FP32 correction
- [[pokemon_tcg_train_config_reference]] — every flag that steers architecture choices
- [[pokemon_tcg_mlx_migration]] — migration record (historical)
- [[pokemon_tcg_temporal_learning]] — sequence preservation and minimal recurrence
- [[pokemon_tcg_tbptt_training_contract]] — TBPTT accounting on top of these scratch registers
- [[pokemon_tcg_bc_curriculum_ablation]] — the ablation that validated this architecture at scale
- [[pokemon_tcg_ladder_and_research]] — deferred research (RoPE-ND, Mamba, etc.)