Pokemon TCG AI Battle — Prospective V2 and Fita GRPO Boundary (historical)

Status

The Prospective V2 sidecar planner described here is not present in the current runtime. The data/bc_data/prospective_*.npy sidecar files do not exist under the current dataset layout and no code path in scripts/ or rl/ reads them. The design has been reframed: the prospective signal, if it returns, will be an integrated auxiliary head or an offline GRPO objective, not a separate second model. See the private memory note for that decision.

This page is preserved as the record of the earlier design and the strict-GRPO boundary. Do not build against it for the current pipeline; use pokemon_tcg_training_pipeline and pokemon_tcg_torch_inference instead.

Evidence Boundary

This page reflects the live project code at commit acbbea0 and the earlier planner correction commit 10c3750 in the Pokemon TCG repository. The source task transcript confirms the same implementation sequence: reward/action coverage fixes, real sidecar validation, prospective planner training, then parallel sidecar materialization. Claims about long-run Elo impact remain unproven until full training and tournament evidence exists.

Position In The Agent

Prospective V2 is a lateral planner. It does not replace the entity/action Transformer, recurrent TBPTT memory, legal masks, BC loss, or the would-KO feature. The current stack is:

real replay or live observation
    -> TokenTransformer + TBPTT memory
    -> detached trunk context [CLS, pooled, scratch]
    -> prospective branch tree with RoPE-ND
    -> branch policy score and auxiliary heads
    -> optional PyTorch runtime rerank

The planner consumes real branch tokens and the already-computed trunk context. It is additive: if runtime search, determinization, validation, or reranking fails, the agent falls back to the checkpoint-compatible Transformer/TBPTT path.

Sidecar Contract

scripts/bc/build_prospective_groups.py materializes a prospective_v2 sidecar from real replay decisions. A prospective root must join an existing BC row by (episode_id, side, step_id); unmaterialized roots are skipped and audited rather than synthesized.

Field	Current Contract
`max_branches`	64 by default
`trials`	1 in the current config
`horizon`	2 in the current config
`gamma`	1.0
hidden opponent deck	never used as a feature
synthetic fill	rejected, not silently accepted
common random numbers	one determinization, root search state, and trial seed shared across root branches
workers	operational throughput only; excluded from the semantic fingerprint

The published sidecar stores prospective_nodes.npy, prospective_branches.npy, prospective_actions.npy, prospective_groups.npy, prospective_group_offsets.npy, and prospective_episode_sides.npy, plus a manifest with schema versions, source archive hashes, BC dataset fingerprint, storage layout, audit counts, and a semantic fingerprint.

prospective_v2.work is the crash-safe staging directory. Completed shards must be a deterministic prefix of the selected groups. After atomic publication, the shards are disposable implementation scaffolding, not an additional source of truth.

Reward And Return

The immediate scalar reward is:

[ r = \mathrm{terminal}_{+1/-1} + \frac{\mathrm{prizes_taken}-\mathrm{prizes_lost}}{6}. ]

Its nominal immediate range is [-2, 2]. scalar_return is the backward discounted sum over the configured horizon. KO is represented as an engine-derived proxy from prizes taken after the action, not as a hidden-state peek. The historical sidecar recorded absent behavior/reference log-probabilities as explicit has_*_logprob=false flags; NaN was forbidden. This describes the removed planner artifact, not the current runtime.

Group-Relative Objective

The planner objective computes sibling-relative advantages within each real branch group:

[ A_i=\frac{R_i-\operatorname{mean}(R_{\mathrm{sibling}})} {\sqrt{\operatorname{var}(R_{\mathrm{sibling}})+\epsilon}}. ]

Policy terms are valid only when a sibling set has at least two valid branches and non-zero return variance. The trainer uses the shared group_relative_objective helper and prints the method name at startup:

Available data	Method label	Meaning
behavior log-probs present	`grpo_strict`	clipped group-relative policy objective with behavior probabilities
behavior log-probs absent	`group_relative_ranking_distillation`	group-relative ranking/distillation signal, not strict on-policy GRPO

This is the precise boundary for the "Fita GRPO" phrase. The implemented planner is group-relative today. Strict on-policy GRPO remains future research unless the training data includes the required behavior-policy probabilities and on-policy collection loop.

Prediction Heads

The lateral planner has one policy score head plus auxiliary heads:

Head	Target
`branch_policy_score`	group-relative branch ranking
`scalar_return`	discounted scalar return
`scalar_value`	value estimate aligned to scalar return
`ko_logit`	KO proxy classification
`expected_prizes`	prize expectation
`terminal_logit`	terminal outcome classification
`uncertainty`	heteroscedastic return scale with floor

Losses are FP32 reductions. The current default weights are 1.0 for policy, return, value, KO, prize, terminal, and uncertainty. Rare-target balancing is computed from the real training split for non-zero returns, positive KO, positive terminal, and non-zero prizes.

RoPE-ND Geometry

The planner uses four additive rotary axes:

real match time (step_id);
rollout depth (0 for replay context, 1+ for simulated actions);
branch/action identity;
entity-zone-relation coordinate.

The entity-zone-relation coordinate packs source position, target position, and verb into an exact FP32-representable integer. The attention mask lets branch queries attend to context, self, and strictly causal ancestors.

Runtime Contract

The submission runtime is PyTorch-only. A checkpoint enables the prospective planner only after at least one completed planner optimizer step. Runtime selection builds the same bounded action tree from the live player-view observation, scores it with the PyTorch planner, chooses one complete legal action tuple, and commits the Transformer memory once. If anything fails, the autoregressive Transformer fallback remains legal and unchanged.

What survived from this design

Even though the sidecar was removed, several ideas from the Prospective V2 experiment shipped in the current pipeline:

Auxiliary heads on the primary model. The sidecar's rationale — "score counterfactual continuations before committing an action" — motivated the four aux heads (ko, prize_delta, terminal, return) integrated into the primary Transformer at commit a942373 (2026-08-03). Same intent, integrated instead of separate. See pokemon_tcg_agent_architecture and pokemon_tcg_training_pipeline.
Meta features per row. The sidecar prototyped opponent-agent/opponent-deck Elo buckets as inputs; those became opponent_agent_bucket and opponent_deck_bucket columns in the Parquet corpus, plus per-token meta_bucket columns for hand/discard/units. See pokemon_tcg_parquet_dataset.
Would-KO trio as pure input features. The sidecar planner consumed (P(KO), expected prizes, P(ending game)) per option; those three floats are today packed into opt_attr[OPT_WK : OPT_WK+3] and continue to feed the primary model without any second-model machinery. See pokemon_tcg_would_ko_prospective_search.
Group-relative supervision framing. The GRPO framing (advantage over a group of counterfactual continuations) remains the shape of any future offline RL objective. Not implemented yet; kept as a research direction in pokemon_tcg_ladder_and_research.
Strict-GRPO boundary. The line between "supervised aux training" and "strict RL update" remains crisp: current pipeline does not perform strict GRPO updates. When it does, they will be gated by the same offline-safety criteria this page originally established.

What died with the sidecar

.npy sidecar files. prospective_nodes.npy, prospective_branches.npy, prospective_actions.npy, prospective_groups.npy, prospective_group_offsets.npy, prospective_episode_sides.npy — none of these exist in the current dataset layout and no code path reads them. The whole sidecar-file abstraction is gone.
RoPE-ND positional encoding. The lateral planner used a multidimensional Rotary Positional Encoding to distinguish counterfactual branches. The primary model does not use RoPE-ND and does not carry any commitment to it; positional info comes from zone-typed embeddings and explicit reference gather (see pokemon_tcg_agent_architecture). RoPE-ND remains research backlog only.
Runtime rerank fallback. The plan-and-rerank runtime path with autoregressive-Transformer fallback is gone. The current arena runtime is exclusively autoregressive multi-select (pokemon_tcg_torch_inference).
Second-model deployment burden. A separate planner model to serialize, version, load, and keep in sync with the primary — this operational complexity is what actually pushed the design out.

Lesson preserved for any future prospective work

Any prospective-planning signal that returns will be an integrated auxiliary head or an offline GRPO objective on the primary model, not a second separate model. Explicitly recorded in the private project memory project-pokemon-tcg-sidecar as a directional rule from Alefita.

Provenance / anchor commits

770364b (2026-07-29) — add real prospective RoPE-ND planner foundation
e5de589 (2026-07-29) — build prospective sidecars from train config
2996583 (2026-07-29) — complete compact would-ko prospective pipeline
45a6f43 (2026-07-29) — ship pytorch-only recurrent prospective inference
ca91b74 (2026-07-29) — train temporal planner with bounded fp16 batches
acbbea0 (2026-07-29) — parallelize prospective rollout generation
10c3750 (2026-07-29) — fix prospective reward balancing and action coverage
a942373 (2026-08-03) — sidecar removed, aux heads + meta features + parquet pipeline (the removal commit; the good ideas graduate into the primary model)

Full timeline in pokemon_tcg_repository_timeline.

pokemon_tcg_would_ko_prospective_search — would-KO (shipped) and prospective-search boundary.
pokemon_tcg_action_coverage — legal action enumeration and offline/runtime parity.
pokemon_tcg_training_overhaul_2026_07_29 — recurrent trainer, optimizer state, progress, checkpoint and smoke evidence.
pokemon_tcg_temporal_learning — causal past, decision chunks, and TBPTT.
pokemon_tcg_ladder_and_research — evaluation boundary and deferred strict GRPO research.
pokemon_tcg_training_pipeline — where the aux heads and meta buckets live in the current pipeline.
pokemon_tcg_agent_architecture — model home of the ideas that survived the pivot.
pokemon_tcg_repository_timeline — commit-level story of the design's rise and removal.

Pokemon TCG AI Battle — Prospective V2 and Fita GRPO Boundary (deprecated design record)