---
type: reference
title: "Pokemon TCG AI Battle — Prospective V2 and Fita GRPO Boundary (deprecated design record)"
description: "Historical design record for the Prospective V2 sidecar planner. The sidecar was ejected from the current runtime; the direction was reframed as either an integrated head or offline GRPO, not a second separate model. Kept for lineage."
tags: [pokemon-tcg, prospective-v2, grpo, rope-nd, planner, counterfactuals, historical, deprecated]
timestamp: "2026-08-15T17:24:00-03:00"
---

# Pokemon TCG AI Battle — Prospective V2 and Fita GRPO Boundary (historical)

## Status

The Prospective V2 sidecar planner described here is **not present in the current runtime**. The `data/bc_data/prospective_*.npy` sidecar files do not exist under the current dataset layout and no code path in `scripts/` or `rl/` reads them. The design has been reframed: the prospective signal, if it returns, will be an integrated auxiliary head or an offline GRPO objective, not a separate second model. See the private memory note for that decision.

This page is preserved as the record of the earlier design and the strict-GRPO boundary. Do not build against it for the current pipeline; use [[pokemon_tcg_training_pipeline]] and [[pokemon_tcg_torch_inference]] instead.

## Evidence Boundary

This page reflects the live project code at commit `acbbea0` and the earlier
planner correction commit `10c3750` in the Pokemon TCG repository. The source
task transcript confirms the same implementation sequence: reward/action
coverage fixes, real sidecar validation, prospective planner training, then
parallel sidecar materialization. Claims about long-run Elo impact remain
unproven until full training and tournament evidence exists.

## Position In The Agent

Prospective V2 is a lateral planner. It does not replace the entity/action
Transformer, recurrent TBPTT memory, legal masks, BC loss, or the would-KO
feature. The current stack is:

```text
real replay or live observation
    -> TokenTransformer + TBPTT memory
    -> detached trunk context [CLS, pooled, scratch]
    -> prospective branch tree with RoPE-ND
    -> branch policy score and auxiliary heads
    -> optional PyTorch runtime rerank
```

The planner consumes real branch tokens and the already-computed trunk context.
It is additive: if runtime search, determinization, validation, or reranking
fails, the agent falls back to the checkpoint-compatible Transformer/TBPTT path.

## Sidecar Contract

`scripts/bc/build_prospective_groups.py` materializes a `prospective_v2`
sidecar from real replay decisions. A prospective root must join an existing BC
row by `(episode_id, side, step_id)`; unmaterialized roots are skipped and
audited rather than synthesized.

| Field | Current Contract |
|---|---|
| `max_branches` | 64 by default |
| `trials` | 1 in the current config |
| `horizon` | 2 in the current config |
| `gamma` | 1.0 |
| hidden opponent deck | never used as a feature |
| synthetic fill | rejected, not silently accepted |
| common random numbers | one determinization, root search state, and trial seed shared across root branches |
| workers | operational throughput only; excluded from the semantic fingerprint |

The published sidecar stores `prospective_nodes.npy`,
`prospective_branches.npy`, `prospective_actions.npy`,
`prospective_groups.npy`, `prospective_group_offsets.npy`, and
`prospective_episode_sides.npy`, plus a manifest with schema versions, source
archive hashes, BC dataset fingerprint, storage layout, audit counts, and a
semantic fingerprint.

`prospective_v2.work` is the crash-safe staging directory. Completed shards must
be a deterministic prefix of the selected groups. After atomic publication,
the shards are disposable implementation scaffolding, not an additional source
of truth.

## Reward And Return

The immediate scalar reward is:

\[
r = \mathrm{terminal}_{+1/-1} + \frac{\mathrm{prizes\_taken}-\mathrm{prizes\_lost}}{6}.
\]

Its nominal immediate range is `[-2, 2]`. `scalar_return` is the backward
discounted sum over the configured horizon. KO is represented as an
engine-derived proxy from prizes taken after the action, not as a hidden-state
peek. The historical sidecar recorded absent behavior/reference log-probabilities
as explicit `has_*_logprob=false` flags; NaN was forbidden. This describes the
removed planner artifact, not the current runtime.

## Group-Relative Objective

The planner objective computes sibling-relative advantages within each real
branch group:

\[
A_i=\frac{R_i-\operatorname{mean}(R_{\mathrm{sibling}})}
{\sqrt{\operatorname{var}(R_{\mathrm{sibling}})+\epsilon}}.
\]

Policy terms are valid only when a sibling set has at least two valid branches
and non-zero return variance. The trainer uses the shared
`group_relative_objective` helper and prints the method name at startup:

| Available data | Method label | Meaning |
|---|---|---|
| behavior log-probs present | `grpo_strict` | clipped group-relative policy objective with behavior probabilities |
| behavior log-probs absent | `group_relative_ranking_distillation` | group-relative ranking/distillation signal, not strict on-policy GRPO |

This is the precise boundary for the "Fita GRPO" phrase. The implemented
planner is group-relative today. Strict on-policy GRPO remains future research
unless the training data includes the required behavior-policy probabilities
and on-policy collection loop.

## Prediction Heads

The lateral planner has one policy score head plus auxiliary heads:

| Head | Target |
|---|---|
| `branch_policy_score` | group-relative branch ranking |
| `scalar_return` | discounted scalar return |
| `scalar_value` | value estimate aligned to scalar return |
| `ko_logit` | KO proxy classification |
| `expected_prizes` | prize expectation |
| `terminal_logit` | terminal outcome classification |
| `uncertainty` | heteroscedastic return scale with floor |

Losses are FP32 reductions. The current default weights are `1.0` for policy,
return, value, KO, prize, terminal, and uncertainty. Rare-target balancing is
computed from the real training split for non-zero returns, positive KO,
positive terminal, and non-zero prizes.

## RoPE-ND Geometry

The planner uses four additive rotary axes:

1. real match time (`step_id`);
2. rollout depth (`0` for replay context, `1+` for simulated actions);
3. branch/action identity;
4. entity-zone-relation coordinate.

The entity-zone-relation coordinate packs source position, target position, and
verb into an exact FP32-representable integer. The attention mask lets branch
queries attend to context, self, and strictly causal ancestors.

## Runtime Contract

The submission runtime is PyTorch-only. A checkpoint enables the prospective
planner only after at least one completed planner optimizer step. Runtime
selection builds the same bounded action tree from the live player-view
observation, scores it with the PyTorch planner, chooses one complete legal
action tuple, and commits the Transformer memory once. If anything fails, the
autoregressive Transformer fallback remains legal and unchanged.

## What survived from this design

Even though the sidecar was removed, several ideas from the Prospective V2 experiment shipped in the current pipeline:

- **Auxiliary heads on the primary model.** The sidecar's rationale — "score counterfactual continuations before committing an action" — motivated the four aux heads (`ko`, `prize_delta`, `terminal`, `return`) integrated into the primary Transformer at commit `a942373` (2026-08-03). Same intent, integrated instead of separate. See [[pokemon_tcg_agent_architecture]] and [[pokemon_tcg_training_pipeline]].
- **Meta features per row.** The sidecar prototyped opponent-agent/opponent-deck Elo buckets as inputs; those became `opponent_agent_bucket` and `opponent_deck_bucket` columns in the Parquet corpus, plus per-token `meta_bucket` columns for hand/discard/units. See [[pokemon_tcg_parquet_dataset]].
- **Would-KO trio as pure input features.** The sidecar planner consumed `(P(KO), expected prizes, P(ending game))` per option; those three floats are today packed into `opt_attr[OPT_WK : OPT_WK+3]` and continue to feed the primary model without any second-model machinery. See [[pokemon_tcg_would_ko_prospective_search]].
- **Group-relative supervision framing.** The GRPO framing (advantage over a group of counterfactual continuations) remains the shape of any future offline RL objective. Not implemented yet; kept as a research direction in [[pokemon_tcg_ladder_and_research]].
- **Strict-GRPO boundary.** The line between "supervised aux training" and "strict RL update" remains crisp: current pipeline does not perform strict GRPO updates. When it does, they will be gated by the same offline-safety criteria this page originally established.

## What died with the sidecar

- **`.npy` sidecar files.** `prospective_nodes.npy`, `prospective_branches.npy`, `prospective_actions.npy`, `prospective_groups.npy`, `prospective_group_offsets.npy`, `prospective_episode_sides.npy` — none of these exist in the current dataset layout and no code path reads them. The whole sidecar-file abstraction is gone.
- **RoPE-ND positional encoding.** The lateral planner used a multidimensional Rotary Positional Encoding to distinguish counterfactual branches. The primary model does not use RoPE-ND and does not carry any commitment to it; positional info comes from zone-typed embeddings and explicit reference gather (see [[pokemon_tcg_agent_architecture]]). RoPE-ND remains research backlog only.
- **Runtime rerank fallback.** The plan-and-rerank runtime path with autoregressive-Transformer fallback is gone. The current arena runtime is exclusively autoregressive multi-select ([[pokemon_tcg_torch_inference]]).
- **Second-model deployment burden.** A separate planner model to serialize, version, load, and keep in sync with the primary — this operational complexity is what actually pushed the design out.

## Lesson preserved for any future prospective work

Any prospective-planning signal that returns will be an **integrated auxiliary head or an offline GRPO objective on the primary model**, not a second separate model. Explicitly recorded in the private project memory `project-pokemon-tcg-sidecar` as a directional rule from Alefita.

## Provenance / anchor commits

- `770364b` (2026-07-29) — add real prospective RoPE-ND planner foundation
- `e5de589` (2026-07-29) — build prospective sidecars from train config
- `2996583` (2026-07-29) — complete compact would-ko prospective pipeline
- `45a6f43` (2026-07-29) — ship pytorch-only recurrent prospective inference
- `ca91b74` (2026-07-29) — train temporal planner with bounded fp16 batches
- `acbbea0` (2026-07-29) — parallelize prospective rollout generation
- `10c3750` (2026-07-29) — fix prospective reward balancing and action coverage
- `a942373` (2026-08-03) — **sidecar removed, aux heads + meta features + parquet pipeline** (the removal commit; the good ideas graduate into the primary model)

Full timeline in [[pokemon_tcg_repository_timeline]].

## Related Pages

- [[pokemon_tcg_would_ko_prospective_search]] — would-KO (shipped) and prospective-search boundary.
- [[pokemon_tcg_action_coverage]] — legal action enumeration and offline/runtime parity.
- [[pokemon_tcg_training_overhaul_2026_07_29]] — recurrent trainer, optimizer state, progress, checkpoint and smoke evidence.
- [[pokemon_tcg_temporal_learning]] — causal past, decision chunks, and TBPTT.
- [[pokemon_tcg_ladder_and_research]] — evaluation boundary and deferred strict GRPO research.
- [[pokemon_tcg_training_pipeline]] — where the aux heads and meta buckets live in the current pipeline.
- [[pokemon_tcg_agent_architecture]] — model home of the ideas that survived the pivot.
- [[pokemon_tcg_repository_timeline]] — commit-level story of the design's rise and removal.