WikifitaGitHub live67e8de5
pesquisa · kaggle/pokemon_tcg_long_horizon_vision

Pokémon TCG AI Battle — Long-Horizon Architectural Vision

The multi-stage architectural direction Alefita has articulated for the project: BC bootstrap → GRPO offline / self-play → 900-Elo teacher as frozen world model → additional scratch zones for latent rollout → MoE specialists → two competition tracks. Not implemented; documented so any inheriting agent understands the trajectory the current pipeline is designed to accommodate.

Baixar raw

Pokémon TCG AI Battle — Long-Horizon Architectural Vision

Boundary

None of what follows is implemented in the current runtime. This page records the design direction Alefita has articulated over multiple working conversations, so any agent inheriting the project understands the trajectory the current architecture is deliberately built to accommodate. The current pipeline stops at behavioral cloning with aux heads and TBPTT (pokemon_tcg_training_pipeline). Everything on this page is downstream of that.

The distinction between "implemented" and "vision" is enforced: no code path here has landed on develop. When a piece of it lands, it will get its own current-state page and this section here will point to it.

The staged plan

Alefita has stated the flow in her own words: "o Behavourial Clonning, ele é pra ser apenas um bootstrap de policies, pra gente ter policies base, pra dai engajar no GRPO e Self Play."

Stage 1  Behavioral Cloning (current)
          - trunk BC on real Kaggle replays
          - aux heads (ko, prize, terminal, return)
          - meta features (agent/deck buckets, day scalar)
          - TBPTT recurrence over scratch registers

Stage 2  GRPO offline
          - use the BC-trained model as the prior
          - offline objective, not online RL
          - integrated aux head with a group-relative loss
          - NOT a separate second model (explicit rejection of the sidecar shape)

Stage 3  Self-play with the 900-Elo teacher as a frozen world model
          - the earlier 900 Elo BC (from public_agents/submissions/first_sub_kaggle_2707)
            is kept as an embedded, frozen world model
          - the primary model calls into it via new scratch zones
          - roll out counterfactual continuations without leaving the primary graph

Stage 4  MoE specialists
          - after the base model is solid, MoE specialists on top
          - the DeepSeek "thinking with visual tokens" paper is the reference
          - split by play-pattern archetype ("linhas e retas" vs "complexos e curvas")
          - MoE never precedes the base; it is a strict later stage

Stage 5  Two competition tracks
          - the local dashboard-driven arena runs continuously
          - the Kaggle bundle is a self-contained submission

Alefita has said explicitly: "o mixture of experts entra depois. Tipo, a gente vai fazer o base model, e aí o g r p o e o mixture of experts entram em cima do base model." MoE is not a preemptive optimization; it is the last stage.

Frozen 900-Elo teacher as world model

The idea in her voice: "podemos usar esse teacher model, super barato já existente, como um modelo de mundo de scratch, dai podemos introduzir mais tokens de scratch porém, criar embeddings de zonas tipo já fazemos na arquitetura base, para que essa segunda zona, seja utilizada para o rollout de diversas possiveis jogadas e suas consequências ou para manter estruturado uma comunicação entre os 2 modelos."

Key architectural commitments:

  • The teacher is frozen — no gradient flows into it. It is used exactly like a static resource, similar to how the static card feature table is used today (pokemon_tcg_agent_architecture).
  • The teacher is embedded, not called externally — no RPC, no sidecar file, no second process. In-graph, so the primary model can attend to teacher-emitted signals as ordinary tokens.
  • The teacher's outputs land in a new "scratch zone" of the primary model — the current scratch registers stay for the primary's own workspace. A second zone (extending type_emb with new zone codes) exposes teacher-emitted rollout summaries.
  • Cheap because CPU-only at Kaggle — the teacher was trained at d_model=128, so its forward is CPU-friendly. The whole thing fits in the Kaggle CPU budget.

What the rejected sidecar was trying to do (and how the vision fixes it)

The sidecar experiment (see pokemon_tcg_prospective_v2) tried to answer the right question with the wrong factoring. Alefita's own reconstruction:

"Imagina o seguinte cenário, turno 16, dai faz o forward, pega o sidecar, faz o forward, dai pega o output do sidecar, e invés de usar como resposta final, eu 'simulo' a ação e a próxima, tipo, nesse cenário, o segundo forward do modelo base, seria simulando um jogo onde a ultima ação dele foi a indicada pelo sidecar no primeiro pass, e digamos que nessa simulação, o inimigo sempre não faça nada e passe o turno, dai a gente calcula se o turno do inimigo não tendo feito nada, se com os cards visiveis, a seguinte pergunta 'ele não fez nada no ultimo turno, será que a carta que ele pescou, em combinação com a quantidade de cartas que ele tem na mão, e as cartas dispostas em campo e descarte, será que na proxima rodada qual a chance dele me dar um k.o sem nenhuma carta nova e considerando que ele vai pescar mais uma carta da distribuição do deck inimigo' e dai a gente passa pelo sidecar, pega as probabilidades de vitória, e como o sidecar compara vários, podemos fazer várias."

The right shape for this is not a second model — it is an in-graph rollout using the frozen teacher and additional scratch zones. Which is exactly stage 3 above.

Engram / DeepSeek references

Two research references Alefita has cited (both future direction, not current):

  • DeepSeek "Engram" paper. Referenced as "a tese do paper do Engram da deepseek" — the direction for the next-phase memory design. Details not fully unpacked in the working conversation; captured here so an inheriting agent knows to consult it before designing the next iteration.
  • DeepSeek "thinking with visual tokens" paper. Referenced by Alefita as: "tem um paper da de psique, thinking with visual tokens, que eles falam sobre isso, que eles primeiro treinam base model." Motivates the "base first, MoE later" ordering.

Local infrastructure vision

Alefita has articulated a "continuous local arena" idea:

"eu queria na real, que o torneio local fosse algo tipo o kaggle sabe, um processo que ficasse sempre rodando, mas isso é pro futuro, até pensei na estrutura de uma ledger futuramente pra gerenciar os torneios nesse modo live local, mas futuramente, o ponto é, que precisaria garantir que no build do dataset, ao puxar novos dias do kaggle, o sqlite também fosse syncado e os elos remotos fossem recomputados."

The local arena keeps running as a background service, replaying models continuously against each other and updating local Elo. The BC curriculum ablation's intra-suite round-robin (45 pairs × 30 games) is a concrete prototype of this pattern — see pokemon_tcg_tournament_system "Round-robin (proto-self-play)."

Provenance safety commitment

Alefita has been explicit about a provenance concern: "precisamos de provenancia, basicamente, o agente precisa saber que o tempo está externamente passando, tipo, ontem o meta era esse, hoje o meta é esse, e não que ele está recebendo sinais controversos." Any future data-selection stage must expose the temporal boundary of the meta to the model, not blur days together.

The day_index_norm scalar and days.competition_day monotonic integer already implement this at the current baseline (see pokemon_tcg_parquet_dataset and pokemon_tcg_sqlite_schema_current). The vision preserves and extends this: as the corpus grows, the model should always be able to tell what era a decision belongs to.

What in the current design was chosen with this vision in mind

The current pipeline is not a random point along this trajectory. Specific choices anticipate stages 2–4:

  • Scratch registers as a first-class capacity axis. --scratch-registers can grow from 16 to 32 to more without retraining the rest. Anticipates stage 3's "second zone" for the teacher.
  • Meta buckets already zone-typed. meta_bucket_emb, agent_bucket_emb, deck_bucket_emb mean adding a new zone (teacher-emit zone, MoE-router zone) is a table-size change, not a structural refactor. See pokemon_tcg_agent_architecture.
  • Aux heads architecturally identical. Any new head (GRPO advantage, rollout confidence, MoE gate) is a Linear(d_model, K) on the CLS output, integrated by a fixed loss weight or a learned one (Kendall). Same shape as the four aux heads that shipped.
  • TBPTT already exercises memory across decisions. Stage 3's cross-decision rollouts pass through the same recurrent contract (pokemon_tcg_tbptt_training_contract).
  • Tournament substrate already supports model-vs-model. Round-robin between the 10 ablation models is the exact substrate self-play would sample opponents from (pokemon_tcg_tournament_system).
  • Data selection primed for stratification. The SQLite catalog already carries per-deck, per-card, per-agent Elo populations. Stratified selection (the direction beyond --top-elo) drops in without a rewrite. See pokemon_tcg_top_elo_curriculum_filter "Interpretation" section.

Cross-references