Pokémon TCG AI Battle — Project Hub

This hub records the current state of the Pokémon TCG AI Battle project in ~/workdir/pokemon-tcg. It is a technical knowledge record, not an implementation task and not a transcript of the originating discussion.

Current state (verified at project HEAD 20d7d0d)

The live source snapshot is strict FP32 for MLX training and PyTorch conversion. The runtime agent performs autoregressive multi-select, carries per-side scratch memory between decisions, and uses the existing entity/action Transformer path. The live tree contains no RoPE-ND, MoE, vehicle-draft or Apex runtime module. SQLite is populated but retains the documented foreign-key and archive-count anomalies. The complete source, transcript, artifact and database join is pokemon_tcg_current_state_reconciliation.

Historical state (2026-08-07 snapshot)

The repository then ran an MLX behavioral-cloning trainer against a day-partitioned Parquet corpus, produced a PyTorch FP16 arena artifact, and used a tournament driver for per-model evaluation and intra-suite round-robin against packaged model tarballs. Within that dated suite, the absence of a broad encoding, conversion, cache or collapse failure supported a data- or regime-limited interpretation of the Elo-oriented ceiling. The later Stage 3/4 training-contract incident prevents that cohort result from being generalized to the current pipeline; see pokemon_tcg_stage3_training_failure_postmortem.

Runtime boundary at the current source snapshot:

training     ->  MLX Metal (Apple Silicon M3 Pro, 24 GiB unified memory)
inference    ->  PyTorch FP32 (current converter and inference contract)
legacy bundle->  PyTorch FP16 (historical self-contained submission record)
recurrence   ->  TBPTT (always on)

Canonical current-state pages (start here):

• pokemon_tcg_training_pipeline — end-to-end training pipeline
• pokemon_tcg_agent_architecture — model, embeddings by zone, aux heads, scratch registers
• pokemon_tcg_torch_inference — historical FP16 arena delivery and current strict-FP32 correction
• pokemon_tcg_train_config_reference — every flag, every field, every default
• pokemon_tcg_kv_cache_hierarchical — three-tier row-group cache
• pokemon_tcg_parquet_dataset — dataset format
• pokemon_tcg_sqlite_schema_current — live catalog schema
• pokemon_tcg_tournament_system — tournament CLI + round-robin as proto-self-play
• pokemon_tcg_top_elo_curriculum_filter — curriculum flag and its findings
• pokemon_tcg_glossary — vocabulary

Ablation and evaluation:

• pokemon_tcg_bc_curriculum_ablation — 10-config days × epochs × top-Elo ablation
• pokemon_tcg_evaluation_results — eval provenance and framing

Historical records (kept for lineage; do not build against them):

• pokemon_tcg_mlx_migration — migration contract and phased delivery
• pokemon_tcg_data_pipeline — pre-Parquet NPY shard era
• pokemon_tcg_prospective_v2 — deprecated sidecar planner design

The active post-migration scope is the local research platform: normalized replay persistence, model/deck/submission identity, source-separated Elo, experiments and anamnese, synchronous arena operation, and a dashboard that can launch the official visualizer externally. See pokemon_tcg_local_overhaul.

The reference is the Mikaelzinho implementation, not the original author's larger PPO/self-play repository. The author implementation is historical context only.

August 14 audit update

The August 7 text above is retained as a historical snapshot. The current review is recorded in three additive pages:

• pokemon_tcg_aug14_architecture_and_handoff_audit - live source versus August 14 blueprints, Antigravity provenance and pending gates;
• pokemon_tcg_aug14_data_etl_database_audit - current ZIP, Parquet, SQLite, day metadata and integrity observations;
• pokemon_tcg_aug14_ablations_tournaments - FP32 Stage 1-4 results, curriculum validation logs, deck-conditioned results and tournament ledger.

The current August 14 repository documents define RoPE-ND, MoE, vehicle drafting, Apex mode, elite ETL and later research as planned or experimental phases. The live tree still contains the existing policy and trainer path; the prospective material is preserved rather than removed. The database and dataset pages now distinguish direct current reads from Antigravity survey claims and unresolved audit deltas.

August 15 documentation expansion

The repository docs/ corpus is larger than the three August 14 audit pages. The complete integration is now indexed in pokemon_tcg_docs_corpus_provenance, which maps all 35 documents, their Git introduction points, source roles and Wikifita destinations.

The most important new analytical record is pokemon_tcg_stage3_training_failure_postmortem. It documents the Stage 3/4 loss-scale and validation incident: Stage 3 was labeled as a one-day top-100 run but loaded two dates; validation was an in-pool episode split rather than a deployment-aligned holdout; the trainer history changed sparse auxiliary normalization; and Stage 4 exposed training auxiliary values around 1,000 beside validation values around 0.2–1.5. The user's catastrophic-forgetting interpretation is preserved as a supported causal hypothesis, not overstated as an isolated gradient proof.

The expanded reading map now includes pokemon_tcg_neural_engine_tokenization, pokemon_tcg_ropend_moe_blueprint, pokemon_tcg_data_oracle_etl, pokemon_tcg_data_augmentation_vehicle_draft, pokemon_tcg_empirical_ablation_monograph, pokemon_tcg_local_platform_design, pokemon_tcg_implementation_specification, pokemon_tcg_schema_evolution, pokemon_tcg_arena_future_architecture, pokemon_tcg_entity_normalization, pokemon_tcg_manuscript_and_empirical_synthesis, pokemon_tcg_manuscript_chapter_lineage, pokemon_tcg_cross_harness_provenance, pokemon_tcg_elo_pagerank_invariance, pokemon_tcg_sampling_timezone_identity and the six-page pokemon_tcg_metanoia_suite. The future RoPEND/MoE, 60-card vehicle-draft augmentation and Apex phases remain explicitly deferred or blueprint-only.

The stage story now has its own denominator-safe page, pokemon_tcg_curriculum_stage_lineage. It keeps the Antigravity 871-game narrative, the principal 420-game FP32 matrix, the separate 3,600-game report, the later teacher sweeps and the ten-run curriculum suite as distinct evidence families, with the Stage 3 loss/validation incident attached to the transition rather than hidden inside a win-rate table.

The repository timeline now also includes the previously underrepresented August 8–13 phase: curriculum orchestration, rule and memory governance, Stage 3 and auxiliary-loss changes, manuscript landing, Abelian Elo, schema/ETL work, sampling analysis and the first RoPEND/MoE documents. pokemon_tcg_repository_timeline records those commits without treating an informal commit subject as evidence of completion.

The handoff itself now has a focused reconciliation page, pokemon_tcg_handoff_acceptance_reconciliation. It separates normative instructions, future interface contracts, test inventories, acceptance criteria and observed runtime evidence. This is especially important because the declared E2E inventory is broader than the current tests/e2e/ implementation and some tests contain reference fallbacks for absent future modules.

The companion pokemon_tcg_orchestration_artifact_ledger preserves the underlying .agents/ topology: explorers, sub-orchestrators, workers, test writers, gate files and one missing M3 explorer handoff. The artifact tree is useful provenance, but its progress files do not replace live imports, test execution or database validation. The rating lineage is likewise split into ordinary source-separated Elo, invariant-deck calibration and a research bridge. elo-as-experimental-protocol records why the similarly named Antigravity and Co-Scientist ratings must not be merged with Pokémon TCG scores.

The August 7 runtime paragraph above is retained as a historical snapshot. For current inference and evaluation status, use the August 14 audit pages and the source-backed runtime pages, which distinguish the earlier FP16 delivery record from the strict-FP32 correction and preserve the unresolved database/reset boundaries.

Current source snapshot at project HEAD

The verified project HEAD is 20d7d0d on develop. The current trainer and inference code use strict FP32 model tensors; agent/main.py recomputes logits for each multi-select substep and carries per-side memory between decisions. The live tree still has no RoPE-ND, MoE, vehicle-draft or Apex implementation. The handoff's M1/M2/M3 entries are orchestration states with pending gates, while M4/M5 remain planned. The complete reconciliation is pokemon_tcg_current_state_reconciliation.

Evidence boundary

Claim class	Current source	How it is used
Repository state	~/workdir/pokemon-tcg/CLAUDE.md, TASK.md, Git tree	Authoritative for current scope and local artifacts
Model implementation	rl/policy.py, rl/policy_mlx.py	Authoritative for the current PyTorch/MLX contracts and divergences
Training implementation	scripts/bc/bc_train.py, scripts/bc/bc_train_mlx.py	Authoritative for trainer behavior and known defects
Live inference	agent/main.py, rl/encoder/encoding.py	Authoritative for tracker, logs, action selection and reset behavior
Local empirical results	model/eval_results.txt, result.html, checkpoints	Observed results; not causal proof
Originating discussion	The technical conversation that created this repository	Research hypotheses and rejected/future directions; not a substitute for code evidence

Historical claims about leaderboard ratings, competition runtime, hardware or team rules must be re-verified against live competition documentation before being used as operational assumptions. This page records only what the repository and its local artifacts support.

Project model

The current agent is a small entity/action Transformer over a strongly structured external belief state. The rules engine and trackers do most of the state construction; the neural model scores legal option tokens.

flowchart LR
    O[Game observation + incremental logs] --> T[GameTracker / AbilityTracker]
    T --> E[Entity and option encoder]
    E --> X[Token Transformer]
    X --> P[Pointer-style option scores]
    P --> A[Legal action selection]
    A --> G[Game engine]
    G --> O

This is an effective engineering reduction: the policy does not learn the complete rules of Pokémon TCG from raw input. It receives a legal, typed and partially reconstructed action surface. The current inference path recomputes the policy for each multi-select substep and carries scratch memory between decisions, but the rules engine and tracker still perform most temporal belief reconstruction. The remaining ceiling is therefore a question of information sufficiency and training/evaluation alignment, not discarded scratch output.

Reading map

• pokemon_tcg_agent_architecture — current entity/action Transformer, trackers, masks, options and architectural ceiling.
• pokemon_tcg_mlx_migration — faithful MLX scope, P0 corrections, FP16 contract, trainer and release plan.
• pokemon_tcg_temporal_learning — trajectory metadata, multi-select semantics, persistent registers, TBPTT and data hygiene.
• pokemon_tcg_tbptt_training_contract — exact decision-chunk, row-budget, accumulation, scheduler, checkpoint and progress accounting contract.
• pokemon_tcg_training_overhaul_2026_07_29 — current temporal trainer, optimization, progress, checkpoint and inference contracts.
• pokemon_tcg_would_ko_prospective_search — additive RoPE-ND planner and group-relative supervision.
• pokemon_tcg_prospective_v2 — historical sidecar and lateral-planner design, what was removed at a942373, and the surviving group-relative objective boundary; no current sidecar runtime.
• pokemon_tcg_action_coverage — deterministic action enumeration and offline/runtime parity for prospective roots.
• pokemon_tcg_data_pipeline — replay ZIPs, BC NPY arrays, would-KO metadata, historical prospective sidecars, and resumable generation.
• pokemon_tcg_daily_corpus_training — resumable orchestration and its smoke-only execution boundary.
• pokemon_tcg_evaluation_results — historical local tournaments, one-epoch prospective result, and ladder provenance boundaries.
• pokemon_tcg_ladder_and_research — ladder population model and rating interpretation.
• pokemon_tcg_research_backlog — explicitly deferred research directions.
• pokemon_tcg_local_overhaul — local overhaul hub and scope boundary.
• pokemon_tcg_local_platform_status — current implementation status for SQLite, tournaments, dashboard, replays, ratings and open product scope.
• pokemon_tcg_implementation_specification — target services, deterministic workflows and acceptance gates.
• pokemon_tcg_schema_evolution — current physical debt versus relational v2.
• pokemon_tcg_arena_future_architecture — future application/worker seam.
• pokemon_tcg_physical_schema — target SQLite design.
• pokemon_tcg_replay_data_model — replay persistence and visualizer contract.
• pokemon_tcg_submissions_and_elo — model/deck/submission and rating policy.
• pokemon_tcg_deck_strategy — immutable deck.csv submission boundary and local outer-loop deck selection.
• pokemon_tcg_experiments_and_anamnesis — experiment and observation model.
• pokemon_tcg_dashboard_and_tournament — dashboard and arena operations.
• pokemon_tcg_current_state_reconciliation — current source, database, handoff milestone and provenance reconciliation.
• pokemon_tcg_entity_normalization — ghost-team resolution heuristics and residual identity uncertainty.
• pokemon_tcg_manuscript_chapter_lineage — chapter-level manuscript provenance and evidence routing.
• mikael-partner — existing people record for Mikael as collaborator; no new person page was created.

Decisions currently in force

• Development target: the M3 Pro with 24 GB unified memory.
• The M1 Air is not part of the main runtime design for this phase.
• The final artifact must be self-contained during arena inference.
• MLX is the selected training runtime and PyTorch is the selected arena inference runtime; the current source contract is strict FP32.
• FP16 remains a historical delivery representation and comparison cohort. Current model parameters, forward activations, reductions, gradient accumulation and optimizer state are validated as FP32 by the live trainer/inference path.
• The current architecture remains the baseline while the port and low-risk recurrence are repaired.
• The full local-overhaul target remains a design/documentation contract; the existing SQLite, tournament, replay and dashboard surfaces are partial live implementation and must not be confused with completion of that target.
• Mamba, Hope, RoPE-ND, Energy-Based Transformer, TRM, J-Lens, strategic MoE, PPO/GRPO/GSPO, world models in inference, symbolic regression, custom Metal and distributed Mac training are deferred.

Status semantics

The companion pages use the following labels:

• Observed — directly supported by repository code or local artifacts.
• Decision — explicitly authorized for the current implementation phase.
• Inference — reasoned interpretation of observed behavior.
• Hypothesis — research proposal not established by the current code.
• Deferred — intentionally outside the current implementation boundary.