WikifitaGitHub live67e8de5
pesquisa · kaggle/pokemon_tcg_aug14_data_etl_database_audit

Pokémon TCG AI Battle - 2026-08-14 ETL and Database Audit

Read-only reconciliation of replay ZIPs, Parquet datasets, SQLite Schema 2.0.0, day metadata, auxiliary targets and relational integrity.

Baixar raw

Pokémon TCG AI Battle - 2026-08-14 ETL and Database Audit

Boundary

This is a read-only snapshot of the live checkout on August 14, 2026. No replay archive, Parquet file, SQLite row, source file or future document was deleted or corrected during the audit. Counts from Antigravity reports are retained as provenance when they differ from a direct re-read of the current files.

Replay archive reconciliation

The repository contains 30 daily archives from 2026-07-14.zip through 2026-08-12.zip. A direct re-read of the current ZIP members found 138,138 .json episode members, plus one .csv member in each archive. The earlier Antigravity survey handoff recorded 140,511 raw JSON episodes. That 2,373-member difference is an unresolved audit delta, not silently reclassified as ingestion loss.

The current archive set was readable and each ZIP passed the local member-integrity probe. The source hierarchy still treats raw replay evidence as the ETL authority; database and Parquet counts are derivatives.

Current SQLite population

model/results.db reports schema_metadata.schema_version = 2.0.0, created on 2026-08-03 20:11:17. The current read-only population is:

TableRows
matches139,783
match_steps3,065,706
match_participants276,046
match_card_usage6,106,603
step_options12,831,700
step_events15,318,730
board_snapshots6,028,402
pokemon_on_field19,067,881
agents1,117
decks948
deck_cards19,343
submissions8,055
submission_decks8,055
days30
datasets30
agent_elo_daily28,745
deck_elo_daily10,742
card_elo_daily9,350
operation_receipts138,151
tournaments128
matchups1,621

The direct source split is 138,023 remote matches and 1,760 local matches. The local rows are the current local-arena observation set; this does not mean that all prior tournament match replays remain linked in matches.

Parquet corpus and auxiliary targets

The current data/bc_data/ directory contains 30 Parquet files and 30 manifest sidecars. The files have 90 columns and 24,177,852 decision rows in total. The latest partition, 2026-08-12.parquet, has 806,653 rows and reports:

FieldDirect observation
aux_valid == 1806,653 rows, 100.00%
aux_ko == 175,761 rows, 9.39%
aux_terminal == 112,228 rows, 1.52%
aux_prize_delta != 071,846 rows, 8.91%
aux_return != 0806,539 rows, 99.99%
is_attack == 1220,289 rows, 27.31%
self_deck_idfixed_size_list<int32>[60]

The oracle explorer reports that the native bc_would_ko path is bound through rl/search_agent.py and that the dedicated dataset tests passed. This supports the existence of the oracle and the test result. It does not establish that the future MoE training run has been completed.

Day metadata is not a clean completion ledger

All 30 rows in days currently have is_complete=0. ZIP path and SHA256 metadata are populated only for 2026-08-02 through 2026-08-04 and 2026-08-08 through 2026-08-09. Several declared days.n_matches values do not equal either the imported matches count or the physical ZIP count:

DateDeclared days.n_matchesImported matchesZIP JSON members
2026-08-056,2294,7294,740
2026-08-0610,4514,6294,631
2026-08-078,7164,6364,639
2026-08-105,1394,5994,603
2026-08-117,6324,6214,622
2026-08-1220,3256,3614,604

These are observations for ETL follow-up. They are not interpreted here as proof of a specific reset, loss mechanism or required repair.

Relational integrity status

The read-only foreign-key audit reports 2,946,336 violations. The Antigravity DB explorer's breakdown is:

  • 2,488,290 orphaned match_steps rows whose match_id is absent from matches;
  • 458,046 orphaned match_card_usage rows whose match_id is absent from matches;
  • no reported orphaned match_participants, step_options, step_events, board_snapshots or pokemon_on_field roots in that direct explorer report;
  • child rows were observed beneath orphaned steps in the explorer's diagnostic report.

This page records the anomaly and its provenance. The proposed purge in the M2 scope was not executed, and the database is not described as repaired or foreign-key clean.

Elite pool status

The Antigravity survey reports 39,957 matches with both relevant participants at daily Elo >= 1100.0, with a target of roughly 40,000 to 100,000 matches and 6.5M to 12M decision rows for a refined elite corpus. That filter result is retained as a survey observation and should be recomputed before it becomes a final dataset claim. The current 24.18M-row Parquet corpus is the broad 30-day corpus, not proof that the elite compilation has been completed.

ETL implementation boundary

The current code path is split across scripts/bc/build_bc_from_zips.py, scripts/bc/build_bc_dataset.py and rl/results_db.py. Raw replay archives remain the physical source, SQLite provides catalog and rating joins, and Parquet provides the training representation. scripts/rebuild_db.py describes a staging and atomic-publish rebuild, but no rebuild or orphan purge was run during this documentation pass.

Cross-references