Pokémon TCG — Entity Normalization and Ghost-Team Resolution

Status and provenance

docs/normalization_heuristics.md entered the project in commit eb83bef on 2026-08-13. It is an analytical blueprint supported by Antigravity brain scratch probes, not a production normalizer or a completed database migration. The current project source tree has no canonical rl/ or scripts/ module that implements this L1/L2 resolver. The analysis therefore belongs in the provenance and ETL layer, while original team strings and replay rows remain untouched.

This page records both the reported result and a methodological discrepancy between the prose document and the saved scratch implementation. That discrepancy is itself part of the research history.

The apparent disappearance problem

The replay corpus contains teams with substantial historical volume that stop appearing before the competition merge boundary. A naïve interpretation is mass banning, deletion or platform failure. The working hypothesis is more specific: a strict limit of two active submissions per TeamId causes older submissions to leave Elo matchmaking after team mergers, while the people and strategies continue under a new team identity.

The historical rows therefore behave like “ghost” subgraphs: they are no longer sampled in new episodes, but their old matches remain valid observations of the earlier platform state. A disappearance is not permission to delete or overwrite a historical identity.

Hierarchical resolution model

flowchart TD
    G[Historical high-volume identity] --> L1[L1: leaderboard and member matching]
    L1 -->|match| M[Candidate current team identity]
    L1 -->|no match| L2[L2: temporal deck-footprint tracing]
    L2 -->|candidate heir| C[Probabilistic continuity candidate]
    L2 -->|no candidate| R[Residual anomaly]
    M --> H[Preserve original row and record mapping]
    C --> H
    R --> U[Unresolved, not merged]

L1: external topology conservation

The source report crosses high-volume identities that stopped before the merge deadline against a current leaderboard. It checks both:

• member usernames in TeamMemberUserNames, split into individual accounts;
• TeamName, for cases where an internal replay name differs from the public team label.

The report states that 142 of 144 high-volume ghosts had a direct leaderboard correspondence. Those figures are source-reported results from the August 13 analysis; they were not recomputed during this documentation pass and should not be promoted to an immutable identity fact.

L2: temporal deck footprint

For identities without an L1 collision, the document proposes matching a stable 60-card deck signature across a short EpisodeId distance. A matching deck near the disappearance boundary is continuity evidence, not proof that two team records are the same legal entity.

The source report states that one additional ghost was resolved by this route. The residual example is Dieter, reported with 1,068 matches and a last episode of 88337213, without a leaderboard collision or lateral deck-footprint match. It remains an unresolved anomaly in the report.

Methodological discrepancy in the artifacts

The saved brain script ~/.gemini/antigravity-cli/brain/<session-id>/scratch/hierarchical_unifier.py does not exactly implement the prose document:

Aspect	Documentation prose	Saved scratch probe
L1 matching	username/team-name collision against the leaderboard	lowercased member/team set plus substring matching
L2 identity	exact 60-card deck signature within < 2,000 episodes	same primary_deck ID in a +60,000 episode window
Baseline deck	not stated in the prose rule	primary deck 1 is excluded from the L2 trace and treated as noise/anomaly
Output	one additional resolved ghost and a residual Dieter case	prints counts and candidates; no persisted mapping table

The second brain probe, timeline_chain_v2.py, also uses a +60,000 episode window and excludes primary deck 1. This means the two distances must not be collapsed into one “implemented heuristic”. The narrow <2,000 claim is a design statement; the wider +60,000 probe is an exploratory method. A future ETL pass must choose and validate a distance with explicit precision/recall evidence.

Safe normalization record

Until a resolver is validated, a candidate mapping should retain at least:

original_team_name
historical_team_id
original_episode_range
candidate_current_team_id
candidate_current_team_name
matching_rule        # L1 username, L1 team name, L2 deck footprint, or none
episode_distance
deck_fingerprint
confidence
unresolved_alternatives
source_snapshot

The canonical row remains attached to its original temporal identity. A candidate heir is an additional relation or analysis record, not an in-place rewrite. This preserves the distinction between a team merger, a submission slot cutoff, a true extinction, a rename and an unobserved competitor.

Relation to sampling, ETL and ratings

The analysis changes how coverage and Elo should be read:

• visible daily teams are a sample of the official population, not the full leaderboard;
• a missing later episode may indicate matchmaking deactivation rather than player disappearance;
• name normalization can reduce false “death” counts but cannot repair missing graph edges or archive-window bias;
• rating aggregates should carry source, archive day and identity-resolution provenance;
• no future-derived Elo, tournament result or synthetic deck should be written into an observed replay row.

These constraints connect the normalization work to pokemon_tcg_sampling_timezone_identity, pokemon_tcg_data_oracle_etl and pokemon_tcg_schema_evolution.

Evidence boundary

What is established by the artifacts:

• the project documented a two-layer resolution hypothesis;
• Antigravity scratch probes exist for leaderboard matching, temporal tracing and orphan profiling;
• the report records 144 high-volume ghosts, 142 L1 matches, one L2 candidate and a residual Dieter anomaly;
• the live project does not contain a production resolver module or a persisted mapping table.

What remains unresolved:

• whether every L1 match is a true team continuation;
• which episode-distance threshold is defensible;
• whether the current leaderboard snapshot is complete for the historical window;
• whether the residual identity was banned, abandoned or simply unobserved;
• how these mappings should enter the target relational schema without collapsing original evidence.

Source anchors

• docs/normalization_heuristics.md, introduced at eb83bef.
• Antigravity probes hierarchical_unifier.py, timeline_chain_v2.py, orphan_profiler.py and pure_dna_map.py in the recorded main brain.
• data/kaggle_leaderboard.csv, leaderboard snapshot used by the probes.
• pokemon_tcg_aug14_data_etl_database_audit, current read-only integrity boundary.

Cross-references

• pokemon_tcg_sampling_timezone_identity — coverage and archive inference.
• pokemon_tcg_data_oracle_etl — zero-trust ETL and source fields.
• pokemon_tcg_sqlite_schema_current — current team, agent and match tables.
• pokemon_tcg_schema_evolution — target identity and provenance relations.
• pokemon_tcg_cross_harness_provenance — brain and transcript ledger.
• pokemon_tcg_repository_timeline — introduction of the analysis in Git.