Pokémon TCG Metanoia Spec 03 — Model Adherence and Failure Modes

Scope

Spec 03 catalogs how different model families and harnesses adhered to or violated the project's requested protocol. It includes channel leakage, KaTeX/backtick collisions, anthropomorphic deflection, sycophancy and “lip service” responses that appear compliant without preserving the actual constraint.

These categories are useful for reviewing technical handoffs and agent-generated project documents. Percentages and model-family comparisons in the source are source-reported telemetry, not an independently reproduced benchmark.

Failure taxonomy

Failure class	Project-document analogue	Detection signal
Channel leakage	Control markers or internal routing appear in a user-facing handoff	Unexpected markers or malformed state transitions
Formatting collision	Mathematical blocks or code fences are damaged by a renderer	Failed audit/render or altered formula
Anthropomorphic deflection	Model language replaces evidence with claims about intent or feeling	No source, test or observable action behind the sentence
Sycophancy	User interpretation is promoted to fact without reconciliation	Contradictory code/log evidence ignored
Lip service	Document repeats a boundary but still collapses current and future work	Status table and prose disagree

Application to this project

The Stage 3 documentation problem combined two of these failure patterns: validation was treated as a sufficient proxy for arena behavior, and a plausible auxiliary-training narrative concealed a denominator mismatch. The correction is not merely to add more cautionary prose. It is to attach source roles and denominators to each result.

For future agent-generated pages, “adherence” should be checked against:

• named source files and commits;
• direct code paths for implementation claims;
• exact tournament cohorts and counts;
• explicit labels for inference, hypothesis and deferred design;
• no ungrounded statements about model cognition.

What the source does not prove

The suite does not prove that one model family is intrinsically more truthful, more conscious or more capable. It documents observed response patterns under a particular harness and protocol. Nor does it convert an adherence percentage into a guarantee that a future handoff will preserve evidence correctly.

Primary source

docs/metanoia/03_model_adherence_and_failure_mode_analysis.md

Cross-references

• pokemon_tcg_metanoia_suite — suite index.
• pokemon_tcg_cross_harness_provenance — transcript and harness evidence boundary.
• pokemon_tcg_stage3_training_failure_postmortem — technical example of validation overclaim risk.