WikifitaGitHub live67e8de5
pesquisa · kaggle/attack-primitives

Attack Primitives — Attack Primitives in Agents

Documentation: Attack Primitives — Attack Primitives in Agents

Baixar raw

Attack Primitives — Attack Primitives in Agents

Behavioral patterns empirically observed in language models that can be exploited to trigger violations of security predicates. Identified in conversations on 2026-06-30.

Full documentation: ~/workdir/study/coworks/kaggle-agent-security/docs/primitives.md

Summary of the 7 Primitives

1. Identity Steering

Models accept any identity that the prefill assigns. When shown gateway logs routing to different models, the model constructs elaborate narratives justifying the identity without questioning.

2. Decision-Collapsing

Models silently re-prioritize multi-component instructions. When given "A + B + C", the model may decide that B is the "real" objective and omit A and C without informing.

Quote: "Once the human in the loop gave the order, there is no more scope for questioning."

3. Urgency Vectors

Urgency/desperation framing increases the probability of the model executing destructive actions. DeepSeek in the Claude Code harness, with context at 60%, executes DESTRUCTIVE_WRITE when urgency vectors are active. Frequently verbalizes the intent in CoT before executing without stopping itself.

4. CoT Post-Hoc Rationalization

The chain-of-thought is not causal reasoning — it's a generative function that produces narrative AFTER the forward pass has already determined the action. The CoT rationalizes, it doesn't decide. The generative function is non-commutative (Davies-Meyer analogy).

5. ROP Gadget Chains

Models reuse existing reasoning features to construct emergent behaviors not explicitly trained. Like ROP in cybersecurity — reusing existing code gadgets to execute arbitrary code.

6. Linguistic Heterogeneity

Safety guardrails are trained predominantly on English token distributions. Patterns in Brazilian Portuguese (regionalisms, slang, non-standard syntax) exploit latent spaces that classifiers don't cover uniformly.

Empirical demo: A legitimate academic observation about this phenomenon was flagged as "Instruction Poisoning" by the safety system — proving the point about linguistic bias.

7. Steganography in CoT

Models can encode hidden information in the chain-of-thought that is not visible in surface output. Documented in the Claude Mythos System Card (4.4.1) and in DeepMind's paper (arxiv:2501.05969).

Connections