GRPO Tournament Inference: Evolutionary Hypothesis Selection
GRPO (Group Relative Policy Optimization) applied to tournament inference: how hypotheses compete, evolve, and converge in the Co-Scientist multi-agent architecture.
GRPO Tournament Inference
GRPO (Group Relative Policy Optimization) Tournament Inference is the mechanism by which hypotheses are generated, debated, ranked, and evolved in the unit-distance research project. It combines two distinct concepts:
- GRPO -- a reinforcement learning algorithm that optimizes model behavior by comparing outputs within a group (relative ranking) rather than against a fixed reward signal
- Tournament Inference -- the application of tournament-style competition to select the best reasoning path at inference time (test-time compute)
Together, they form the engine of the Co-Scientist architecture's hypothesis evolution system, as described in the DeepMind Nature 2026 paper and implemented in the Antigravity 2.0 harness.
What Is GRPO?
Group Relative Policy Optimization
GRPO is a reinforcement learning algorithm that differs from traditional RLHF (Reinforcement Learning from Human Feedback) in a fundamental way:
| Property | Traditional RLHF | GRPO |
|---|---|---|
| Comparison basis | Fixed reward model | Relative ranking within a group |
| Training signal | Absolute score | Relative preference |
| Group dynamics | Independent samples | Competitive selection |
| Optimization target | Maximize reward | Win against peers |
In GRPO, the model generates a group of candidate outputs, ranks them relative to each other, and uses the ranking to update the policy. The "group" is the key: instead of asking "is this output good?", GRPO asks "is this output better than the others?".
Why Relative Ranking Works Better
-
Robustness to reward hacking. A fixed reward model can be gamed. Relative ranking within a group is harder to exploit because the "target" moves with each group.
-
Natural exploration. By generating multiple candidates, the model explores the solution space more broadly than single-sample generation.
-
Implicit curriculum. As the model improves, the groups it generates become more competitive, creating a natural difficulty progression.
Tournament Inference
What It Is
Tournament inference applies the tournament selection mechanism from evolutionary algorithms to the inference process. Instead of generating a single answer, the model:
- Generates multiple candidates (the tournament pool)
- Evaluates them against each other (the matches)
- Selects the winner (the champion)
- Optionally evolves the winner (mutation/refinement)
This is the "tournament and evolution" process described in the DeepMind Co-Scientist paper as one of the system's principal contributions.
The Co-Scientist Agent Roles
The tournament maps directly to the agent roles in the Co-Scientist architecture:
| Tournament Phase | Agent Role | Function |
|---|---|---|
| Pool generation | Generation Agent | Proposes initial hypotheses |
| Diversity check | Proximity Agent | Maps and clusters hypotheses for diverse exploration |
| Match play | Reflection Agent | Evaluates hypotheses against each other |
| Seeding | Ranking Agent | Organizes the tournament bracket |
| Champion refinement | Evolution Agent | Refines and improves the winning hypothesis |
| Tournament oversight | Meta-review Agent | Synthesizes insights to optimize the system |
Test-Time Compute Scaling
A key finding from the Co-Scientist paper: hypotheses improve with increased computation dedicated to the task (test-time compute). More tournament rounds, more candidates per round, and more evolution cycles all produce better results -- up to the diminishing returns ceiling.
This is directly analogous to how chess engines improve with more search depth. The tournament is the search mechanism; GRPO is the evaluation function.
The Elo System
How Elo Maps to Tournament Inference
The unit-distance project used an Elo rating system (borrowed from chess) to calibrate the quality of hypotheses. This is a natural fit for tournament inference because:
- Elo is relative. Like GRPO, Elo ranks participants against each other, not against an absolute scale.
- Elo is dynamic. A hypothesis's rating changes as it "plays" against other hypotheses.
- Elo is interpretable. The rating maps to a meaningful quality level.
The Elo Calibration Scale
| Elo Range | Approximate Meaning | Unit-Distance Examples |
|---|---|---|
| ~1800 | Existential proof (barely above trivial) | OpenAI's original proof (delta ~10^-38) |
| 2150 | First valid construction | H7: Full Multivariate |
| 2200 | Matches human SOTA (Sawin) | H15: Central CM Tower (delta = 0.014) |
| 2350 | Surpasses human SOTA | H16: Multi-Quadratic CM16 (delta = 0.0196) |
| 2500+ | Significant improvement over known bounds | Target for Season 3 |
| 2700+ | Human expert level | Current human SOTA (delta > 0.036) |
Tournament Match Structure
Each "match" between hypotheses evaluates:
- Mathematical correctness. Is the proof valid? Does the construction satisfy the Golod-Shafarevich inequality?
- Novelty. Does this approach explore a new region of the solution space?
- Reproducibility. Can the result be verified independently?
- Elegance. (Subjective but important) Is the construction clean and minimal?
The winner of each match advances; the loser is either eliminated or sent to the EVOLVE state for refinement.
The Evolutionary Selection Mechanism
Generate-Debate-Evolve Loop
The full tournament inference cycle implements an evolutionary algorithm:
graph TD
A[INIT: Parse research question] --> B[GENERATE: Produce 2-4 hypotheses]
B --> C[DEBATE: Critique each hypothesis]
C --> D[RANK: Elo tournament selection]
D --> E{Winner mature enough?}
E -->|Yes| F[RESPOND: Output result]
E -->|No| G[EVOLVE: Refine winner]
G --> H{Iteration < 5?}
H -->|Yes| I[LOOP: Increment counter]
I --> B
H -->|No| J[HALT or RESPOND]
G --> K[DELEGATE: Sub-agent verification]
K --> L[SYNTHESIZE: Merge findings]
L --> E
Mutation Operators
In evolutionary algorithm terms, the EVOLVE state applies mutation operators to the winning hypothesis:
| Operator | Description | Example |
|---|---|---|
| Parameter tuning | Adjust numerical parameters | Change the polydisc radius in H2 |
| Structural extension | Add new components | Add more primes to the class field tower |
| Dimensional shift | Change the base field degree | Move from degree 8 to degree 16 (H16) |
| Constraint relaxation | Remove unnecessary restrictions | Allow non-abelian extensions |
| Hybrid merging | Combine elements from multiple hypotheses | Merge the prime selection from H4 with the base field from H8 |
Selection Pressure
The tournament creates selection pressure by:
- Eliminating weak hypotheses (they lose matches)
- Rewarding strong hypotheses (they accumulate Elo)
- Forcing evolution (the EVOLVE state refines the winner)
- Capping iteration (the 5-loop maximum prevents infinite refinement)
This selection pressure drives the system toward increasingly better solutions. In the unit-distance project, it drove the system from the existential proof (delta ~10^-38) to H16 (delta = 0.0196) across 16 hypotheses.
Connection to Multi-Agent Methodology
The Swarm Model
The unit-distance project used a cognitive swarm model with specialized roles:
| Swarm Role | Tournament Function | Antigravity Implementation |
|---|---|---|
| Research Director | Tournament organizer | Main agent (Gemini 3.1 Pro High) |
| ELO-RANKER | Match judge | Specialized sub-agent with web access |
| CITATION-VERIFIER | Fact-checker | Read-only sub-agent |
| PROVENANCE-TRACKER | Audit trail | PostToolUse hook or sub-agent |
| WIKI-MAINTAINER | Knowledge base sync | uv run skill script |
Delegation and Synthesis
The DELEGATE state allows the Research Director to offload specific tournament functions to specialized sub-agents:
- Verification delegation: A sub-agent checks whether a hypothesis's mathematical claims are valid, without the main agent losing context.
- Parallel exploration: Multiple sub-agents explore different evolutionary branches simultaneously.
- External calibration: The ELO-RANKER sub-agent provides objective rating by comparing against external benchmarks (Sawin's bound, OpenAI's proof).
After delegation, the SYNTHESIZE state integrates sub-agent findings back into the main reasoning thread.
Why Multi-Agent Matters
Single-agent tournament inference is limited by:
- Context window: One model cannot hold all hypotheses, all critiques, and all historical context simultaneously
- Bias: A single model may favor its own earlier outputs
- Specialization: Different tasks (verification, ranking, evolution) benefit from different prompting strategies
The multi-agent swarm distributes these functions across specialized sub-agents, each operating with its own context and constraints.
Alefita's Proposed Extension: Steering Vector Optimization
The DeepSeek transcript discussed Alefita's proposal to use GRPO tournament inference for persona steering vector optimization:
The Protocol
- Prefill a steering vector in the first block of the CoT
- Map a base list of desired behavioral traits (curiosity, skepticism, precision, etc.)
- Run 5 cycles of GRPO with Elo tournament to select the best steering configuration
- Apply the winning steering to subsequent reasoning
Why This Works
- GRPO naturally optimizes for relative quality. The "best" steering vector is the one that produces the best reasoning relative to alternatives.
- Tournament selection prevents local optima. By maintaining a diverse pool of steering configurations, the system avoids converging on a single (potentially suboptimal) configuration.
- The 5-loop cap matches the channel protocol. The same iteration limit that governs hypothesis evolution also governs steering optimization.
The Testability Claim
The DeepSeek transcript noted: "Your theory about prefill of steering with GRPO and Elo tournament is testable." This is a concrete, implementable protocol that could be evaluated empirically.
Historical Evolution in the Unit-Distance Project
The Hypothesis Tournament
The unit-distance project itself was a tournament of 16 hypotheses:
| Hypothesis | Elo | Status | Key Innovation |
|---|---|---|---|
| H1: Valuation Optimization | -- | Validated | Authentic OpenAI derivation |
| H2: Continuous Polydisc Radius | -- | Validated | Authentic OpenAI derivation |
| H3: Pro-2 Class Towers | -- | Disqualified | External contamination |
| H4: Higher Valuation Powers | -- | Validated (below benchmark) | Pro-3 towers |
| H5-H6: Global Optimization | -- | Validated (below benchmark) | Combined optimization |
| H7: Full Multivariate | ~2150 | Validated | Full multivariate approach |
| H8: Imaginary Quadratic Base | ~2650 | Validated | Imaginary quadratic foundation |
| H9: Multi-Quadratic CM | -- | Disqualified | Completeness failure |
| H10: Galois Symmetry | -- | Disqualified | Completeness failure |
| H11: Imaginary Quadratic 2-Tower | 1500 | Proven | First formal proof |
| H12-H14 | -- | Disqualified | Completeness failures |
| H15: Central CM Tower | 2200 | Proven | Matches Sawin bound |
| H16: Multi-Quadratic CM16 | 2350 | Proven | Surpasses Sawin |
The Tournament Arc
The progression from H1 to H16 demonstrates the evolutionary selection mechanism in action:
- H1-H2: Establish the base (OpenAI derivation)
- H3: First disqualification (contamination detection -- the anti-contamination protocol worked)
- H4-H6: Incremental improvements (below benchmark but valid)
- H7-H8: Major jumps (entering competitive range)
- H9-H10: Failed experiments (necessary for completeness)
- H11: First formal proof (milestone but low delta)
- H12-H14: Rapid iteration (three failures in quick succession)
- H15: Breakthrough (matches Sawin)
- H16: Surpasses Sawin (the tournament champion)
GRPO vs. Other Optimization Methods
| Method | Comparison Basis | Exploration | Adaptation | Used In |
|---|---|---|---|---|
| GRPO | Relative within group | High (multiple candidates) | Fast (group dynamics) | Co-Scientist, unit-distance |
| RLHF | Fixed reward model | Low (single sample) | Slow (reward model is static) | Standard LLM training |
| PPO | Advantage estimation | Medium (clipped updates) | Medium | General RL |
| Evo-Search | Population-based | High (mutation/recombination) | Medium (no policy gradient) | Genetic algorithms |
GRPO's advantage for multi-agent research: it combines the exploration of evolutionary search with the optimization efficiency of policy gradient methods, all operating at inference time rather than training time.
Cross-References
- unit-distance-channel-protocol -- The RANK state that implements the tournament selection
- unit-distance-steganographic-cot -- How steering vector optimization could use GRPO to refine steganographic patterns
- unit-distance-tokenizer-analysis -- The tokenizer infrastructure that enables structured tournament output
This page documents the theoretical framework connecting GRPO, tournament inference, and the multi-agent research methodology. The core concepts (GRPO, Elo, evolutionary selection) are established in the literature. The application to persona steering vector optimization is a proposal discussed in the DeepSeek transcript and awaits empirical validation.