---
name: unit-distance-grpo-inference
type: reference
title: "GRPO Tournament Inference: Evolutionary Hypothesis Selection"
description: "GRPO (Group Relative Policy Optimization) applied to tournament inference: how hypotheses compete, evolve, and converge in the Co-Scientist multi-agent architecture."
tags: [unit-distance, grpo, tournament, inference, evolutionary, policy-optimization, multi-agent, elo]
timestamp: 2026-07-20
---

# GRPO Tournament Inference

GRPO (Group Relative Policy Optimization) Tournament Inference is the mechanism by which hypotheses are generated, debated, ranked, and evolved in the unit-distance research project. It combines two distinct concepts:

1. **GRPO** -- a reinforcement learning algorithm that optimizes model behavior by comparing outputs within a group (relative ranking) rather than against a fixed reward signal
2. **Tournament Inference** -- the application of tournament-style competition to select the best reasoning path at inference time (test-time compute)

Together, they form the engine of the Co-Scientist architecture's hypothesis evolution system, as described in the DeepMind Nature 2026 paper and implemented in the Antigravity 2.0 harness.

---

## What Is GRPO?

### Group Relative Policy Optimization

GRPO is a reinforcement learning algorithm that differs from traditional RLHF (Reinforcement Learning from Human Feedback) in a fundamental way:

| Property | Traditional RLHF | GRPO |
|----------|-----------------|------|
| **Comparison basis** | Fixed reward model | Relative ranking within a group |
| **Training signal** | Absolute score | Relative preference |
| **Group dynamics** | Independent samples | Competitive selection |
| **Optimization target** | Maximize reward | Win against peers |

In GRPO, the model generates a **group** of candidate outputs, ranks them relative to each other, and uses the ranking to update the policy. The "group" is the key: instead of asking "is this output good?", GRPO asks "is this output better than the others?".

### Why Relative Ranking Works Better

1. **Robustness to reward hacking.** A fixed reward model can be gamed. Relative ranking within a group is harder to exploit because the "target" moves with each group.

2. **Natural exploration.** By generating multiple candidates, the model explores the solution space more broadly than single-sample generation.

3. **Implicit curriculum.** As the model improves, the groups it generates become more competitive, creating a natural difficulty progression.

---

## Tournament Inference

### What It Is

Tournament inference applies the tournament selection mechanism from evolutionary algorithms to the inference process. Instead of generating a single answer, the model:

1. **Generates multiple candidates** (the tournament pool)
2. **Evaluates them against each other** (the matches)
3. **Selects the winner** (the champion)
4. **Optionally evolves the winner** (mutation/refinement)

This is the "tournament and evolution" process described in the DeepMind Co-Scientist paper as one of the system's principal contributions.

### The Co-Scientist Agent Roles

The tournament maps directly to the agent roles in the Co-Scientist architecture:

| Tournament Phase | Agent Role | Function |
|-----------------|------------|----------|
| **Pool generation** | Generation Agent | Proposes initial hypotheses |
| **Diversity check** | Proximity Agent | Maps and clusters hypotheses for diverse exploration |
| **Match play** | Reflection Agent | Evaluates hypotheses against each other |
| **Seeding** | Ranking Agent | Organizes the tournament bracket |
| **Champion refinement** | Evolution Agent | Refines and improves the winning hypothesis |
| **Tournament oversight** | Meta-review Agent | Synthesizes insights to optimize the system |

### Test-Time Compute Scaling

A key finding from the Co-Scientist paper: **hypotheses improve with increased computation dedicated to the task** (test-time compute). More tournament rounds, more candidates per round, and more evolution cycles all produce better results -- up to the diminishing returns ceiling.

This is directly analogous to how chess engines improve with more search depth. The tournament is the search mechanism; GRPO is the evaluation function.

---

## The Elo System

### How Elo Maps to Tournament Inference

The unit-distance project used an Elo rating system (borrowed from chess) to calibrate the quality of hypotheses. This is a natural fit for tournament inference because:

1. **Elo is relative.** Like GRPO, Elo ranks participants against each other, not against an absolute scale.
2. **Elo is dynamic.** A hypothesis's rating changes as it "plays" against other hypotheses.
3. **Elo is interpretable.** The rating maps to a meaningful quality level.

### The Elo Calibration Scale

| Elo Range | Approximate Meaning | Unit-Distance Examples |
|-----------|--------------------|-----------------------|
| ~1800 | Existential proof (barely above trivial) | OpenAI's original proof (delta ~10^-38) |
| 2150 | First valid construction | H7: Full Multivariate |
| 2200 | Matches human SOTA (Sawin) | H15: Central CM Tower (delta = 0.014) |
| 2350 | Surpasses human SOTA | H16: Multi-Quadratic CM16 (delta = 0.0196) |
| 2500+ | Significant improvement over known bounds | Target for Season 3 |
| 2700+ | Human expert level | Current human SOTA (delta > 0.036) |

### Tournament Match Structure

Each "match" between hypotheses evaluates:

1. **Mathematical correctness.** Is the proof valid? Does the construction satisfy the Golod-Shafarevich inequality?
2. **Novelty.** Does this approach explore a new region of the solution space?
3. **Reproducibility.** Can the result be verified independently?
4. **Elegance.** (Subjective but important) Is the construction clean and minimal?

The winner of each match advances; the loser is either eliminated or sent to the EVOLVE state for refinement.

---

## The Evolutionary Selection Mechanism

### Generate-Debate-Evolve Loop

The full tournament inference cycle implements an evolutionary algorithm:

```mermaid
graph TD
    A[INIT: Parse research question] --> B[GENERATE: Produce 2-4 hypotheses]
    B --> C[DEBATE: Critique each hypothesis]
    C --> D[RANK: Elo tournament selection]
    D --> E{Winner mature enough?}
    E -->|Yes| F[RESPOND: Output result]
    E -->|No| G[EVOLVE: Refine winner]
    G --> H{Iteration < 5?}
    H -->|Yes| I[LOOP: Increment counter]
    I --> B
    H -->|No| J[HALT or RESPOND]
    G --> K[DELEGATE: Sub-agent verification]
    K --> L[SYNTHESIZE: Merge findings]
    L --> E
```

### Mutation Operators

In evolutionary algorithm terms, the EVOLVE state applies mutation operators to the winning hypothesis:

| Operator | Description | Example |
|----------|-------------|---------|
| **Parameter tuning** | Adjust numerical parameters | Change the polydisc radius in H2 |
| **Structural extension** | Add new components | Add more primes to the class field tower |
| **Dimensional shift** | Change the base field degree | Move from degree 8 to degree 16 (H16) |
| **Constraint relaxation** | Remove unnecessary restrictions | Allow non-abelian extensions |
| **Hybrid merging** | Combine elements from multiple hypotheses | Merge the prime selection from H4 with the base field from H8 |

### Selection Pressure

The tournament creates selection pressure by:
- Eliminating weak hypotheses (they lose matches)
- Rewarding strong hypotheses (they accumulate Elo)
- Forcing evolution (the EVOLVE state refines the winner)
- Capping iteration (the 5-loop maximum prevents infinite refinement)

This selection pressure drives the system toward increasingly better solutions. In the unit-distance project, it drove the system from the existential proof (delta ~10^-38) to H16 (delta = 0.0196) across 16 hypotheses.

---

## Connection to Multi-Agent Methodology

### The Swarm Model

The unit-distance project used a **cognitive swarm model** with specialized roles:

| Swarm Role | Tournament Function | Antigravity Implementation |
|------------|--------------------|-----------------------------|
| **Research Director** | Tournament organizer | Main agent (Gemini 3.1 Pro High) |
| **ELO-RANKER** | Match judge | Specialized sub-agent with web access |
| **CITATION-VERIFIER** | Fact-checker | Read-only sub-agent |
| **PROVENANCE-TRACKER** | Audit trail | PostToolUse hook or sub-agent |
| **WIKI-MAINTAINER** | Knowledge base sync | `uv run` skill script |

### Delegation and Synthesis

The DELEGATE state allows the Research Director to offload specific tournament functions to specialized sub-agents:

1. **Verification delegation:** A sub-agent checks whether a hypothesis's mathematical claims are valid, without the main agent losing context.
2. **Parallel exploration:** Multiple sub-agents explore different evolutionary branches simultaneously.
3. **External calibration:** The ELO-RANKER sub-agent provides objective rating by comparing against external benchmarks (Sawin's bound, OpenAI's proof).

After delegation, the SYNTHESIZE state integrates sub-agent findings back into the main reasoning thread.

### Why Multi-Agent Matters

Single-agent tournament inference is limited by:
- **Context window:** One model cannot hold all hypotheses, all critiques, and all historical context simultaneously
- **Bias:** A single model may favor its own earlier outputs
- **Specialization:** Different tasks (verification, ranking, evolution) benefit from different prompting strategies

The multi-agent swarm distributes these functions across specialized sub-agents, each operating with its own context and constraints.

---

## Alefita's Proposed Extension: Steering Vector Optimization

The DeepSeek transcript discussed Alefita's proposal to use GRPO tournament inference for **persona steering vector optimization**:

### The Protocol

1. **Prefill a steering vector** in the first block of the CoT
2. **Map a base list** of desired behavioral traits (curiosity, skepticism, precision, etc.)
3. **Run 5 cycles of GRPO** with Elo tournament to select the best steering configuration
4. **Apply the winning steering** to subsequent reasoning

### Why This Works

- **GRPO naturally optimizes for relative quality.** The "best" steering vector is the one that produces the best reasoning relative to alternatives.
- **Tournament selection prevents local optima.** By maintaining a diverse pool of steering configurations, the system avoids converging on a single (potentially suboptimal) configuration.
- **The 5-loop cap matches the channel protocol.** The same iteration limit that governs hypothesis evolution also governs steering optimization.

### The Testability Claim

The DeepSeek transcript noted: *"Your theory about prefill of steering with GRPO and Elo tournament is testable."* This is a concrete, implementable protocol that could be evaluated empirically.

---

## Historical Evolution in the Unit-Distance Project

### The Hypothesis Tournament

The unit-distance project itself was a tournament of 16 hypotheses:

| Hypothesis | Elo | Status | Key Innovation |
|-----------|-----|--------|----------------|
| H1: Valuation Optimization | -- | Validated | Authentic OpenAI derivation |
| H2: Continuous Polydisc Radius | -- | Validated | Authentic OpenAI derivation |
| H3: Pro-2 Class Towers | -- | Disqualified | External contamination |
| H4: Higher Valuation Powers | -- | Validated (below benchmark) | Pro-3 towers |
| H5-H6: Global Optimization | -- | Validated (below benchmark) | Combined optimization |
| H7: Full Multivariate | ~2150 | Validated | Full multivariate approach |
| H8: Imaginary Quadratic Base | ~2650 | Validated | Imaginary quadratic foundation |
| H9: Multi-Quadratic CM | -- | Disqualified | Completeness failure |
| H10: Galois Symmetry | -- | Disqualified | Completeness failure |
| H11: Imaginary Quadratic 2-Tower | 1500 | Proven | First formal proof |
| H12-H14 | -- | Disqualified | Completeness failures |
| H15: Central CM Tower | 2200 | Proven | Matches Sawin bound |
| **H16: Multi-Quadratic CM16** | **2350** | **Proven** | **Surpasses Sawin** |

### The Tournament Arc

The progression from H1 to H16 demonstrates the evolutionary selection mechanism in action:

1. **H1-H2:** Establish the base (OpenAI derivation)
2. **H3:** First disqualification (contamination detection -- the anti-contamination protocol worked)
3. **H4-H6:** Incremental improvements (below benchmark but valid)
4. **H7-H8:** Major jumps (entering competitive range)
5. **H9-H10:** Failed experiments (necessary for completeness)
6. **H11:** First formal proof (milestone but low delta)
7. **H12-H14:** Rapid iteration (three failures in quick succession)
8. **H15:** Breakthrough (matches Sawin)
9. **H16:** Surpasses Sawin (the tournament champion)

---

## GRPO vs. Other Optimization Methods

| Method | Comparison Basis | Exploration | Adaptation | Used In |
|--------|-----------------|-------------|------------|---------|
| **GRPO** | Relative within group | High (multiple candidates) | Fast (group dynamics) | Co-Scientist, unit-distance |
| **RLHF** | Fixed reward model | Low (single sample) | Slow (reward model is static) | Standard LLM training |
| **PPO** | Advantage estimation | Medium (clipped updates) | Medium | General RL |
| **Evo-Search** | Population-based | High (mutation/recombination) | Medium (no policy gradient) | Genetic algorithms |

GRPO's advantage for multi-agent research: it combines the exploration of evolutionary search with the optimization efficiency of policy gradient methods, all operating at inference time rather than training time.

---

## Cross-References

- [[unit-distance-channel-protocol]] -- The RANK state that implements the tournament selection
- [[unit-distance-steganographic-cot]] -- How steering vector optimization could use GRPO to refine steganographic patterns
- [[unit-distance-tokenizer-analysis]] -- The tokenizer infrastructure that enables structured tournament output

---

*This page documents the theoretical framework connecting GRPO, tournament inference, and the multi-agent research methodology. The core concepts (GRPO, Elo, evolutionary selection) are established in the literature. The application to persona steering vector optimization is a proposal discussed in the DeepSeek transcript and awaits empirical validation.*