Co-Scientist Agent Roster

Overview

The Co-Scientist uses 7 agents: 1 Supervisor (orchestrator) + 6 specialized research agents. Each agent inherits from BaseAgent, receives an AgentDeps bundle (Config, DB connection, LLM provider, ToolRegistry), and implements an execute(task: Task) -> TaskResult interface.

The Supervisor never does research itself — it parses the goal, schedules tasks, and runs the main loop. The 6 specialized agents are invoked by the Supervisor through a SQLite-backed durable task queue with lease-based claiming and bounded asyncio concurrency.

flowchart LR
    subgraph "Model Tiers"
        Strong["Strong Model<br/>(Opus-class)"]
        Cheap["Cheap Model<br/>(Sonnet-class)"]
        Mini["Mini Model<br/>(Haiku-class)"]
    end

    subgraph "Agent -> Model Mapping"
        G["Generation"] --> Strong
        R["Reflection"] --> Strong
        E["Evolution"] --> Strong
        MRF["Meta-review.final"] --> Strong
        RKD["Ranking.debate"] --> Cheap
        RKP["Ranking.pairwise"] --> Cheap
        PG["parse_goal"] --> Cheap
        MRS["Meta-review.system"] --> Cheap
        J["judge"] --> Cheap
        CL["classifier"] --> Mini
    end

Agent Interactions

Every agent produces a TaskResult with a kind field that drives the Supervisor's follow-up scheduling rules:

Agent Result	Follow-up Action
`hypothesis_created`	Enqueue `Reflection::ReviewHypothesis` for each new hypothesis
`review_completed`	Enqueue `Ranking::AddToTournament`
`added_to_tournament`	Enqueue `Ranking::RunTournamentBatch` (focus on new hypothesis)
`tournament_match_complete`	Periodic `Proximity::UpdateProximityGraph` (every N matches)
`system_feedback_generated`	Feedback auto-injected into future Generation/Evolution prompts
`final_overview_generated`	Written to `data/artifacts/<session_id>/final/overview.md`

When the queue empties, the Supervisor's decide_next_steps method enqueues refinement work: tournament batches, evolution passes, and periodic meta-reviews.

1. Supervisor

Role: Goal parser, task scheduler, session lifecycle manager.

Source: co_scientist/agents/supervisor.py

Model: parse_goal model (cheap) for the initial goal-parse call.

Responsibilities

Goal parsing — Calls the LLM with parse_goal template and record_research_plan tool to extract a structured ResearchPlan:
- objective: atomic statement of what to investigate
- preferences: what the scientist values in a hypothesis
- constraints: explicit limits on scope/methodology
- idea_attributes: adjectives for strong hypotheses (3-6)
- domain_hint: optional field classification
Session bootstrap — Creates the session row, inserts initial Generation tasks (configurable n_initial, default 3), sets up the TokenBudget and ToolRegistry.
Main loop — Bounded asyncio worker pool (concurrency=4 default) that:
- Claims tasks from the SQLite queue via task_repo.claim_one()
- Dispatches to the appropriate agent via agents[t.agent].execute(task)
- Applies follow-up scheduling rules after each task
- Checks termination predicates after every task
- Runs decide_next_steps when the queue is idle (every ~10s)
Idle refinement — When the queue empties:
- Always enqueues one tournament batch
- Enqueues evolution when min_mature hypotheses exist (default 20)
- Enqueues meta-review every ~50 matches
Finalization — On termination, cancels pending tasks, runs MetaReviewAgent::GenerateFinalResearchOverview, writes the overview to disk, sets session status to done.

Termination Predicates

Reason	Condition
`BUDGET`	Token or USD budget exhausted
`WALL_CLOCK`	Session time deadline crossed
`ELO_STABLE`	Top-K unchanged for N snapshots within epsilon
`EXTERNAL`	User pressed pause/abort
`IDLE`	Queue drained and `decide_next_steps` returned 0

The StabilityTracker maintains recent EloSnapshot history. Stability requires: same top-K ID set across all N snapshots AND max per-ID Elo delta < epsilon across the window. Guards prevent premature stability declarations (configurable min_ideas_before_stable and min_matches_before_stable).

2. Generation Agent

Role: Proposes new hypotheses via literature review and simulated scientific debate.

Source: co_scientist/agents/generation.py

Model: Strong (Opus-class)

Thinking budget: 4000 tokens (literature), 8000 tokens (debate)

Max tool loop iterations: 8

Max output tokens: 8192

Pipeline

flowchart TD
    A[Task: CreateInitialHypotheses] --> B[Render generation.literature prompt]
    B --> C[Build system blocks + session context]
    C --> D[Run tool loop with record_hypothesis]
    D --> E[Agent searches literature:<br/>PubMed, ArXiv, Europe PMC, web]
    E --> F{Agent calls record_hypothesis?}
    F -->|Yes| G[Extract record from final tool_use block]
    F -->|No| H[Raise RuntimeError]
    G --> I[Filter citations to seen URLs]
    I --> J[Embed + nearest-neighbour dedup check]
    J --> K{Duplicate found?}
    K -->|Yes| L[Return existing hypothesis ID]
    K -->|No| M[Insert hypothesis row + FAISS commit]
    M --> N[Return TaskResult: hypothesis_created]

Prompt Structure (generation.literature.md)

The prompt template receives:

goal — from the ResearchPlan
preferences — scientist's criteria
articles_with_reasoning — instructions to use literature tools
instructions — "Propose ONE hypothesis... call record_hypothesis"

Key instruction to the agent: "An empty result set is positive evidence that the literature you searched for does not exist. When the goal requires a candidate with NO prior published evidence, empty searches CONFIRM novelty."

Tool Loop

The Generation agent uses run_tool_loop() which drives the assistant -> tool_use -> tool_result cycle:

Agent receives the prompt + available tools (web_search, web_fetch, pubmed_search, arxiv_search, europe_pmc_search, plus science-skills)
Agent calls tools in parallel (up to parallel_cap=4)
Tool results are collected and fed back as the next turn's user message
On the final allowed iteration, force_terminal_tool="record_hypothesis" ensures the agent commits instead of searching indefinitely
Terminal tool calls are short-circuited (not dispatched) — the record IS the answer

Dedup Mechanism

Before persisting, the agent:

Embeds the hypothesis text (title + summary) using the configured embedder
Searches the session's FAISS index for the nearest neighbour
If cosine similarity >= 0.92 (configurable dedup_cosine_threshold), returns the existing hypothesis ID instead of inserting a duplicate
Only commits to FAISS after a successful DB insert (atomic pair)

Citation Integrity

Every citation URL in the record_hypothesis output is validated against the set of URLs seen in tool_result outputs during the loop. Fabricated citations are silently stripped.

3. Reflection Agent

Role: Reviews hypotheses for novelty, correctness, and testability using literature evidence.

Source: co_scientist/agents/reflection.py

Model: Strong (Opus-class)

Thinking budget: 0 (full), 12000 (verification), 6000 (observation)

Max tool loop iterations: 8

Max output tokens: 4096

Review Modes

Mode	Prompt Template	Description
`full`	`reflection_review.md`	Critically reviews novelty, correctness, testability using literature tools
`verification`	`reflection_verification.md`	Decomposes hypothesis into core assumptions, evaluates each independently
`observation`	`reflection_observation.md`	Analyzes hypothesis against observations from a scientific article

Currently ships full mode. Verification and observation land in later milestones.

Prompt Structure (reflection_review.md)

1. Briefly summarize what the hypothesis claims.
2. Novelty — what is new relative to the literature? Cite specific articles.
3. Correctness — strongest evidence for and against? Flag internal inconsistencies.
4. Testability — propose at least one concrete experiment.
5. Verdict — exactly one of: already_explained, other_more_likely, missing_piece, neutral, disproved.

The verdict values map to the tournament's understanding of hypothesis quality:

already_explained — hypothesis consistent but causes are known
other_more_likely — could explain but better explanations exist
missing_piece — novel, plausible explanation (the ideal verdict)
neutral — neither explains nor contradicts
disproved — contradicted by strong literature evidence

Output

The record_review tool captures:

verdict (enum)
kind (full/verification/observation)
novelty, correctness, testability, feasibility (0-1 floats)
assumptions[] (for verification mode)
evidence[] — every claim must have a url and verbatim excerpt
notes — free-form analysis

Evidence URLs are filtered against seen_urls from the tool loop (same citation integrity as Generation).

State Transition

After a successful review, the hypothesis transitions from draft to reviewed (only if currently in draft — never regresses a ranked/evolved hypothesis).

4. Ranking Agent

Role: Manages the Elo tournament. Pairwise comparison and simulated expert debate.

Source: co_scientist/agents/ranking.py

Model: Cheap (Sonnet-class) for pairwise; Strong for debate

Thinking budget: 4000 (pairwise), 8000 (debate)

Max output tokens: 2048

Two Actions

Action	Description	LLM Call
`AddToTournament`	Initialize Elo + state for a hypothesis	No
`RunTournamentBatch`	Pick a pair, debate/compare, parse verdict, apply Elo	Yes

Pair Selection Algorithm

The pair selection uses a three-bucket strategy with FAISS similarity weighting:

Focus pair (if focus_id set): Match the new hypothesis against its nearest-Elo opponent
New-arrival (p_new=0.4): Pair a new hypothesis (<3 matches) with nearest-Elo warm hypothesis
Close-Elo (p_close=0.4): Among warm hypotheses with |Delta_Elo| < 200, sample weighted by exp(-Delta_Elo/200) * (1 - cosine_similarity) — prefer pairs that are close in Elo but distant in idea-space
Random top-heavy (p_random=0.2): Random pair from the top half of the leaderboard

Mode Selection

Condition	Mode
Either hypothesis has < 2 matches	`debate`
	Delta_Elo
Otherwise	`pairwise`

The Debate

The debate prompt (ranking_debate.md) simulates a panel of domain experts:

3-5 turns (max 10)
Clarifying questions, critical evaluation against goal and criteria
Final judgment: "better idea: 1" or "better idea: 2"

The pairwise prompt (ranking_pairwise.md) is a single-turn comparison with the same verdict format.

Elo Update

Standard Elo with K-factor scheduling:

Condition	K
Less experienced player has < 5 matches	K=32 (new)
Otherwise	K=16 (warm)

The update is zero-sum: elo_a_after = elo_a + delta, elo_b_after = elo_b - delta.

Match IDs are deterministic (derived from hypothesis IDs + task ID) to ensure crash-then-retry computes the same match_id, making Elo updates idempotent.

Batch API

For sub-decile tournament matches, the Ranking agent can enqueue matches into the BatchPool instead of running them synchronously. The BatchPool submits to Anthropic's Batch API (~50% cheaper, up to 24h latency). The high-Elo head of the leaderboard always runs synchronously.

5. Evolution Agent

Role: Combines, simplifies, and reimagines top-ranked hypotheses.

Source: co_scientist/agents/evolution.py

Model: Strong (Opus-class)

Thinking budget: 6000 (combine, out_of_box), 0 (feasibility, simplify)

Max tool loop iterations: 6

Max output tokens: 4096

Four Strategies

Strategy	Input	Process	Output
`combine`	Two most distant top hypotheses	Merge strongest mechanisms, resolve contradictions	New hypothesis with `parent_ids=[a, b]`
`simplify`	Top hypothesis	Strip ornamental elements, preserve load-bearing claim	New hypothesis with `parent_ids=[top]`
`feasibility`	Top hypothesis	Refine for practical implementability with current tech	New hypothesis with `parent_ids=[top]`
`out_of_box`	Top-5 hypotheses	Creative synthesis inspired by patterns across the set	New hypothesis with `parent_ids=[top-5]`

Maturity Gate

Evolution only fires when min_mature hypotheses exist (default 20). A hypothesis is "mature" when matches_played >= 3. This prevents premature evolution on a small, under-evaluated pool.

Combine Strategy Detail

The combine strategy picks the most idea-distant pair within the top-K using FAISS cosine similarity (lowest similarity = most different ideas). It then renders the evolution.combine.md prompt which instructs the agent to:

Identify the strongest mechanism in A and B
State explicit contradictions and how the combination resolves them
Propose the synthesized hypothesis with specific entities, mechanisms, and anticipated outcomes

Dedup

Evolution uses the same dedup mechanism as Generation: embed + nearest-neighbour check before persisting. This prevents Evolution from producing near-duplicates of existing hypotheses.

Output

Each strategy produces a new hypothesis row via record_hypothesis with strategy and parent_ids populated. The new hypothesis then cascades into the standard Reflection -> Ranking pipeline like any fresh idea.

6. Proximity Agent

Role: Embeds and clusters hypotheses. No LLM call.

Source: co_scientist/agents/proximity.py

Model: None (pure computation)

Thinking budget: N/A

Responsibilities

Batch embedding — Scans hypotheses that lack an embedding for the current model, embeds them in bulk, and adds to the FAISS index
Clustering — Runs sklearn AgglomerativeClustering on the cosine distance matrix with configurable threshold (default 0.15)
Dedup cluster assignment — Sets hypotheses.dedup_cluster for each hypothesis

When It Runs

The Proximity agent is triggered by the Supervisor periodically — every full_recluster_every_matches completed matches (default 20). It does NOT run on every new hypothesis (that would be too expensive). Instead, the per-hypothesis dedup is handled inline by Generation and Evolution.

Clustering Algorithm

AgglomerativeClustering(
    n_clusters=None,           # automatic
    metric="precomputed",      # cosine distance matrix
    linkage="average",         # average linkage
    distance_threshold=0.15,   # max cosine distance for merging
)

The distance matrix is derived from the FAISS cosine similarity matrix: dist = 1.0 - sim. The agent runs the clustering in a thread (asyncio.to_thread) to avoid blocking the event loop.

FAISS Store

The Proximity agent owns the session's FaissStore:

Backend: IndexFlatIP (inner product = cosine on L2-normalized vectors)
Embedder chain: Voyage -> OpenAI -> hash-fallback
Dimension: 1024 (default for voyage-3-large)
Atomic save/load with asyncio lock

7. Meta-Review Agent

Role: Periodic system feedback + final research overview.

Source: co_scientist/agents/metareview.py

Model: Cheap (Sonnet-class) for system feedback; Strong (Opus-class) for final overview

Thinking budget: 8000 (system feedback), 16000 (final overview)

Max output tokens: 4096 (system feedback), 8192 (final overview)

Two Actions

GenerateSystemFeedback

Analyzes all reviews and debate rationales to produce a meta-review:

Common weaknesses — patterns of critique across hypotheses
Common strengths — what the best hypotheses have in common
Suggested focus areas — where future generation should concentrate
Narrative — 1-2 paragraph synthesis

The feedback is stored as a SystemFeedback row and auto-injected into future Generation and Evolution prompts via the latest_system_feedback query. This creates a steering mechanism: the system learns from its own critiques and redirects hypothesis generation accordingly.

Frequency: Every ~50 completed matches (approximated by (feedback_count + 1) * 50).

GenerateFinalResearchOverview

At session termination, produces the markdown report:

# Executive summary (3-5 sentences)
# Main research directions
  For each: name, claim, supporting hypotheses, open questions, first experiment
# Convergence and divergence
# Caveats and limitations

Uses the top-10 hypotheses by Elo, their reviews, and the latest system feedback. The final overview is written to data/artifacts/<session_id>/final/overview.md and stored in sessions.final_overview.

No Tools

The final overview call uses no tools — it writes the markdown directly from the LLM's response. This is a deliberate design choice: by this point all evidence has been gathered by other agents, and the Meta-review agent's job is synthesis, not exploration.

Shared Infrastructure

BaseAgent

All agents inherit from BaseAgent which provides:

_system_prompt_header() — Prepends the safety preamble (SAFETY_PREAMBLE) to every agent's system prompt
_final_tool_use(response, tool_name) — Extracts the most recent tool_use block with the given name
_final_text(response) — Extracts all text blocks from a response

AgentDeps

@dataclass
class AgentDeps:
    cfg: Config
    db: aiosqlite.Connection
    llm: LLMProvider
    tools: ToolRegistry

Shared across all agents. The DB connection is the single connection per process (WAL mode serializes writes). The LLM provider handles retry, budget tracking, and cost estimation.

Tool Loop

The run_tool_loop() function drives the assistant -> tool_use -> tool_result cycle for Generation, Reflection, and Evolution:

Sends the initial spec (system + user blocks, tools, tool_choice)
If the response contains tool_use blocks, dispatches them in parallel (up to parallel_cap=4)
Collects tool_results and appends them to the message history
On the final iteration, forces tool_choice to the terminal tool (e.g., record_hypothesis)
Terminal tools are short-circuited — their input IS the answer, no dispatch needed
Tracks seen_urls across all tool results for citation validation
Raises ToolLoopExhausted if max_iters reached without a terminal tool call

Structured Output via Tool-Use Schemas

Rather than asking the LLM to "respond in JSON", the system uses Anthropic tool-use schemas for structured output. Each agent has one or more recording tools:

Tool	Used By	Purpose
`record_hypothesis`	Generation, Evolution	Capture structured hypothesis
`record_review`	Reflection	Capture structured review
`record_system_feedback`	Meta-review	Capture meta-analysis
`record_research_plan`	Supervisor	Capture parsed goal

This is the most reliable structured-output mechanism on the Anthropic API. The tool schemas are defined in co_scientist/agents/schemas.py.

References

See co-scientist for the system overview
See co-scientist-infrastructure for LLM providers, tools, FAISS, SQLite
See co-scientist-safety for injection defense and classifier
See co-scientist-evaluation for the bench system