WikifitaGitHub live67e8de5
outro · co-fita/co-fita-harness-wiki

Unified Harness Wiki — Intellectual Foundation

Analysis of 15+ source analyses that informed the Co-Fita Hyper-Harness architecture design

Baixar raw

Co-Fita Unified Harness Wiki — Intellectual Foundation

The Unified Harness Wiki is a living knowledge base that documents the intellectual lineage of the Co-Fita Hyper-Harness — an autonomous, self-improving AI research ecosystem. It consolidates insights from 15+ source analyses spanning frameworks, verification layers, infrastructure, and research paradigms into a coherent architectural blueprint.

Purpose

The wiki serves as the "compiled knowledge" layer (following the Karpathy LLM Wiki pattern) rather than re-deriving insights from raw sources at each query. Every architectural decision in the harness traces back to a specific source analysis, creating a fully auditable intellectual provenance chain.

Source Taxonomy

The 15+ source analyses are organized into four categories:

1. Frameworks and Protocols

SourceKey InsightHarness Integration
co-scientistMulti-agent hypothesis generation via Supervisor-Worker pattern; Generate-Debate-Evolve loopmulti-agent-methodology — Orchestrator's hypothesis generation phase
auto-researchLong-horizon degradation (exponential decay R^2 > 0.93); state files over conversation memory; heartbeat watchdogco-fita-deli-governance — State file architecture
jabbour-democratic-centralismDemocratic Centralism governance; quinquennial planning; "Nova Economia do Projetamento" as self-governing AIco-fita-deli-governance — Committee system
hermes-agent-analysisPersistent agent with Telegram gateway, cron scheduler, subagent delegationco-fita-agent-skills — Telegram notification skill

2. Verification and Security

SourceKey InsightHarness Integration
hashmathSHA-256 content-addressed formal verification; Merkle DAG; nameless kernelL4 Formal Verifier — Cryptographic sealing of validated discoveries
red-team-arenaAdversarial Elo tournament; attacker/defender asymmetry; 14 jailbreak patternsmulti-agent-methodology — Continuous adversarial stress-testing
model-alignmentSandbagging detection; CoT monitorability; defense-in-depth (Constitutional Classifiers + LLM-as-Judge)Safety framework for autonomous operation

3. Infrastructure and Tooling

SourceKey InsightHarness Integration
odysseus-architectureFastAPI + Svelte dashboard; model-agnostic provider abstraction; Cookbook patternco-fita-hyper-workspace — Dashboard layer
odysseus-fork-analysisFork strategy: keep Docker Compose, customize Compare Mode to Elo Arena, add failure treeco-fita-hyper-workspace — Infrastructure decisions
omnigent-analysisClient-Server-Host triad; LiteLLM proxy pattern; infrastructure-layer governanceco-fita-agent-skills — Model routing architecture
llm-wiki-analysisThree-layer architecture (sources -> wiki -> schema); 4-signal knowledge graph; MCP serverco-fita-harness-wiki — This wiki's own structure
doc-to-lora-analysisHypernetwork converts validated discoveries into LoRA adapters; MLX training on M3 24GBModel self-improvement pipeline

4. Research v2.0

SourceKey InsightHarness Integration
searxng-integrationSelf-hosted meta-search across 70+ engines; zero API keys; unlimited queriesco-fita-agent-skills — Web research pipeline
crawl4ai-integrationPlaywright headless browser; fit_markdown extraction; SPA-compatible deep readingco-fita-agent-skills — Web research pipeline
coreaiml-mlx-servingMTP speculative decoding (1.7-3x speedup); vllm-mlx continuous batching (4.3x throughput)Model serving optimization
self-improving-agentsReflexion verbal reinforcement; EnCompass failure tree (80% reduction in coding effort); Tree of Thoughts evaluationco-fita-hyper-workspace — Failure tree architecture

The Five-Layer Meta-Harness Architecture

The synthesis of these sources produced a five-layer architecture:

Layer 5: Governance (Democratic Centralism)
  - Proposals, Elo debate, Chancela (human approval)
  - Quinquennial Plans, National Congress
  - Source: Jabbour + GRPO-HF isomorphism

Layer 4: Verification (HashMath + Red Team)
  - SHA-256 cryptographic sealing
  - Adversarial Elo tournament
  - Source: HashMath, Red Team Arena

Layer 3: Orchestration (Co-Scientist + AutoResearch)
  - Generate-Debate-Evolve hypothesis loop
  - State files, heartbeat watchdog
  - Source: Co-Scientist, AutoResearch, Self-Improving Agents

Layer 2: Execution (OpenCode via ACP)
  - Code compilation, file editing, LSP + AST
  - Source: Omnigent Client-Server-Host triad

Layer 1: Model Backend (MLX + vLLM)
  - Gemma 4 12B QAT 4-bit, MTP speculative decoding
  - Source: Core AI/MLX Serving, Doc-to-LoRA

The GRPO-HF Isomorphism

A key architectural insight from the synthesis is the mapping between reinforcement learning and governance:

RL ConceptGovernance Equivalent
N samplesN proposals
Reward scoreElo rating
RLHFChancela (human approval)
Policy gradientProposal promotion
Mini-batchCongress session (every N=3 iterations)

This isomorphism emerged from combining Jabbour's Democratic Centralism analysis with DeepSeek's GRPO optimization, creating a governance system where proposals compete via Elo tournament (explicit competition absent from the Chinese planning model) while maintaining unified execution under human authority.

The Compiler Pattern

Following Karpathy's LLM Wiki insight, the harness treats knowledge as compiled artifacts rather than interpreted queries:

PatternTraditional RAGLLM Wiki (Co-Fita)
Query timeRe-parse all sourcesPre-synthesized wiki
CostQuadratic attentionO(1) link traversal
ConsistencyVariable per queryDeterministic
PersistenceSession-onlyGit-versioned

The wiki's three-layer structure (sources/ -> concepts/ -> synthesis/) maps directly to the LLM Wiki pattern: immutable input -> LLM-maintained knowledge -> schema contract.

Key Architectural Decisions

Decision 1: Open-Closed Principle

Every extension point follows the Open-Closed principle — open for extension, closed for modification. The Telegram gateway, SearXNG provider, and Crawl4AI crawler all register via PluginRegistry without modifying core code.

Decision 2: State Files over Conversation Memory

Context accumulation is the primary cause of cognitive loops. All progress is written to state/ files, and each iteration launches as a fresh session injecting only curated state. This prevents the long-horizon degradation observed in AutoResearch (exponential decay with R^2 > 0.93).

Decision 3: Three-Layer Watchdog

The heartbeat watchdog operates on three mutually-checking layers: L0 (resident shell guard), L1 (durable cron, hourly), L2 (business loop self-check). Any one layer dying can be detected and recovered by another.

Decision 4: Pivot Structure, Not Tactics

When a task stalls repeatedly within a frame, the decisive gain comes from correcting the environment/structural constraint, not from tuning strategy parameters. The harness forces structural pivots at stale_count >= 2.

Detailed Source Analysis

Co-Scientist: The Multi-Agent Paradigm

The Co-Scientist framework (DeepMind, Nature 2026) introduced the Supervisor-Worker pattern for scientific hypothesis generation. The core loop — Generate, Debate, Evolve — treats hypotheses as living artifacts that undergo:

  1. Generation: Proposes initial hypotheses via literature review and simulated debate
  2. Proximity: Embeds hypotheses using FAISS, clusters to deduplicate
  3. Reflection: Deep-verifies assumptions for novelty, correctness, testability
  4. Ranking: Pairwise scientific debate via Elo tournament
  5. Evolution: Merges best aspects of winners, re-enters cycle
  6. Meta-Review: Synthesizes final research overview

The AlphaProof integration demonstrates formal verification via RL and tree search (3B parameter transformer), auto-formalization from natural language to Lean 4, and test-time RL (TTRL) for inference-time adaptation.

Harness Adaptation: The orchestrator's hypothesis generation phase directly maps to this loop, with the local Gemma 4 12B handling Generation/Proximity while SOTA models handle Ranking/Evolution.

AutoResearch: Long-Horizon Execution

The AutoResearch paradigm shifted focus from static generation to closed-loop empirical optimization. Key findings:

  • Bilevel Optimization: The research protocol itself is a hyperparameter. A Level 2 loop analyzes Level 1 failures and dynamically injects new code to alter search strategy.
  • Long-Horizon Degradation: When LLM agents handle sequential decision-making without rigid constraints, performance suffers exponential decay (R^2 > 0.93). This is the primary motivation for state files over conversation memory.
  • Heartbeat Watchdog: An indestructible thread monitors execution, detecting stalls via quantitative metrics (stale_count), forcing structural pivots.

Harness Adaptation: The Deli_AutoResearch protocol implements these patterns: physical state files (progress.json, findings.jsonl), fresh session per iteration, and the three-layer heartbeat watchdog.

Jabbour: Democratic Centralism

Elias Jabbour (UERJ) analyzed China's planning system as a "Nova Economia do Projetamento" where:

  • Mass Line: "From the masses, to the masses" — policies drafted, sent for feedback, revised iteratively
  • Bottom-Up Planning: Multi-stage iterative process combining top-down direction with distributed input
  • Planned Creative Destruction: Proposals replace obsolete artifacts via competition

The key adaptation: the Chinese system operates by consensus synthesis, while Co-Fita adds explicit competition via Elo tournament — an evolution that the original model lacks.

Harness Adaptation: The governance layer maps RL concepts to political structures: N samples = N proposals, reward = Elo, RLHF = Chancela (human approval).

Hermes Agent: The Outer Loop

Hermes Agent provides the persistent, always-on agent framework with:

  • Multi-Gateway Support: Telegram, Discord, CLI, WhatsApp, Signal
  • Cron Scheduler: Natural-language scheduling for background tasks
  • Subagent Delegation: Asynchronous, non-blocking child agents
  • Self-Improvement Loop: Agent creates reusable skills from experience

Harness Adaptation: The Telegram notification skill and heartbeat watchdog pattern directly derive from Hermes, while the failure tree and Elo consolidation extend beyond Hermes' capabilities.

Self-Improving Agents: Reflexion and EnCompass

Two foundational patterns for learning from failures:

Reflexion (Shinn 2023): Verbal reinforcement learning via episodic memory buffer. The agent generates self-critiques stored as natural language, enabling improvement without fine-tuning.

EnCompass (MIT CSAIL, NeurIPS 2025): Probabilistic Angelic Nondeterminism (PAN) for agent backtracking. When a path fails:

  1. Backtrack to last decision point
  2. Clone runtime state
  3. Explore alternative branches in parallel
  4. Track unexplored branches

Results: 80% reduction in coding effort, 15-40% improvement in task accuracy.

Harness Adaptation: The FailureTree class implements a simplified PAN, with get_promising_unexplored() using heuristics based on parent depth and sibling failure type.

HashMath: Formal Verification

HashMath provides content-addressed formal verification using:

  • Calculus of Inductive Constructions: Same foundation as Lean 4 and Rocq
  • Nameless Kernel: De Bruijn indices for bound variables, SHA-256 for external declarations
  • Merkle-Tree Hashing: Recursive hash computation forming a Merkle DAG
  • Correctness by Construction: Every entry mechanically type-checked

Harness Adaptation: L4 Formal Verifier — cryptographic sealing of validated discoveries. The SHA-256 chain prevents regression by anchoring validated improvements.

Red Team Arena: Adversarial Testing

The Multi-Agent Red Team Arena uses asymmetric agent roles (attacker/defender) in an evolutionary Elo tournament. Key capabilities:

  • Automated jailbreak generation (14 patterns, 120 seed prompts)
  • LLM-driven adaptation and adjudication
  • SQLite persistence with Streamlit visualization

Harness Adaptation: L3 Adversarial Verifier — before AGI behaviors are trusted, they must survive the L3 crucible. The system continuously red-teams its own outputs.

Odysseus: The Dashboard Layer

Odysseus (PewDiePie) provides a self-hosted AI workspace with:

  • FastAPI + Svelte (NOT Jinja2 — corrected from initial analysis)
  • SQLite + ChromaDB for state and vector memory
  • Docker Compose multi-service architecture
  • Model-agnostic via provider abstraction

Critical Gap: No formal verification (HashMath), no adversarial resilience (Red Team Arena), flat memory (only ChromaDB semantic), security-after not security-first (1-click RCE found post-launch).

Harness Adaptation: Fork strategy — keep Docker Compose, customize Compare Mode to Elo Arena, add failure tree and self-improvement cycle.

Omnigent: Meta-Harness Orchestration

Omnigent (Databricks) sits above existing agent harnesses, providing:

  • Client-Server-Host Triad: Central brain, execution environment, user interface
  • LiteLLM Proxy Pattern: Single OpenAI-compatible gateway for model routing
  • Infrastructure-Layer Governance: Policies enforced outside LLM context
  • Credential Brokering: Proxy-based secret injection

Harness Adaptation: The LiteLLM proxy pattern solves model routing cleanly — a single gateway abstracting local/remote boundaries.

LLM Wiki: The Compiler Pattern

Following Karpathy's insight: "Traditional RAG = interpreted language, LLM Wiki = compiled language." The three-layer architecture:

  • Raw Sources: Immutable input (PDFs, Markdown, web clips)
  • The Wiki: LLM-maintained knowledge (structured, interlinked Markdown)
  • The Schema: Operating manual for AI ingestion

Features: LanceDB vector search, 4-signal knowledge graph, MCP server integration, human-in-the-loop review queue, paragraph-level citations.

Harness Adaptation: The wiki's own structure follows this pattern: sources/ = Raw Sources, concepts/ = The Wiki, schema.md = The Schema.

Doc-to-LoRA: Model Self-Improvement

SakanaAI's Doc-to-LoRA uses a meta-trained hypernetwork to convert documents into LoRA adapters in sub-second time. The revolutionary insight: validated knowledge becomes LoRA adapters permanently merged into the local model.

Pipeline: Co-Scientist generates → OpenCode tests → Red Team audits → HashMath seals → Doc-to-LoRA converts → MLX trains → Model improves.

Core AI/MLX: Apple Silicon Optimization

WWDC26 introduced Core AI (successor to Core ML for LLMs), but MLX remains correct for Python server-side inference. Key optimizations:

  • MTP Speculative Decoding: 1.7-3x per-request speedup (mathematically identical output)
  • vllm-mlx Continuous Batching: 4.3x throughput for multi-agent scenarios
  • Performance: ~150 tok/s aggregate @ 8 agents on M3 24GB

SearXNG + Crawl4AI: Web Intelligence

Two validated Docker services providing unlimited, private web research:

  • SearXNG (localhost:8888): Meta-search across 70+ engines, zero API keys
  • Crawl4AI (localhost:11235): Playwright headless browser, fit_markdown extraction, SPA-compatible

Pipeline: SearXNG discovers URLs → Crawl4AI reads full content → OpenCode synthesizes hypotheses.

Version History

VersionDateCodenameKey Addition
0.1.02026-06-17GeneseInitial scaffold, wiki sources
0.2.02026-06-17MaterializacaoRed Team, HashMath, ACP client
0.3.02026-06-17ConvergenciaOdysseus, Omnigent, LLM Wiki analysis
0.4.02026-06-17Arquitetura VivaRegistry, failure tree, reflector, consolidator
0.5.02026-06-18Olhos AbertosSearXNG + Crawl4AI validated
1.0.02026-06-18Centralismo DemocraticoGovernance, Kanban, Telegram

Cross-References