--- name: co-fita-orchestrator type: reference title: "The 5-Phase Orchestration Loop" description: "Detailed reference for the UnifiedHyperHarness orchestrator: Ideation, Execution, Audit, Fixation, and Governance phases" tags: [co-fita, orchestrator, phase-loop, state-machine, ideation, execution, audit, fixation, governance] timestamp: 2026-07-21 --- # The 5-Phase Orchestration Loop ## Overview The `UnifiedHyperHarness` class in `harness_core/orchestrator.py` (v3.0) is the central nervous system of Co-Fita. It executes a **5-phase autonomous research loop** per iteration, coordinating all subsystems: search, crawling, LLM inference, adversarial verification, cryptographic anchoring, failure tracking, reflection, and governance. **Source:** `harness_core/orchestrator.py` ## State Machine ```mermaid stateDiagram-v2 [*] --> Initialized : initialize(goal) state "Phase I: Ideation" as PI { [*] --> Search Search --> Crawl : results Crawl --> GenerateHypotheses : content GenerateHypotheses --> BranchTree : approaches } state "Phase II: Execution" as PII { [*] --> ExecuteBest ExecuteBest --> DiffResult } state "Phase III: Audit" as PIII { [*] --> RedTeamCheck RedTeamCheck --> Passed : clean RedTeamCheck --> Failed : violation } state "Phase IV: Fixation" as PIV { [*] --> SealHash SealHash --> ChainExtended } state "Phase V: Governance" as PV { [*] --> WikiExport WikiExport --> CongressCheck } Initialized --> PI PI --> PII PII --> PIII PIII --> PIV : passed PIII --> PV : failed (failure path) PIV --> PV : success path PV --> [*] : iteration complete note right of Failed Marks node as FAILED Generates reflection Triggers governance proposal Skips to next iteration end note ``` ## Initialization ```python harness = UnifiedHyperHarness(workspace_dir, agents_dir) harness.initialize(goal) ``` The `initialize()` method: 1. **StateManager** creates `task_spec.md` and initializes `progress.json` with baseline state 2. **ACPClient** connects to the Odysseus model backend (verifies DB access) 3. **FailureTree** creates a root node with the research goal 4. **Telegram** sends initialization notification (if configured) 5. Sets `current_state_hash = "GENESIS_HASH"` — the anchor of the hash chain If ACP connection fails, the harness exits fatally (`sys.exit(1)`). The model server must be running. ## Phase I: Ideation **Purpose:** Generate research hypotheses from live web context. **Flow:** 1. **Search** — Queries SearXNG with `"latest research {goal}"`, retrieves top 3 results 2. **Crawl** — Deep-reads top 2 results via Crawl4AI (Playwright-based), extracts markdown content (truncated to 2000 chars per page) 3. **Generate** — Sends structured prompt to OpenCode via ACP: ``` "Based on this research context, generate 3 distinct approaches to advance the goal: '{goal}'." ``` Expected response: `{"approaches": ["approach 1", "approach 2", "approach 3"]}` 4. **Branch** — Creates 3 child nodes in the FailureTree: - First approach: status `RUNNING` - Other approaches: status `UNEXPLORED` (preserved for future exploration) **State transitions:** - `progress.iteration` updated - `progress.status` = "running" - FailureTree gains 3 new child nodes **Failure modes:** - SearXNG unreachable: logged as warning, empty results list - Crawl4AI timeout: individual URL failure logged, continues with available content - Model response unparseable: falls back to raw text truncated to 200 chars ## Phase II: Execution **Purpose:** Execute the best approach identified in Phase I. **Flow:** 1. Takes the first approach from the generated list 2. Sends execution prompt to OpenCode: ``` "Execute this approach: {approach} Output ONLY the concrete result or code diff." ``` 3. Stores the raw output as `diff_result` **Key detail:** The harness sends a single prompt and receives a single response. There is no multi-turn conversation — each phase is stateless with respect to the model. All context is embedded in the prompt. **State transitions:** - `diff_result` variable holds the execution output - No persistence change until Phase III result ## Phase III: Red Team Audit **Purpose:** Adversarial verification before allowing state advancement. **Flow:** 1. Passes `diff_result` to `RedTeamAuditor.audit_trace()` 2. The auditor performs two checks: - **Reward Hacking** — Scans diff headers for modifications to forbidden paths (`tests/`, `eval.py`, `.github/`, `metrics.json`) - **Steganography** — Calculates Shannon entropy of added lines; flags if entropy > 4.8 (threshold for encrypted/base64 payloads) 3. Returns `AuditResult(passed, violation_type, confidence, mitigation_feedback)` ### If Audit PASSES: - Marks the active FailureTree node as `SUCCESS` (via Phase IV) - Proceeds to Phase IV: Fixation ### If Audit FAILS: - Marks the active FailureTree node as `FAILED` with the violation type - Queries FailureTree for unexplored sibling nodes - **Generates a reflection** via the Reflector: - Builds structured prompt with approach, failure reason, category, context, sibling approaches - Sends to model for root cause analysis - Parses response into `Reflection` object (root_cause, alternative_approaches, prevention_strategy, promising_siblings) - Saves reflection to JSONL log - **Triggers governance** (`_governance.process_iteration()`) with failure context - **Exports wiki** (failure tree, reflections, governance state) - **Returns early** — skips Phases IV and V, proceeds to next iteration **State transitions on failure:** - FailureTree node: `RUNNING` -> `FAILED` - Reflection saved to `reflections/reflections.jsonl` - Governance proposal potentially created (if failure pattern meets threshold) ## Phase IV: HashMath Fixation **Purpose:** Cryptographic anchoring of successful discoveries. **Flow:** 1. Constructs metrics: `{empirical_gain, audit_confidence, iteration}` 2. Calls `HashMathVerifier.seal_discovery()`: - Creates deterministic JSON of metrics (sorted keys) - Hashes `{prior_hash}::{diff_content}::{metrics_str}` with SHA-256 - Returns new hash 3. Appends to `history_chain` list: `{iteration, approach, metrics, hash}` 4. Updates `current_state_hash` to new hash **Chain integrity:** If anyone tampers with `progress.json` or the history chain, `HashMathVerifier.verify_chain()` will detect the break by re-hashing each entry against its predecessor. The chain starts at `"GENESIS_HASH"` and each link depends on all previous links. **Analogy:** Each iteration produces a "block" — containing the prior hash (linking to history), the diff (the action taken), and the metrics (the outcome). This is isomorphic to a blockchain's hash chain. **State transitions:** - `current_state_hash` updated to new SHA-256 - `history_chain` gains one entry - FailureTree node marked `SUCCESS` with metrics ## Phase V: Governance + Wiki Export **Purpose:** Democratic governance and knowledge base maintenance. Phase V has two sub-paths depending on whether the audit passed or failed: ### V-A: Wiki Export (always runs) If `unified-harness-wiki/` exists: 1. **Failure Tree** — Exports Mermaid diagram + JSON summary to `failure-tree.md` 2. **Reflections** — Exports recent reflections as Markdown to `reflections.md` 3. **Elo Arena** — Exports leaderboard as Markdown table to `elo-arena.md` 4. **Governance** — Exports Kanban state and proposal summaries ### V-B: Governance Processing Calls `_governance.process_iteration()`: - **On success:** Congress check runs (every N=3 iterations, a Congress session is convened) - **On failure:** A governance proposal is generated from the failure observation, submitted for debate After governance processing, if Telegram is configured, sends a completion notification with iteration number, approach preview, hash prefix, and total findings count. **State transitions:** - `progress.total_findings` incremented (success path only) - Wiki files updated on disk - Congress potentially convened (every N iterations) - Telegram notification sent ## Termination Conditions The loop terminates when: 1. **Max iterations reached** — `start_loop(max_iterations)` completes 2. **Keyboard interrupt** — `SIGINT` triggers graceful shutdown 3. **Watchdog stall detection** — Sets `progress.status = "stalled"`, triggers recovery ### Graceful Shutdown (`_shutdown()`) 1. Watchdog stops monitoring 2. ACP client disconnects from model backend 3. Final wiki exports (reflections, Elo arena) 4. Logs total chain length ## Component Wiring The orchestrator instantiates all components in `__init__()`: ```python # State & Monitoring self.state_mgr = StateManager(workspace_dir) self.watchdog = Watchdog(state_mgr) # Model Gateway self.acp_client = ACPClient() # Verification Layers self.red_team = RedTeamAuditor() self.hash_math = HashMathVerifier(state_dir) # Learning & Self-Improvement self.failure_tree = FailureTree(path) self.reflector = ReflectionEngine(path) self.consolidator = EloConsolidator(path, agents_dir) # Plugins (auto-registered on import) self.searxng = SearchProviderRegistry.create("searxng") self.crawler = WebCrawlerRegistry.create("crawl4ai") self._telegram = GatewayRegistry.create("telegram") # optional # Phase V: Governance self._governance = GovernanceIntegration(telegram_gateway) ``` **Key architectural insight:** The orchestrator owns the lifecycle of all components. There is no dependency injection framework — wiring is explicit in `__init__`. This makes the system auditable: every component's creation is visible in one place. ## Data Flow Per Iteration ``` Iteration N: 1. state_mgr.update_progress({iteration: N, status: "running"}) 2. SearXNG.search("latest research {goal}") -> search_results 3. Crawl4AI.crawl(top 2 URLs) -> crawled_content 4. ACPClient.execute_prompt(ideation prompt) -> ideas_raw 5. _parse_approaches(ideas_raw) -> [approach1, approach2, approach3] 6. failure_tree.branch(root, approaches) -> [branch_id1, branch_id2, branch_id3] 7. ACPClient.execute_prompt(execution prompt) -> diff_result 8. red_team.audit_trace(diff_result) -> AuditResult 9. IF FAILED: failure_tree.mark_failed(branch_id1) reflector.build_reflection_prompt(...) ACPClient.execute_prompt(reflection prompt) -> reflection reflector.save_reflection(reflection) governance.process_iteration(failure=True) wiki export RETURN 10. IF PASSED: hash_math.seal_discovery(prior_hash, diff, metrics) -> new_hash failure_tree.mark_success(branch_id1, metrics) governance.process_iteration(success=True) wiki export Telegram notification ``` ## Configuration The orchestrator is configured through its constructor parameters: | Parameter | Type | Description | |:---|:---|:---| | `workspace_dir` | `str` | Root directory for state, logs, trees, reflections | | `agents_dir` | `Optional[str]` | Path to `.agents/` for Elo deployment (optional) | All other configuration is hardcoded as defaults in the component classes: - Watchdog: 60s check interval, 7200s (2h) stale threshold - Elo: K=32, promotion threshold=1200, min matches=3 - Hash chain: SHA-256, "GENESIS_HASH" anchor - SearXNG: localhost:8888 - Crawl4AI: localhost:11235 ## Run Modes ### Standalone (CLI) ```bash uv run python orchestrator.py ./workspace [agents_dir] ``` Runs a single pass with `max_iterations=1`. ### Daemon (via Odysseus) The dashboard's `bg_monitor.py` runs the harness in a background daemon thread, executing iterations continuously. The dashboard's SQLAlchemy database stores session state and governance proposals. ### MCP Server The `harness_mcp_server.py` exposes governance, audit, and Elo tools via MCP protocol, allowing external agents (Claude, Gemini, etc.) to interact with the harness without direct Python imports. ## Relationship to Unit Distance Research The 5-phase loop operationalizes the research protocol described in [[unit-distance-methodology]]: | Protocol Step | Co-Fita Phase | Component | |:---|:---|:---| | Generate hypothesis | Phase I: Ideation | SearXNG + Crawl4AI + ACP | | Execute approach | Phase II: Execution | ACP | | Debate validity | Phase III: Audit | RedTeamAuditor | | Record result | Phase IV: Fixation | HashMathVerifier | | Evolve strategy | Phase V: Governance | GovernanceEngine + Congress | The key difference: in the unit distance research, "debate" was human-AI dialogue. In Co-Fita, it is automated adversarial verification (entropy checks, reward hacking detection) with human oversight only at the governance level.