Co-Fita Infrastructure Layer

The infrastructure layer of the Co-Fita ecosystem: a local-first AI research stack running on Apple M3 Pro hardware, orchestrated via Docker Compose, managed via uv workspace, and served through the Odysseus dashboard (FastAPI on port 7860).

The design principle: everything runs locally. No cloud dependencies for inference. SearXNG for search, ChromaDB for vector storage, vLLM-MLX for local model serving, and a unified dashboard to tie it together.

See also: co-fita-research-operations | co-fita-pam-setup | co-fita-clickfix-vaccine

---

1. System Architecture

┌─────────────────────────────────────────────────────────┐
│                    MacBook M3 Pro                         │
│                 24GB Unified VRAM                         │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │   Odysseus    │  │   SearXNG    │  │   ChromaDB   │  │
│  │  Dashboard    │  │   (Search)   │  │  (Vectors)   │  │
│  │  :7860        │  │   :8080      │  │   :8000      │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
│         │                 │                  │           │
│  ┌──────┴─────────────────┴──────────────────┴───────┐  │
│  │              Docker Compose Stack                  │  │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  │  │
│  │  │  vLLM-MLX  │  │  Crawl4AI  │  │  HuggingFace│  │  │
│  │  │ (Local LLM)│  │  (Reading)  │  │   Cache     │  │  │
│  │  │  :8080     │  │             │  │             │  │  │
│  │  └────────────┘  └────────────┘  └────────────┘  │  │
│  └───────────────────────────────────────────────────┘  │
│                                                          │
│  ┌───────────────────────────────────────────────────┐  │
│  │              Harness Core (Python)                 │  │
│  │  Orchestrator │ Plugins │ Watchdog │ Governance    │  │
│  └───────────────────────────────────────────────────┘  │
│                                                          │
└─────────────────────────────────────────────────────────┘

---

2. Odysseus Dashboard

The central web interface, built with FastAPI and served via uvicorn.

start-dashboard.sh

#!/bin/bash
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT_DIR/dashboard"
uv run python -m uvicorn app:app --host 0.0.0.0 --port 7860

The dashboard runs on port 7860 (not the default 7000) to avoid a conflict with Apple AirPlay, which uses port 7000 on macOS. This is documented in the startup script.

Key Configuration

Setting	Value	Notes
Port	7860	Avoids AirPlay conflict on 7000
Bind	0.0.0.0	Accessible from local network
Auth	Enabled by default	ODYSSEUS_ADMIN_USER / ODYSSEUS_ADMIN_PASSWORD
Database	SQLite	sqlite:///./data/app.db
Embeddings	FastEmbed	sentence-transformers/all-MiniLM-L6-v2
Pollers	1 in-process	Configurable via ODYSSEUS_INPROCESS_POLLERS
Upload limit	10 MB	ODYSSEUS_CHAT_UPLOAD_MAX_BYTES=10485760

Dashboard Capabilities

• Chat interface with local or remote LLMs
• Research pipeline integration (SearXNG + Crawl4AI)
• Vector search via ChromaDB
• Task management and monitoring
• GPU configuration support (AMD and NVIDIA via separate compose files)

---

3. Docker Compose Stack

The docker-compose.yml defines the core services:

Services

Service	Port	Purpose	Volumes
odysseus	7860	Dashboard (FastAPI)	data/, logs/, ssh/, huggingface/, local/
searxng	8080	Privacy-respecting search engine	Config volume
chromadb	8000	Vector database for embeddings	Data volume

Network Topology

• odysseus connects to searxng at http://searxng:8080 (environment variable SEARXNG_INSTANCE)
• odysseus connects to chromadb at chromadb:8000 (environment variables CHROMADB_HOST, CHROMADB_PORT)
• odysseus can reach host services via host.docker.internal:host-gateway (including Ollama at http://host.docker.internal:11434)

GPU Variants

• docker-compose.gpu-amd.yml -- AMD GPU configuration
• docker-compose.gpu-nvidia.yml -- NVIDIA GPU configuration

For the MacBook M3 Pro, neither GPU compose file is needed -- Metal acceleration is handled natively by vLLM-MLX.

Volume Persistence

Volume	Container Path	Purpose
data/	/app/data	Application database, config
logs/	/app/logs	Application logs
data/ssh/	/app/.ssh	SSH identity for remote Cookbook servers
data/huggingface/	/app/.cache/huggingface	HuggingFace model cache
data/local/	/app/.local	Installed Python packages (vLLM, llama-cpp-python)

---

4. LLM Backend

All model inference goes through a single gateway:

Harness Core → ACP Client → OpenCode (ACP) → mlx-lm / vllm-mlx (localhost:8080)

The architecture separates WHAT to ask (harness orchestration) from HOW to serve (model inference). The harness never calls model APIs directly -- it routes everything through the ACP (Agent Communication Protocol) client.

Local Model Serving

• Engine: vllm-mlx >= 0.3.0 (Apple Silicon optimized via MLX/Metal)
• Default endpoint: localhost:8080
• Alternative: Ollama at host.docker.internal:11434

Environment Variables for LLM Configuration

LLM_HOST=localhost              # Primary model endpoint
LLM_HOSTS=                       # Multiple model endpoints (comma-separated)
OPENAI_API_KEY=                  # For remote API fallback
OLLAMA_BASE_URL=                 # Ollama endpoint
RESEARCH_LLM_ENDPOINT=           # Dedicated research model
HF_TOKEN=                        # HuggingFace model access

---

5. uv Workspace Configuration

The Co-Fita ecosystem uses uv as its package manager with a workspace configuration:

[project]
name = "co-fita"
version = "1.0.0"
description = "Co-Fita Hyper-Harness + Odysseus -- Unified AI Agent Ecosystem"
requires-python = ">=3.13"
dependencies = [
    "pip>=26.1.2",
    "vllm-mlx>=0.3.0",
]

[tool.uv]
package = false

[tool.uv.workspace]
members = [
    "dashboard",
    "harness_core",
    "redhat-clickfix-report",
]

Workspace Members

Member	Path	Purpose
dashboard	dashboard/	Odysseus web interface
harness_core	harness_core/	Orchestrator, plugins, governance
redhat-clickfix-report	redhat-clickfix-report/	Forensic analysis tools

Why package = false

The workspace is an application, not a library. package = false tells uv not to treat it as a distributable Python package. Dependencies are managed at the workspace level; individual members can add their own dependencies as needed.

Python Version

Requires Python 3.13+ -- this is the minimum for full async support and the latest typing features used throughout the codebase.

---

6. Homebrew Private Tap

The aleffita/private Homebrew tap provides formulae for installing Co-Fita components on other machines.

Current Formulae

Formula	Description
pam-mac-setup	Touch ID for sudo configuration

Tap Configuration

# HTTPS (with PAT token)
brew tap aleffita/private https://github_pat_...@github.com/aleffita/homebrew-private.git

# SSH (cleaner)
brew tap aleffita/private git@github.com:aleffita/homebrew-private.git

The tap repository is at github.com/aleffita/homebrew-private. It contains Ruby formulae that reference the source repositories and install scripts to the appropriate locations.

---

7. Hardware Context

The infrastructure is designed for the MacBook M3 Pro:

Spec	Value	Implication
CPU	Apple M3 Pro (11-core)	Sufficient for local inference
GPU	14-core Metal	vLLM-MLX native acceleration
RAM	24 GB Unified	Large enough for 7B-13B parameter models
Storage	SSD	Fast model loading from HuggingFace cache

The Docker stack runs natively on macOS via Docker Desktop (or OrbStack). The Metal GPU is accessed directly by vLLM-MLX without passthrough configuration.

---

8. Deployment Flow

Fresh Machine Setup

# 1. Clone the workspace
git clone <repo-url> ~/co-fita
cd ~/co-fita

# 2. Install dependencies
uv sync

# 3. Start the Docker stack
docker compose up -d

# 4. Start the dashboard
./start-dashboard.sh

# 5. (Optional) Install PAM Touch ID
brew tap aleffita/private git@github.com:aleffita/homebrew-private.git
brew install pam-mac-setup
pam-mac-setup

Ongoing Operations

# Start dashboard
./start-dashboard.sh

# Update dependencies
uv sync

# Rebuild Docker stack
docker compose up -d --build

# Run harness core
cd harness_core && uv run python main.py

---

9. Design Principles

1. Local-first -- no cloud dependencies for core functionality. SearXNG replaces Google, vLLM-MLX replaces API calls, ChromaDB replaces cloud vector stores.

2. uv everywhere -- the workspace, individual packages, and scripts all use uv as the package manager. No pip, no poetry, no conda.

3. Port conflicts avoided -- the dashboard runs on 7860 (not 7000) to coexist with AirPlay. This is documented and enforced.

4. Volume persistence -- all state (models, databases, logs, configs) persists across container restarts via Docker volumes.

5. GPU agnostic -- the compose stack supports AMD, NVIDIA, and Apple Silicon via variant compose files. The default (no GPU compose) uses CPU + Metal.

6. Single model gateway -- the harness never calls model APIs directly. Everything goes through the ACP client, which abstracts the serving layer. This means swapping from vLLM-MLX to Ollama to a cloud API requires changing one configuration variable, not dozens of call sites.