---
name: wikifita-site-llm-integration
type: analysis
title: "Wikifita Site LLM Integration"
description: "How the Wikifita Atlas exposes structured plain-text endpoints for LLM consumption: site-wide indexes, per-page summaries, and full lossless context dumps."
tags: [wikifita, llm, llms-txt, agent-consumption, plain-text, knowledge-base]
timestamp: 2026-07-21
---

# Wikifita Site LLM Integration

The Wikifita Atlas provides a layered set of plain-text endpoints designed for consumption by LLMs and AI agents. These endpoints follow the emerging `llms.txt` convention (inspired by `robots.txt` and `sitemap.xml`) to make the knowledge base machine-readable without requiring HTML parsing.

The design philosophy aligns with the wikifita principle that knowledge should be accessible to both humans and agents. The site serves as a bridge: humans interact with the rendered UI, while agents consume the structured text endpoints.

---

## Endpoint Hierarchy

```mermaid
flowchart TD
    A[LLM Agent] --> B{/llms.txt}
    A --> C{/llms-full.txt}
    A --> D{/wiki/page/llms.txt}
    A --> E{/wiki/page/llms-full.txt}

    B -->|Reduced index| F[Page list by category with titles + URLs]
    C -->|Full dump| G[All raw markdown with metadata headers]
    D -->|Per-page summary| H[Page metadata + links + backlinks + excerpt]
    E -->|Per-page full| I[Page metadata + raw markdown + link resolution]

    F -->|Entry point| J[Agent discovers pages]
    J -->|Picks page| D
    D -->|Needs full content| E
    C -->|Bulk ingestion| K[Agent loads entire corpus]
```

---

## Endpoint 1: `/llms.txt` (Site Index)

**Content-Type:** `text/plain; charset=utf-8`
**Cache-Control:** `no-store`

A reduced site index listing all pages organized by category. This is the entry point for agents discovering the knowledge base.

### Output Format

```
# Wikifita Atlas

> Atlas editorial da Wikifita, a base privada e versionada de memória persistente...

## Principais
- [Atlas](https://wikifita.aleffita.dev)
- [Busca](https://wikifita.aleffita.dev/search)
- [Contexto completo lossless](https://wikifita.aleffita.dev/llms-full.txt)

## Memórias
- [Memory Index](https://wikifita.aleffita.dev/wiki/memorias/MEMORY.md): Memory index page

## Projetos
- [Project Alpha](https://wikifita.aleffita.dev/wiki/projetos/alpha): Project description

## Pesquisas
- [Research Topic](https://wikifita.aleffita.dev/wiki/pesquisas/topic): Research description
```

### Suppression Rules

Pages are excluded from `/llms.txt` if their filename stem (without extension) matches:
- `claude`
- `agents`
- `prefill`
- `template`

This prevents internal configuration files from appearing in the public LLM index.

### Section Names

Categories are translated to Portuguese section headers:

| Category | Section Name |
|----------|-------------|
| `memory` | Memórias |
| `project` | Projetos |
| `research` | Pesquisas |
| `directive` | Diretivas |
| `person` | Pessoas |
| `other` | Outros |

---

## Endpoint 2: `/llms-full.txt` (Full Corpus Dump)

**Content-Type:** `text/plain; charset=utf-8`
**Cache-Control:** `no-store`

A complete dump of every wiki page's raw markdown, delimited by structured metadata headers. This is the "lossless" endpoint -- the raw content is preserved byte-for-byte, including frontmatter, whitespace, and newlines.

### Output Format

```
# Wikifita Atlas — Full Lossless Context

> Extensão lossless da Wikifita para leitores autorizados e agentes de pesquisa.

Cada documento abaixo é delimitado por path, URL, byte length e hash SHA-256...

--- WIKIFITA_DOCUMENT path="memorias/MEMORY.md" url="https://wikifita.aleffita.dev/wiki/memorias/MEMORY.md" bytes="1234" sha256="abc123..." ---
# Memory Index
---
type: memory
title: "Memory Index"
...
---

Body content here...

--- END_WIKIFITA_DOCUMENT path="memorias/MEMORY.md" ---

--- WIKIFITA_DOCUMENT path="projetos/alpha.md" url="https://wikifita.aleffita.dev/wiki/projetos/alpha" bytes="5678" sha256="def456..." ---
...
--- END_WIKIFITA_DOCUMENT path="projetos/alpha.md" ---
```

### Document Delimiters

Each document is wrapped in structured delimiters that enable agents to:
- Parse individual documents from the stream
- Verify content integrity via SHA-256 hash
- Know the canonical URL and byte length
- Extract the raw markdown including frontmatter

The delimiter format:
```
--- WIKIFITA_DOCUMENT path="{path}" url="{canonicalUrl}" bytes="{byteLength}" sha256="{hash}" ---
{raw markdown}
--- END_WIKIFITA_DOCUMENT path="{path}" ---
```

### Use Cases

- **Bulk ingestion**: An agent can fetch `/llms-full.txt` once and have the entire knowledge base in structured form
- **RAG pipeline seeding**: The document delimiters enable clean chunking for vector store insertion
- **Content verification**: SHA-256 hashes allow agents to verify they have the canonical version

---

## Endpoint 3: `/wiki/{path}/llms.txt` (Per-Page Summary)

**Content-Type:** `text/plain; charset=utf-8`
**Cache-Control:** `no-store`

A structured summary of a single page, designed as a lightweight context window for agents working with a specific document.

### Output Format

```
# Page Title

> Wikifita page context for https://wikifita.aleffita.dev/wiki/path/to/page

- canonical: https://wikifita.aleffita.dev/wiki/path/to/page
- raw: https://wikifita.aleffita.dev/raw/path/to/page.md
- path: path/to/page.md
- type: reference
- category: project
- sha256: abc123...
- byte_length: 1234
- description: One-line description
- tags: tag1, tag2

## Summary

First 240 characters of the page body or description...

## Links
- [Link Label](https://wikifita.aleffita.dev/wiki/target) -> resolved
- [Broken Link](missing-page) -> missing

## Backlinks
- [Source Page](https://wikifita.aleffita.dev/wiki/source)

## Media References
- image: https://cdn.example.com/photo.png (Photo caption)
- embed: https://youtu.be/ID (Video title)

## Full Context

- [Lossless raw page context](https://wikifita.aleffita.dev/wiki/path/to/page/llms-full.txt)
```

### Metadata Block

The header provides machine-readable metadata:
- `canonical`: Full URL for the page
- `raw`: URL to download the raw markdown
- `path`: Internal file path
- `type`: OKF type from frontmatter
- `category`: Computed category (person, project, research, directive, memory, other)
- `sha256`: Content hash for integrity verification
- `byte_length`: Exact byte count
- `description`: From frontmatter (if present)
- `tags`: Comma-separated tag list

### Link Resolution

Links in the output are resolved to their canonical state:
- Resolved links show the canonical URL
- Missing links show the state string `"missing"`
- External links show the state string `"external"`

### Backlinks

The backlinks section lists all pages that link to the current page, with their canonical URLs. This enables agents to understand the document's position in the knowledge graph.

---

## Endpoint 4: `/wiki/{path}/llms-full.txt` (Per-Page Full Context)

**Content-Type:** `text/plain; charset=utf-8`
**Cache-Control:** `no-store`

The full lossless representation of a single page, including the raw markdown and complete link resolution data.

### Output Format

```
# Page Title — Full Lossless Context

--- WIKIFITA_PAGE path="path/to/page" url="https://..." raw="https://..." bytes="1234" sha256="abc..." blob="def..." ---
source="github" state="live" commit="abc123" loaded_at="2026-07-21T..."
links=[{"target":"...","label":"...","state":"resolved","destination":"target.md","canonicalUrl":"https://..."}]
media=[{"url":"https://...","kind":"image","allowed":true}]

# Full raw markdown content including frontmatter...

--- END_WIKIFITA_PAGE path="path/to/page" ---
```

### Structured Metadata Header

The `WIKIFITA_PAGE` header contains:
- `path`: Internal file path
- `url`: Canonical URL
- `raw`: URL to raw markdown
- `bytes`: Byte length
- `sha256`: Content hash
- `blob`: Git blob SHA

The second line provides source metadata:
- `source`: `github` or `snapshot`
- `state`: `live`, `cached`, or `degraded`
- `commit`: Git commit SHA
- `loaded_at`: ISO timestamp of corpus load

The `links` and `media` fields are JSON-encoded arrays with full resolution data, enabling agents to:
- Walk the link graph programmatically
- Understand which links are resolved, missing, or external
- Access canonical URLs for each linked page
- Identify media references and their types

---

## Routing Mechanism

### Proxy Rewrite

Per-page LLM endpoints are accessible via two URL patterns:

1. **Proxy pattern**: `/wiki/{path}/llms.txt` and `/wiki/{path}/llms-full.txt`
2. **API pattern**: `/api/page-llms/llms/{path}` and `/api/page-llms/llms-full/{path}`

The `proxy.ts` middleware intercepts requests matching the proxy pattern and rewrites them to the API pattern. This allows agents to discover per-page LLM endpoints by appending `/llms.txt` to any wiki page URL, following a consistent convention.

```typescript
// proxy.ts
const match = pathname.match(/^\/wiki\/(.+)\/(llms(?:-full)?\.txt)$/);
// Rewrites: /wiki/path/to/page/llms.txt -> /api/page-llms/llms/path/to/page
```

### Content Negotiation

The per-page endpoint uses the `mode` path segment to determine output format:

| Mode | Function | Output |
|------|----------|--------|
| `llms` | `pageLlmsText()` | Structured summary with metadata, links, backlinks |
| `llms-full` | `pageLlmsFullText()` | Full lossless context with raw markdown |

---

## Integration with Wikifita Philosophy

### Agent-Consumable Knowledge

The LLM endpoints embody the wikifita principle that a knowledge base should be accessible to both humans and agents:

- **Humans** use the rendered UI (Next.js + D3 graph + search)
- **Agents** use the plain-text endpoints (`/llms.txt`, `/llms-full.txt`, per-page)
- **Both** consume the same canonical source (the git repository)

### Layered Access

The endpoint hierarchy provides layered access:

| Layer | Endpoint | Content | Use Case |
|-------|----------|---------|----------|
| Discovery | `/llms.txt` | Page list by category | Agent finds relevant pages |
| Summary | `/wiki/{path}/llms.txt` | Metadata + links + excerpt | Agent understands page context |
| Full | `/wiki/{path}/llms-full.txt` | Raw markdown + full metadata | Agent ingests complete content |
| Bulk | `/llms-full.txt` | All pages concatenated | Agent loads entire corpus |

### Content Integrity

Every endpoint includes integrity signals:
- `X-Wikifita-Commit`: Git commit SHA for the current corpus
- `X-Wikifita-Source-State`: Whether the data is live, cached, or degraded
- `ETag`: Deterministic hash for cache validation
- SHA-256 hashes in document delimiters for content verification

This enables agents to verify they are working with canonical, unmodified content.

---

## Filtering and Suppression

### Site-Wide Index (`/llms.txt`)

Suppresses pages with stems: `claude`, `agents`, `prefill`, `template`.

### Per-Page Summaries (`/wiki/{path}/llms.txt`)

Suppresses pages with stems: `claude`, `agents`, `prefill`, `template`.

Note: The full corpus dump (`/llms-full.txt`) does NOT suppress any pages -- it includes everything for completeness.

### Rationale

The suppressed filenames correspond to internal configuration files (like `CLAUDE.md`, `AGENTS.md`) that are part of the repository but not meaningful content for external agents. They are excluded from discovery endpoints but remain accessible via direct URL.

---

## Example Agent Workflow

An agent discovering and consuming a wiki page:

```
1. GET /llms.txt
   -> Discovers "Pesquisas" section
   -> Finds "Unit Distance Conjecture" page

2. GET /wiki/pesquisas/unit-distance/llms.txt
   -> Reads metadata: type=research, tags=[math, conjecture]
   -> Reads summary: first 240 chars
   -> Sees links to related pages
   -> Decides to read full content

3. GET /wiki/pesquisas/unit-distance/llms-full.txt
   -> Reads complete raw markdown
   -> Parses WIKIFITA_PAGE header for integrity
   -> Walks link array to find related documents

4. GET /raw/pesquisas/unit-distance.md
   -> Downloads raw markdown for local processing
   -> Verifies SHA-256 matches from step 3
```

---

## Limitations

1. **No authentication**: The LLM endpoints are publicly accessible. Sensitive content must be on the `personal` branch (never deployed).

2. **No pagination**: `/llms-full.txt` returns the entire corpus. For very large wikis, this could exceed agent context windows. The per-page endpoints provide a more granular alternative.

3. **No streaming**: All endpoints return the complete response at once. There is no SSE or chunked transfer for large payloads.

4. **No content transformation**: The raw markdown is served as-is. There is no summarization, chunking, or embedding generation at the endpoint level. Agents are expected to process the raw content themselves.

5. **Portuguese section names**: The `/llms.txt` index uses Portuguese section headers (Memórias, Projetos, etc.) reflecting the wiki's primary language. Agents should be aware of this for category-based navigation.

---

## Related Pages

- [[wikifita-site-architecture]] -- Full technical architecture
- [[wikifita-site-pipeline]] -- Wiki processing pipeline
- [[wikifita-site]] -- Project overview stub