--- name: wikifita-site-rendering-pipeline type: analysis title: "Rendering Pipeline Deep-Dive" description: "The 14-plugin unified pipeline that transforms markdown into sanitized HTML: remark-parse through rehype-stringify, with Mermaid validation, wiki link resolution, media embeds, KaTeX math, and syntax highlighting." tags: [wikifita, rendering, markdown, rehype, remark, mermaid, katex, sanitization, html] timestamp: 2026-07-21 --- # Rendering Pipeline Deep-Dive The Wikifita Atlas renders wiki pages from raw markdown to sanitized HTML using a 14-plugin `unified` pipeline. This pipeline runs server-side on every request (the site is fully `force-dynamic`) and produces the final HTML that browsers and agents consume. The rendering function `renderMarkdown` lives in `lib/markdown.ts` (lines 290-309). It operates on `page.body` -- the markdown content after frontmatter removal -- not on the raw file. --- ## Pipeline Architecture ```mermaid flowchart TD A["page.body (markdown string)"] --> B["1. remarkParse"] B --> C["2. remarkGfm"] C --> D["3. remarkMath"] D --> E["4. remarkMermaidGuard"] E --> F["5. remarkWikiLinks"] F --> G["6. remarkCanonicalLinks"] G --> H["7. remarkRehype (MDAST -> HAST)"] H --> I["8. rehypeRaw"] I --> J["9. rehypeSanitize"] J --> K["10. rehypeWikiLinkClasses"] K --> L["11. rehypeMediaEmbeds"] L --> M["12. rehypeKatex"] M --> N["13. rehypeHighlight"] N --> O["14. rehypeStringify"] O --> P["HTML string"] style B fill:#2a2a3a style C fill:#2a2a3a style D fill:#2a2a3a style E fill:#2a2a3a style F fill:#2a2a3a style G fill:#2a2a3a style H fill:#2a2a3a style I fill:#2a2a3a style J fill:#2a2a3a style K fill:#2a2a3a style L fill:#2a2a3a style M fill:#2a2a3a style N fill:#2a2a3a style O fill:#2a2a3a ``` The pipeline operates in two phases: 1. **Remark phase** (plugins 1-6): Transforms the Markdown AST (MDAST) 2. **Rehype phase** (plugins 7-14): Converts to HTML AST (HAST), sanitizes, and serializes --- ## Plugin 1: remarkParse **Package:** `remark-parse` Converts the raw markdown string into a Markdown Abstract Syntax Tree (MDAST). This is the foundation -- every subsequent plugin operates on this tree. ### What it produces ```json { "type": "root", "children": [ { "type": "heading", "depth": 1, "children": [{ "type": "text", "value": "Title" }] }, { "type": "paragraph", "children": [{ "type": "text", "value": "Body text." }] }, { "type": "code", "lang": "python", "value": "print('hello')" }, { "type": "link", "url": "target.md", "children": [{ "type": "text", "value": "Link" }] } ] } ``` ### Important properties - Code fences become `code` nodes with `lang` and `value` fields. Their content is not parsed as markdown. - HTML blocks become `html` nodes with raw string values. They are preserved for later processing by `rehypeRaw`. - The parser handles GFM extensions only after the next plugin adds them. --- ## Plugin 2: remarkGfm **Package:** `remark-gfm` Enables GitHub Flavored Markdown extensions on top of the base CommonMark parser: | Extension | Syntax | AST Node | |-----------|--------|----------| | Tables | `\| col1 \| col2 \|` | `table` with `tableRow` and `tableCell` | | Strikethrough | `~~deleted~~` | `delete` node | | Task lists | `- [ ] item` | `listItem` with `checked` property | | Autolinks | `https://example.com` | `link` node (without explicit syntax) | | Footnotes | `[^1]` and `[^1]: text` | `footnoteReference` and `footnoteDefinition` | ### Impact on downstream plugins Tables, strikethrough, and task lists are common in the wikifita corpus. The autolink extension means bare URLs in text are automatically converted to link nodes, which are then processed by `remarkCanonicalLinks` (plugin 6). --- ## Plugin 3: remarkMath **Package:** `remark-math` Parses mathematical expressions into dedicated AST nodes: | Syntax | AST Node | Example | |--------|----------|---------| | `$inline$` | `inlineMath` | `$E = mc^2$` | | `$$block$$` | `math` (block) | `$$\int_0^1 f(x) dx$$` | The math nodes are preserved through the MDAST-to-HAST conversion and rendered to HTML by `rehypeKatex` (plugin 12). The parser handles nested delimiters and escaped dollar signs. --- ## Plugin 4: remarkMermaidGuard **Custom plugin** (`lib/markdown.ts`, lines 99-110) Validates mermaid code blocks against a whitelist of recognized diagram types. Invalid blocks are demoted to plain text to prevent client-side rendering errors. ### Recognized Diagram Types ``` graph, flowchart, sequenceDiagram, classDiagram, stateDiagram, erDiagram, journey, gantt, pie, mindmap, timeline, gitGraph, quadrantChart, xychart-beta, block-beta, packet-beta, sankey-beta ``` ### Validation Logic ```javascript function remarkMermaidGuard() { return (tree) => { visit(tree, 'code', (node) => { if (node.lang !== 'mermaid') return; const firstLine = node.value.trim().split('\n')[0].toLowerCase(); const isValid = DIAGRAM_KEYWORDS.some(kw => firstLine.startsWith(kw)); if (!isValid) { node.lang = 'text'; node.meta = 'mermaid-invalid'; } }); }; } ``` ### Design Rationale The mermaid client-side renderer (`mermaid.render()`) crashes on invalid syntax. By validating on the server, invalid blocks are gracefully degraded to `
` elements rather than causing client-side JavaScript errors. The `mermaid-invalid` meta tag allows CSS styling to indicate the block was intended as a diagram.

### Why Server-Side Validation, Server-Side Rendering Would Fail

Mermaid requires a DOM environment for rendering (it uses SVG generation internally). While `jsdom` could theoretically provide this, the server-side overhead is unnecessary. The validation happens server-side, but the actual SVG rendering happens client-side in the `useMermaid` hook.

---

## Plugin 5: remarkWikiLinks

**Custom plugin** (`lib/markdown.ts`, lines 42-79)

Transforms `WIKI_LINK_TARGET_LABEL` text patterns into proper MDAST link nodes. This is the core wiki-link processor.

### Processing Logic

For each `text` node in the AST, the plugin:

1. Scans for `triple-dot` patterns using regex
2. Looks up the target in `page.links` (pre-resolved during corpus building)
3. Replaces the text node with a sequence of: text-before, link, text-after

### Link Construction

```javascript
const link = {
  type: 'link',
  url: resolved ? `/wiki/${routePath}` : `#${encodedTarget}`,
  children: [{ type: 'text', value: label }],
  data: {
    hProperties: {
      className: resolved ? 'wiki-link' : 'wiki-link unresolved',
      'data-link-state': state  // "resolved", "missing", or "ambiguous"
    }
  }
};
```

### CSS Classes

| State | Class | Visual |
|-------|-------|--------|
| Resolved | `wiki-link` | Standard link styling (accent color) |
| Missing | `wiki-link unresolved` | Dashed underline, muted color |
| Ambiguous | `wiki-link unresolved ambiguous` | Dashed underline, warning color |

---

## Plugin 6: remarkCanonicalLinks

**Custom plugin** (`lib/markdown.ts`, lines 81-97)

Rewrites standard markdown links and images to use canonical wiki routes.

### For internal links (resolved)

```
Before: [text](target.md)
After:  [text](/wiki/target#fragment)
```

### For missing/ambiguous links

```
Before: [text](missing-page)
After:  [text](#missing-page)
  + className: "unresolved"
  + data-link-state: "missing"
```

### For external links

External links are left unchanged. They preserve their original URL (e.g., `https://example.com`).

---

## Plugin 7: remarkRehype

**Package:** `remark-rehype`

Converts the MDAST (Markdown AST) to HAST (HTML AST). This is the critical boundary between the markdown processing domain and the HTML processing domain.

### Configuration

```javascript
remarkRehype({ allowDangerousHtml: true })
```

The `allowDangerousHtml: true` setting preserves raw HTML blocks from the markdown source as `raw` HAST nodes. Without this, HTML blocks in the wiki content would be escaped.

### What changes

| MDAST | HAST |
|-------|------|
| `paragraph` | `p` |
| `heading` | `h1`, `h2`, ... |
| `link` | `a` |
| `emphasis` | `em` |
| `strong` | `strong` |
| `code` | `pre > code` |
| `inlineMath` | Preserved as custom node |
| `math` | Preserved as custom node |
| `html` (raw) | `raw` (string) |

---

## Plugin 8: rehypeRaw

**Package:** `rehype-raw`

Parses raw HTML strings embedded in the HAST into proper HAST elements. This converts `
```

Without `rehypeRaw`, this would remain as a raw string in the HAST. With it, the iframe becomes a proper `element` node with tag name, properties, and children -- enabling `rehypeMediaEmbeds` to process it.

### Security note

This plugin operates BEFORE `rehypeSanitize` (plugin 9). All dangerous HTML is neutralized by the sanitizer, not by `rehypeRaw` itself.

---

## Plugin 9: rehypeSanitize

**Package:** `rehype-sanitize`

The security boundary of the pipeline. All HTML output passes through a whitelist-based sanitizer that removes disallowed elements, attributes, and protocols.

### Custom Schema

The Wikifita sanitizer extends `rehype-sanitize`'s `defaultSchema` with wiki-specific allowances:

#### Allowed Attributes

```javascript
{
  a: ['className', 'data-link-state', 'target', 'rel', 'href', 'title'],
  code: ['className'],
  input: ['type', 'checked', 'disabled'],
  pre: ['data-mermaid-source'],
  span: ['style'],
  img: ['src', 'alt', 'title', 'width', 'height', 'loading'],
  iframe: ['src', 'allow', 'allowfullscreen', 'loading', 'referrerpolicy'],
  video: ['src', 'controls', 'preload', 'poster', 'width', 'height'],
  audio: ['src', 'controls', 'preload'],
  source: ['src', 'type']
}
```

#### MathML Elements

The sanitizer adds all MathML tag names to the allowlist:

```
math, mi, mn, mo, ms, mtext, mspace, mrow, mfrac, msqrt,
msup, msub, msubsup, semantics, annotation
```

These are required for KaTeX's HTML output, which uses MathML elements for accessible math rendering.

#### Additional Elements

| Element | Purpose |
|---------|---------|
| `figure` | Media embed wrapper |
| `figcaption` | Media embed caption |
| `input` | Task list checkboxes |

#### Protocols

Only `http:`, `https:`, and `mailto:` protocols are allowed in `href` and `src` attributes. `javascript:` and other dangerous protocols are stripped.

### What Gets Removed

- `"] --> B
    B --> C
    C --> D
    D -->|"stripped"| G["(removed)"]

    H[""] --> B
    B --> C
    C --> D
    D -->|"onerror removed"| I['']
```

### Layers

1. **remarkParse**: Does not execute HTML. Parses it as raw strings.
2. **rehypeRaw**: Parses raw HTML strings into HAST elements. No execution.
3. **rehypeSanitize**: Whitelist-based. Only allowed elements, attributes, and protocols survive.
4. **Client-side**: Mermaid runs with `securityLevel: "strict"`. No other dynamic content.

### What Survives Sanitization

- Safe HTML elements (`p`, `div`, `span`, `table`, `img`, `video`, `audio`, `iframe` with safe `src`)
- KaTeX MathML elements
- Wiki link attributes (`data-link-state`, `className`)
- Checkbox inputs (for task lists)
- `style` attribute only on `` elements (for KaTeX and highlight.js)

### What Does Not Survive

- `