I Dug Into LLM Wiki, and I Think It Beats RAG and Ontology for Complex Business Agents

TLDR: LLM Wiki means having an AI "compile" your knowledge into a bunch of interlinked markdown files up front, then just reading them — instead of searching and stitching like RAG does on every query. Since Karpathy posted the pattern this April, the gist has 5,000+ stars and the community has built real implementations. My take after comparing all three: it's more stable than RAG (no gambling on retrieval quality) and simpler than Ontology (humans can actually read and change it). For agents with genuinely complex business logic, LLM Wiki > Ontology and RAG. The catch: it doesn't scale — a few hundred pages is the ceiling.

---

What is LLM Wiki, anyway

One sentence: RAG is a search engine; LLM Wiki is a compiler.

This April, Karpathy described the pattern in a GitHub gist. It's not a framework — just a prompt, roughly saying: stop retrieving from raw documents on every question. Have the agent read your sources once, write them up as interlinked markdown pages, and from then on, querying means reading those pages.

One line of his stuck with me: "The knowledge is compiled once and then kept current, not re-derived on every query." And the metaphor: "Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase."

The gist now has 5,000+ stars and 5,000+ forks. A plain-text prompt getting that kind of traction means it hit a nerve.

How people are actually implementing it

I looked around. Roughly four flavors right now:

• Gist + a coding agent, raw. The original way: make an Obsidian vault, paste the gist into Claude Code or Codex as system instructions, drop documents into sources/, let the agent write pages into wiki/. One person ran this for three months and the wiki grew itself to 147 pages — he wrote none of them by hand (source).
• Packaged as a plugin or skill. Like the LLM Wiki Kit from llm-wiki.net for Claude Code, Codex, and OpenCode, with exportable "knowledge checkpoints." There are also coding-agent skills like llm-wiki-agent.
• Desktop apps. Like nashsu/llm_wiki, which turns documents into an organized knowledge base automatically.
• Git-driven team wikis. Some teams have agents maintain a markdown + git knowledge base where every update is a reviewable diff (example).

The common thread: no vector database, no embeddings, no retrieval pipeline. Just a folder, some markdown, and an agent that can read and write. I'll admit I was skeptical of how primitive that sounded. After looking at the data, I'm not.

vs RAG: it wins on "stable"

RAG's problem isn't that it's bad. It's that every single query is a gamble. Will the chunks split right? Will the embedding rank the right fragment first? Will the model actually see the fragment you stuffed in? That gamble fails a lot:

• Gartner predicted that by end of 2025, at least 30% of GenAI projects get abandoned after proof of concept, mostly over data quality and risk controls; through 2026, 60% of AI projects without AI-ready data will be abandoned (Gartner press release).
• The "Lost in the Middle" paper (Liu et al., TACL 2024) showed models genuinely miss information placed in the middle of the context — accuracy follows a U-shaped curve. Where your retrieved chunk lands directly affects whether the answer is right.
• Chroma's Context Rot study this year tested 18 frontier models: longer input makes every single one worse, no exceptions (report).
• There's an arXiv paper cataloging seven classic failure points of RAG systems — missing content, bad chunks, wrong synthesis. Every stage of the pipeline can tank you.

LLM Wiki sidesteps that entire failure chain, because it moves "understanding" from query time to write time:

	RAG	LLM Wiki
Understanding happens	Every query, assembled on the fly	Once, at ingestion
State	Stateless, starts over each time	Persistent, compounds
Relationships between ideas	Guessed via embedding similarity	Written out as markdown links
Contradiction handling	None — conflicts pop up randomly	Flagged actively during ingestion
Infrastructure	Vector DB + embeddings + retrieval pipeline	A folder
Best at	Millions of documents	A few hundred pages

Why it's better for agents specifically

I've been building agents with complex business logic, and two things stood out:

First, it's easier to understand. An agent reading a wiki is just reading files — it gets a full page explaining one concept, with context and cross-links included. RAG hands it five fragments cut from who-knows-where, and the agent has to guess how they relate. When business logic gets hairy, fragment-stitching errors are where hallucinations come from. Links are explicit; similarity is a guess. You feel this difference when debugging: with a wiki, you can follow the links and see exactly why the agent knew something. With RAG, you're digging through an embedding top-k list.

Second, it's more stable. Ask the same question twice, get the same answer — it read the same file. RAG's answer can shift because the retrieval ranking jittered. And a wiki can live in git: knowledge changes show up as diffs, mistakes roll back, changes get reviewed before shipping. If you build business agents, you know "auditable knowledge changes" saves your neck in compliance reviews and in blame games alike.

vs Ontology: it wins on "simple" and "readable"

The older path for complex business knowledge is building an ontology / knowledge graph. That path's problem isn't the theory — it's that it's expensive, slow, and rigid:

• An enterprise knowledge graph typically takes 6-12 months from kickoff to production (Rebase's numbers); even a single domain takes 6-12 weeks (Atlan).
• Cutter Consortium estimates the long-term total cost of an enterprise knowledge graph at $10-20 million (cited in this analysis — take the exact figure with salt, but the order of magnitude isn't crazy).
• And ontology work is schema-first: experts have to define "what concepts exist in this business and how they relate" before any data goes in. Business changes, schema follows.

LLM Wiki lands on the opposite side of all three:

1. Simpler structure. No schema to design up front. Pages and links are the whole structure. The agent organizes as it writes; if it organizes wrong, you edit a file.
2. Humans can read it. Business stakeholders open a markdown file and review it. No Sparql, no graph database. For the first time, the knowledge base is something business and engineering can read together.
3. Easier to maintain. Changing knowledge = editing a file + a git diff. No ontology engineers, no graph migration scripts. Maintenance drops from "employ a team" to "spend half an hour every few weeks letting the agent run a contradiction check."

To be fair: where you need strict formal reasoning and consistency guarantees — medical terminology, regulatory compliance — ontologies are still irreplaceable. But most business agents don't need logical completeness. They need "don't answer wrong, and when it's wrong, be able to trace it and fix it."

My take: for complex business agents, LLM Wiki > Ontology and RAG

It boils down to one point: the more complex the business, the more the value of knowledge lives in relationships and judgment, not facts.

• RAG is good at fetching facts, but relationships and judgment hide between the lines of documents, and chunk-based retrieval loses them by design.
• Ontology can express relationships, but not judgment ("usually handle it this way, unless..."), and its maintenance cost explodes with complexity.
• In a wiki page, facts, relationships, and judgment sit in the same paragraph. Humans and agents can both read it. Compile once, use forever.

A slightly counterintuitive corollary: bigger context windows make this pattern stronger. Today's 150-200 page ceiling exists because of context limits, and when 10M-token windows become normal (many people say around 2027), the ceiling basically disappears. Meanwhile, RAG's retrieval jitter and Ontology's maintenance cost don't go away when models get smarter.

When not to use it

Being honest — three cases where LLM Wiki is the wrong choice:

1. Hundreds of thousands of documents, updated daily — RAG is still the only realistic answer.
2. Strict formal reasoning and consistency guarantees — that's ontology territory.
3. You just occasionally look something up — upload the file to ChatGPT and move on.

And it's not zero-maintenance. The experienced figure is 30-45 minutes every 2-3 weeks on contradiction checks and terminology cleanup. Way lighter than maintaining a knowledge base by hand, but "set it and forget it" will rot it.

The future is probably hybrid: RAG for breadth and freshness, Wiki for depth and relationships. But if I could only pick one foundation for a complex business agent, I'd pick the Wiki.

References

• Karpathy's original gist
• LLM Wiki Setup: Karpathy's Knowledge Base (three-month field report)
• LLM Wiki Kit / llm-wiki-agent / nashsu/llm_wiki
• Gartner: 30% of GenAI projects will be abandoned
• Lost in the Middle (Liu et al.) / Seven RAG failure points / Chroma Context Rot
• Enterprise knowledge graph cost and timeline