The 82-Line README That Almost Beat a Second Brain

📋 The brief

I wanted to run a basic data exercise: take one dataset, organize it three different ways, and measure what each structure actually costs an AI agent to search. A controlled comparison I could point to instead of arguing about it in the abstract.

Three versions of the same data:

1️⃣ Raw — the dataset exactly as downloaded. One file, no structure, no metadata.

2️⃣ ICM — files and folders. A mechanical breakdown into a directory hierarchy, the kind of structure I've been building into ICM workspaces for a while now.

3️⃣ "Second Brain" — an Obsidian-style vault. Same content, but with per-item notes, cross-linked [[wikilinks]], character/theme pages, the whole living-notes treatment.

Then I built a small tool that fires the same question at all three, using a real agent for each run (not a canned lookup), and logs tokens, time, and cost per stage. Point the same question at raw, ICM, and Obsidian, and see what each structure actually buys you.

📚 Why I chose Shakespeare

I needed something big enough to be a real test, public domain, and — critically — already broken down at a fine grain (act, scene, sonnet) so I wasn't inventing structure that wouldn't exist in a messier real dataset. I looked at the U.S. Code first. It's the right shape (title → chapter → section mirrors book → chapter → verse almost exactly), but it's enormous.

Shakespeare's complete works are a fixed, known-size corpus (5.4MB, Project Gutenberg, public domain) that will never change, never need re-downloading, and never go stale. That "always useful" property means this test is reusable as a reference point for other structure comparisons later, not a one-off.

💰 A bit on the cost of setting up

The ICM layer cost almost nothing to build. It's a mechanical script — split on ACT/SCENE headers, extract speaker names by regex, write files. No model calls. 1,579 files, and the token footprint came out at 1.01x the raw file's size. Structure that's this close to free is easy to underrate.

The Obsidian layer was a different story. Getting real interpretive content — scene summaries, character arcs, theme essays, all cross-linked — took real agent labor: roughly 15 parallel builds across two rounds, several of which failed outright along the way (a session token-limit cutoff mid-run, an agent that tried to spawn its own sub-agents instead of doing the work, a content-filter false-positive on a play with heavy battlefield violence, an API formatting crash that lost a batch's work before it hit disk). Every one of those got caught and fixed, but it took real wall-clock time and a fair number of retries to get a clean 3,122-file vault out the other end. Token overhead landed at 1.31x over ICM, 1.35x over raw.

So: ICM costs almost nothing to build. Obsidian costs real time, real tokens, and real debugging.

🏃 The runs

I wrote 8 questions across four categories — exact lookup ("what happens in Act 3, Scene 1 of Hamlet"), thematic search ("which scenes deal with jealousy destroying a marriage"), cross-work synthesis ("which plays feature a character disguised as the opposite gender"), and one interpretive trap question with no clean factual answer ("is Hamlet actually mad, or only pretending"). Each question ran against all three structures — 24 runs total, each one a real spun-up agent scoped only to that structure's folder, capped at a 200-word answer, explicitly told not to lean on memorized knowledge of the text.

Totals across all 8 questions:

🥇 Obsidian vault

1.53M tokens total · 235.1s total · $0.495 total · 169,543 tokens/query avg

🥈 ICM

1.67M tokens total · 235.6s total · $0.547 total · 185,441 tokens/query avg

🥉 Raw

3.46M tokens total · 378.0s total · $0.825 total · 384,738 tokens/query avg

Raw came in at roughly 2.3x the tokens and 1.6x the time of either structured version — that part matched my prediction exactly. No structure means every question is a search problem the agent has to solve from scratch. The part I didn't predict was how close ICM and Obsidian ended up to each other. I thought the dataset would be the type that Obsidian would run away with. It did not.

🎭 On one cross-work question ("which plays feature a character disguised as the opposite gender"), the vault crushed it — 135K tokens against ICM's 566K and raw's 823K.

💀 But on another cross-work question ("find three moments where a character fakes their own death"), ICM actually beat the vault — 214K against the vault's 479K — because the richer link structure sent the agent wandering through wikilinks instead of just grepping.

🎯 Conclusion

I went into this expecting the Obsidian layer to be the clear token-saving winner — that's the whole pitch of a "second brain," after all, more connective tissue should mean less searching. It did win, on average. But it won by less than I expected, and it lost outright on at least one question. Meanwhile the plain files-and-folders ICM structure — built for free, with no model calls — got most of the way to Obsidian's numbers on its own, especially once I gave the folder an 82-line CLAUDE.md explaining the layout, the naming convention, and how the split actually works.

The actual finding, and it's a more useful one than "second brains win": most of the value of structure comes from having any consistent structure at all, plus a short description or index of how to read it. The expensive, interpretive, hand-built layer on top is real — it wins some categories decisively — but it's a smaller marginal gain than its setup cost would suggest, at least at this scale. If I were advising someone standing up an ICM workspace from scratch, I'd tell them to get the free structural layer right first and write the README before reaching for the second-brain treatment. 🛠️

9 comments

The 82-Line README That Almost Beat a Second Brain