secondBrain: A Vectorless Retrieval Architecture for Personal LLM Knowledge Bases

Abstract

I present secondBrain, a vectorless retrieval architecture for personal LLM knowledge bases. The system replaces probabilistic vector retrieval with three composable primitives: a two-step master-index lookup, strict per-category bold-key openings for grep-deterministic recall, and a derived graph.json artifact that captures the structural insight of HippoRAG (Gutiérrez et al. 2024) without an OpenIE pipeline. I report on a reference instance deployed on my personal engineering corpus, named mnzrBrain, at 240 pages distilled from 206 raw sources. I further report on a substrate-completion pass executed on 2026-05-07 in which four parallel sub-agents under hierarchical orchestration applied strict openings to 82 pages in under six minutes of wall-clock time, raising strict-opening coverage from 116 of 240 (48.3 percent) to 233 of 240 (97.1 percent), with the remaining seven pages exempt by rule. After concurrent cross-link enrichment, the reference instance reaches a structurally complete state: 2,201 cross-references across 240 pages, zero broken links, zero orphans, zero pages with fewer than two inbound references. A twelve-query retrieval-performance battery validates that direct fact-block queries resolve in a mean of 1.2 tool calls with 10 of 10 determinism, while the structurally bounded cross-cutting synthesis case averages 2 tool calls per query. I compare the architecture to the LlamaIndex January 2026 filesystem-versus-vector benchmark (LlamaIndex 2026) and argue that vectorless retrieval dominates on quality at sub-1000-page personal corpora.

Keywords: secondBrain, LLM Wiki, vectorless RAG, knowledge graph retrieval, hierarchical tree retrieval, evergreen notes, deterministic retrieval, personal knowledge management.

1. Introduction

1.1 Motivation

Engineering and research knowledge accumulates rapidly across repositories, conversation logs, design documents, lecture transcripts, and ad-hoc artifacts. Recovering specific facts from this corpus traditionally relies on vector retrieval-augmented generation (RAG): chunk the corpus, embed each chunk, retrieve nearest neighbours at query time. This pipeline has known failure modes at the scale of personal corpora. Similarity collisions degrade selectability when entries describe related content (Chroma 2025). Embeddings drift across model versions and require re-indexing on schema change. Retrieval is non-deterministic by construction; the same query against the same corpus can return different passages on different runs because tie-breaking depends on numerical noise in the embedding space.

1.2 Contributions

This paper reports on the design, implementation, and operational completion of secondBrain, a vectorless retrieval architecture intended for personal LLM knowledge bases at the scale individual users actually operate. I make four contributions.

1.3 Note on terminology: distinction from Forte's Second Brain

Tiago Forte's Building a Second Brain (Forte 2022) defines a methodology for human knowledge organisation built on the PARA framework (Projects, Areas, Resources, Archives) and the CODE workflow (Capture, Organise, Distill, Express). The methodology is widely known and shares vocabulary with this work. I use lowercase "secondBrain" deliberately to distinguish the two artifacts. The distinction is structural: Forte's framework governs what a human captures and how they file it before storage, optimised for human re-encounter; secondBrain governs how an LLM retrieves information from storage at query time, optimised for grep determinism and structural cross-reference traversal. The two methodologies are complementary rather than competing. A reader who applies Forte's PARA at the capture step and secondBrain at the retrieval step gets both surfaces. This paper covers only the retrieval surface.

1.4 Paper Organisation

Section 2 surveys related work and concludes with a unified architecture comparison (Table 2). Section 3 describes the three-layer system architecture. Section 4 details the retrieval methodology. Section 5 covers implementation and the reference-instance deployment. Section 6 reports results from the substrate completion, including a twelve-query retrieval-performance battery. Section 7 discusses limitations and future work. Section 8 concludes.

2. Related Work

2.1 Hierarchical Tree Retrieval

RAPTOR (Sarthi et al. 2024) introduced recursive abstractive summarisation over chunked corpora using SBERT embeddings, UMAP dimensionality reduction, and Gaussian mixture model soft clustering. On QuALITY with GPT-4, RAPTOR achieved 82.6 percent accuracy, a 20-point improvement over flat retrieval baselines.

PageIndex (VectifyAI 2025) is the industrial descendant. It removes the vector store entirely; document structure produces an LLM-summarised tree that the retrieval LLM reasons through at inference time. On FinanceBench, PageIndex achieved 98.7 percent versus approximately 30 to 50 percent for vector RAG baselines. The framing borrows from AlphaGo: a learned policy navigates a search tree rather than enumerating it. secondBrain's two-step lookup is a flat instance of PageIndex with hand-curated category indexes acting as level-1 cluster summaries.

2.2 Knowledge Graph Retrieval

Microsoft GraphRAG (Edge et al. 2024) constructs an entity-relationship graph and runs hierarchical Leiden community detection at multiple resolution levels. For global sensemaking queries, GraphRAG wins 70 to 80 percent of head-to-head comparisons against vector RAG.

HippoRAG (Gutiérrez et al. 2024) is the most relevant prior work to this architecture. It extracts a schemaless open-IE knowledge graph using GPT-3.5, with noun-phrase nodes, relation edges, and synonymy edges added via cosine similarity at threshold τ = 0.8. For a query, named entities seed a Personalized PageRank run, and the resulting probability mass over passage-nodes ranks the corpus. On 2WikiMultiHopQA, single-step HippoRAG outperforms ColBERTv2 by 20.9 points on R@5 and 16.2 points on F1, while running 6 to 13 times faster and 10 to 30 times cheaper than iterative methods like IRCoT.

LightRAG (Guo et al. 2025) introduces incremental graph updates so the corpus can grow without re-indexing. The dual-level retrieval pattern (low-level entity facts plus high-level community summaries) is the production-grade extension.

secondBrain inherits HippoRAG's structural insight (cross-references as the retrieval substrate) but parses the graph from hand-authored [[wiki-link]] syntax rather than running OpenIE on the corpus. At a few hundred hand-authored pages, LLM-extracted graphs would degrade signal that human curation already encodes.

2.3 Filesystem Agents

Anthropic's "Effective Context Engineering for AI Agents" (Anthropic 2025) argues for navigational primitives (grep, glob, read_file) over precomputed vectors. Agents maintain "lightweight identifiers (file paths, stored queries, web links)" and load just-in-time. The position is structurally aligned with the Karpathy gist.

The LlamaIndex January 2026 benchmark (LlamaIndex 2026) provides decisive empirical data for the small-to-medium-corpus regime. Tools given to the filesystem agent included read_file, grep_file_content, describe_dir_content, glob_paths, parse_file. Results are summarised in Table 1.

Table 1. LlamaIndex filesystem-versus-vector benchmark (LlamaIndex 2026).

At a few hundred pages, secondBrain sits firmly in the regime where filesystem retrieval dominates on quality. Latency on a personal wiki used several times per day is not the binding constraint.

2.4 Knowledge Management Principles

Andy Matuschak's Evergreen Notes (Matuschak 2020) articulates four principles for densely-linked personal knowledge: notes should be atomic, densely linked, concept-oriented, and organised by associative ontology rather than hierarchical taxonomy. The deeper claim is that the act of finding the right link during authoring is the elaborative encoding that produces understanding. secondBrain operationalises link-density at write time via a cross-reference-integrity rule (Section 4.5) so the graph is dense at query time.

2.5 The Karpathy LLM Wiki

Karpathy (2026) published a public gist describing a three-layer architecture for LLM-maintained personal wikis: raw sources (immutable), wiki pages (LLM-owned), and a schema governing operations. The compile metaphor ("Raw is source code, the LLM is the compiler, wiki is the executable, lint is tests, queries are runtime") is the canonical formulation. The gist defines three operations (ingest, query, lint) and reports an observed scale of approximately 100 articles and 400,000 words on a single research topic.

One historiographical clarification is in order. The phrase "vectorless RAG" is widely attributed to Karpathy in secondary sources but does not appear in the canonical gist; it belongs to PageIndex (VectifyAI 2025), a separately-developed project. This work uses the term in its PageIndex sense and credits Karpathy for the underlying compile-don't-store architecture.

2.6 Architecture Comparison

Table 2 summarises the six retrieval architectures surveyed in this section against secondBrain along five dimensions. The dimensions are (i) the index structure that supports retrieval, (ii) the retrieval mechanism applied at query time, (iii) determinism of the retrieved set under repeated queries, (iv) the re-indexing burden on schema or model change, and (v) the corpus regime where the architecture is reported to dominate.

Table 2. Retrieval architecture comparison.

The architectures cluster along two axes. PageIndex and secondBrain cluster on the "structural index, no embeddings" side; vector RAG sits at the opposite extreme. HippoRAG, GraphRAG, and LightRAG sit between, layering graph structure over an embedding substrate. secondBrain's distinguishing position is that the graph structure is hand-authored at write time (the cross-reference-integrity rule, Section 4.5) rather than extracted at index time, eliminating both the OpenIE pipeline and the model-version-dependence of embeddings.

3. System Architecture

secondBrain follows the three-layer architecture from the canonical Karpathy gist. Each layer has a distinct responsibility, governed by a separate rule set, and a strict boundary against the others.

3.1 Layer 1: Raw Sources (Immutable)

Source materials are never edited by the LLM. Repositories, conversation logs, lecture transcripts, PDFs, and personal artifacts sit under /raw/ and /Conversations/ and accumulate append-only.

3.2 Layer 2: Wiki (LLM-Owned)

Atomic markdown pages with dense [[wiki-link]] cross-references comprise the executable form of the corpus. Pages are authored by an LLM under human curation. The corpus is organised into eight categories: skills, projects, patterns, decisions, stories, problems-solved, career, and learning.

3.3 Layer 3: Schema (Governance)

The schema defines what the LLM is allowed to write. It includes the page-format rule, the cross-reference-integrity rule, the cascading-update rule, the deprecation-with-redirect protocol, and the per-category bold-key openings introduced in Section 4.3.

3.4 The Two Root Files

Two files at the wiki root carry distinct retrieval roles, both inherited from the canonical gist.

3.5 The Three Operations

The system supports three primitives:

4. Retrieval Methodology

4.1 The Two-Step Lookup Pattern

Every query follows a deterministic shape: read index.md (~9K tokens) first, identify 3 to 5 relevant pages, read those pages, synthesise. The pattern is structurally equivalent to PageIndex's tree navigation but flat at this scale, with hand-curated category indexes acting as level-1 cluster summaries (cf. Section 2.1).

4.2 Unique-Token Disambiguation

Chroma's context-rot research (Chroma 2025) shows that LLM selection performance degrades when the input contains semantically similar distractors, even at modest context lengths. The mechanism is similarity collision: when two candidate entries describe related content with related prose, the selector commits non-deterministically.

The fix encoded in secondBrain is unique-token disambiguation. Every entry in index.md carries at least one unique discriminator: a number, a date, a project codename, or a unique phrase. A typical entry reads "70x faster reads, 66x throughput, 2,000 req/s" rather than "improved performance." Numbers anchor the selector deterministically the way primary keys anchor rows.

4.3 Strict Per-Category Openings

Each category has a designated bold-key opening shape immediately following the summary blockquote. Decision pages open with Decision, Verdict, Date locked, Trade-off. Pattern pages open with Mechanics, When to apply, When NOT to apply. Problem pages open with Symptom, Root cause, Fix, Lesson. Skills pages open with Stack, Practiced on, Depth signal, Years. Active project pages open with Status, Stack, Scale, Role, Outcome. Career hub pages open with Span, Anchor numbers, Defining milestone, Current role. Career canonical-prose pages open with Purpose, Last verified, Update protocol. Lecture pages open with Lecture, Length, Source, Key takeaway. Non-lecture story pages (blogs, milestones, guides) open with Setting, Outcome, Lesson. Learning pages open with Topic, Status, Why it matters, Cross-link. The shape makes grep retrieval deterministic for category-shape queries, reducing many lookups from full-page reads to single-line greps. Two categories of pages are exempt by rule: root infrastructure files and a small set of explainer stories where the framing prose itself is the value rather than a fact-block.

4.4 Derived `graph.json` (HippoRAG Without OpenIE)

The cross-reference graph is materialised as graph.json, a Python-derived artifact that parses every [[wiki-link]] in the corpus and emits {nodes, edges}. Lint reads it for orphan detection, dead-link detection, and density-anomaly detection. This captures HippoRAG's structural insight (graph structure as the retrieval substrate; Gutiérrez et al. 2024) without HippoRAG's OpenIE pipeline. The artifact is a compile output, not a parallel store. Deletion loses no information, and the artifact is regenerated by a post-edit hook on every ingest.

4.5 Cross-Reference Integrity

Every wiki page must hold at least three inbound [[wiki-link]] references before commit. The rule operationalises Matuschak's link-density principle (Matuschak 2020) for an LLM-readable corpus. Combined with the cascading-update rule (Section 4.6), the cross-reference graph remains dense without continuous manual audit.

4.6 Cascading Updates

Every ingest must update 5 to 15 affected neighbours. The number is empirical; it emerged from observed practice on production-scale ingests during the reference-instance deployment and is encoded in the secondBrain ruleset. The cascading update protocol is one of secondBrain's elaborations on Karpathy's gist, not a Karpathy directive.

4.7 Rule-Governed LLM Authorship

The human curator collects sources, decides what enters the corpus, and reviews pages before commit. The LLM authors pages and maintains cross-references, with allowed operations bounded by the schema. This division produces a maintainer-curator workflow consistent with Karpathy's observation that "the LLM is the maintainer."

5. Implementation and Reference Instance

5.1 The mnzrBrain Reference Instance

I deployed a reference instance of secondBrain on my personal engineering corpus. The instance, named mnzrBrain, is the running artifact used in the experiments reported in Section 6. The instance demonstrates the architecture under realistic personal-corpus conditions: heterogeneous sources (work logs, lecture transcripts, repository extracts, conversation archives), an actively-curated growth rate, and a binding requirement that retrieval be faithful to the underlying source material.

Table 3 reports the source materials distilled into the reference instance.

Table 3. Source corpus distilled into the mnzrBrain reference instance.

The reference instance produces 240 wiki pages distributed across the eight secondBrain categories: skills (9), projects (48), patterns (37), decisions (31), stories (42), problems-solved (47), career (6), learning (11), plus root infrastructure files. Five categories with more than 25 pages also carry dedicated category indexes (projects, patterns, decisions, stories, problems-solved).

5.2 Page Format

Every wiki page carries a YAML frontmatter block (tags, created, updated, sources). The body opens with a one-line > blockquote summary, followed by the category-specific bold-key block (Section 4.3), followed by free-form sections, followed by ## Related for sister pages. The format is enforced by the page-format.md rule in the secondBrain operating system.

5.3 The Compile Pipeline

The compile pipeline runs three steps.

5.4 Parallel-Agent Ingestion Methodology

The system uses Claude Code as the LLM compiler, with custom skills, agents, and rules in a .claude/ operating directory. Bulk ingestion work distributes across parallel sub-agents under hierarchical orchestration. Each agent receives a narrow scope, an explicit constraint set (no fact invention, no body modification beyond the strict opening, conformant grammar), and a verification grep step at the end. The orchestrator sequences dispatch, validates scope non-overlap, aggregates reports, and corrects any drift introduced.

6. Results

6.1 Final State of the Reference Instance (Post Substrate Completion)

After the substrate-completion pass on 2026-05-07, the mnzrBrain reference instance reached the state shown in Table 4.

Table 4. Final reference-instance state after the 2026-05-07 substrate completion.

6.2 Retrieval Performance on the Reference Instance

To validate the substrate end-to-end, I ran a twelve-query battery against the deployed reference instance, simulating fresh queries with no prior knowledge of which page held the answer. The method per query: read index.md Quick Routes table, identify candidate pages, grep the bold-key block, fall back to a full Read only if grep is insufficient. Tool calls were counted; query verdicts were classified as deterministic (a single grep returned the canonical answer), Quick-Navigation-mediated (a hub page's section anchor returned the answer), or synthesis-required (multiple greps plus an LLM merge step). Table 5 reports per-query tool counts and verdicts.

Table 5. Twelve-query retrieval-performance battery against the mnzrBrain reference instance.

\* Q4's extra Read was a display-wrap quirk where a long bold-key line exceeded the grep tool's content window, not a substrate failure. The strict-opening pattern was intact; one additional Read of a single line fetched the truncated content.

Aggregate metrics. Mean tool calls per direct fact-block query (Q1 through Q10): 1.2 (median 1). Mean tool calls per cross-cutting synthesis query (Q11, Q12): 2 (median 2). Determinism rate on direct queries: 10 of 10. Tokens read per direct query: approximately 50 to 300 (versus an estimated 5,000 to 15,000 for the equivalent flat-wiki query before strict openings). The synthesis queries exhibit the structurally bounded behaviour described in Section 7.3: each component grep is cheap, but the merge across multiple greps remains an LLM step that the substrate cannot eliminate.

Findings. Every direct fact-block query lands in a single grep (with the noted Q4 display-wrap exception). Hub Quick Navigation blocks let queries skip directly to a section anchor instead of reading the full hub. The Quick Routes table at the top of index.md routes recognisable question shapes straight to a canonical entry page without category-index detour. Across 12 queries spanning 10 categories, no query needed an unexpected full-page synthesis.

6.3 Substrate-Completion Throughput

Bulk strict-opening application across 82 pages used four parallel sub-agents under hierarchical orchestration. Table 6 reports per-agent scope and aggregate wall-clock time.

Table 6. Parallel-agent dispatch over 82 pages.

Aggregate wall-clock for the parallel dispatch: under six minutes. Hand-craft work covering 9 skills pages, 2 career hubs, and 4 career canonical-prose pages ran concurrently with the agent dispatch. The orchestrator's overhead (dispatch, scope validation, post-completion verification, drift correction) ran inside the same window.

6.4 Strict-Opening Coverage Trajectory

Coverage grew across three operational phases.

Table 7. Strict-opening coverage trajectory on the reference instance.

The remaining 7 pages are exempt by rule: 5 are root infrastructure files and 2 are explainer stories where framing prose is the value, not a fact-block.

6.5 Cross-Link Enrichment

A concurrent cross-link audit identified 5 pages still at one inbound link prior to enrichment. Each was enriched from the most logical hub: project hubs absorbed the meta-decision and historical-plan pages; the career education page absorbed the academic credential output; the backend-skills page absorbed the coursework and the local-first concept page. After enrichment, every page in the reference instance has at least two inbound links.

7. Discussion

7.1 The Universality Argument

The first 116 strict openings (decisions, patterns, problems-solved) covered approximately 90 percent of the most common query shapes. Adding strict openings to the remaining 117 pages did not double retrieval value; it closed the long tail of queries that previously required full-page reads. The leverage compounds across operations: ingest cascades, lint passes, and category-shape queries become single-grep operations rather than full-page synthesis. The decision to push from "mostly" to "universal" is the load-bearing one for a substrate intended to support arbitrary future query shapes.

7.2 Comparison to Vector RAG at This Scale

Per the LlamaIndex January 2026 benchmark (Table 1), filesystem-agent retrieval wins on quality at the 5-paper scale, ties at 100 papers, and loses to vector RAG only at 1000+ papers. At the reference instance's current 240 pages, the architecture is firmly in the regime where filesystem retrieval dominates on quality. I anticipate no scaling crisis until the corpus exceeds approximately 1,000 pages. At that point Section 7.4 outlines the criteria for adding embeddings as a tiebreaker rather than as the primary index.

7.3 Limitations

7.4 Future Work

8. Conclusion

I have presented secondBrain, a vectorless retrieval architecture for personal LLM knowledge bases. The system is a reference implementation of the canonical Karpathy LLM Wiki specification (Karpathy 2026), extended with project-specific rules drawn from PageIndex's hierarchical tree retrieval (VectifyAI 2025), HippoRAG's structural-graph retrieval (Gutiérrez et al. 2024), and Matuschak's evergreen notes principles (Matuschak 2020). I deployed a reference instance, mnzrBrain, on my personal engineering corpus. At 240 pages with 2,201 cross-references, 97.1 percent strict-opening coverage, and zero orphans or broken links, the substrate is structurally complete. A twelve-query retrieval-performance battery confirms direct fact-block queries resolve in a mean of 1.2 tool calls with 10 of 10 determinism. Future improvements compose on top without rework. I conclude that hand-curated, LLM-authored, rule-governed knowledge bases dominate vector RAG on quality at the sub-1000-page personal-corpus scale, and that the appropriate substrate for personal AI memory is structural rather than statistical.

References

---

The wiki is what the LLM reads. The schema is what the LLM is allowed to write. The cross-references are the retrieval system. Compile, don't store.