Project Hydra: A Sovereign Personal AI with Tiered Memory

Abstract

I present Project Hydra, a sovereign personal AI architecture that targets a single binding property: memory that does not forget. The system factors into five horizontal layers (Inference, Memory, Sync, Provenance, Identity) and four cognitively grounded memory tiers (working, episodic, semantic, procedural) mapped onto the human hippocampus-to-neocortex consolidation pathway. Cross-device synchronisation uses Yjs CRDTs (Shapira et al. 2017) for deterministic-shape merges and a single designated merger for LLM-mediated semantic rewrites, an honest answer to the LLM non-determinism problem that pure CRDT eventual consistency cannot satisfy. Every commit to semantic memory is hashed and anchored to a private permissioned blockchain (Hyperledger Besu, EVM-compatible) for verifiable provenance. Identity uses multi-modal continuous trust (voice via ECAPA-TDNN speaker embedding (Desplanques et al. 2020); behavioural fingerprint; Bayesian fusion). This case study reports the design, decisions, and pre-registered evaluation rather than a shipped artifact: the system is in the planning stage, supported by a 29-page formal thesis and a 2026-05-01 architecture review that produced six binding decisions and surfaced four explicitly under-specified parts. I argue that an architecture-and-decisions case study is the appropriate form for a research-grade integration system at this stage, and I treat the discipline of naming open questions in advance as a strength rather than a deficiency. I compare the design to MemGPT (Packer et al. 2023), Mem0 / Letta (Letta team 2024), and the generative-agents reflection mechanism (Park et al. 2023), and I register the consolidation-faithfulness benchmark as the load-bearing evaluation slice against LongMemEval (Du et al. 2024) and LoCoMo (Maharana et al. 2024). The case study is offered as the architectural register that precedes a 9-month, 7-milestone build.

Keywords: sovereign personal AI, brain-mapped memory, four-tier memory factoring, out-of-band consolidation, LLM-coordinated CRDT, blockchain-anchored provenance, multi-modal identity, consolidation faithfulness.

1. Introduction

1.1 Motivation

Contemporary AI assistants run as ephemeral conversations. Each chat starts fresh, the context window fills and is silently auto-compacted into a lossy summary, and three months of accumulated insight evaporate because the system has no durable memory model. The user experiences forgetting even when no data was deleted, because in-band compaction destroys originals under live-conversation latency budget. The cinematic Jarvis (the Iron Man fantasy of an omniscient, witty, omnipresent assistant) is the cultural reference any engineer who has watched the films carries unspoken; the gap between that silhouette and a stateless turn-by-turn chat product is the space Hydra is designed to occupy. Hydra is the formal project name; Jarvis is the cinematic north star. The thesis (§15) is explicit about the relationship: the fantasy is a north star, and the shipped artifact is one carefully chosen slice of it (Section 7.4 makes this load-bearing for the present scope).

A second motivation is sovereignty. A persistent personal-memory store accumulates a high-resolution model of a single user. Replicating that store onto third-party clouds, even for survivability, is a sovereignty cost paid in perpetuity. Hydra's design constraint is that every byte of episodic and semantic memory must live on hardware the user owns, with cross-device sync running over a user-owned mesh.

1.2 Contributions

This paper reports the design and pre-registered evaluation of Project Hydra, a planning-stage sovereign personal AI. I make four contributions.

1.3 Paper Organisation

Section 2 surveys related work. Section 3 describes the five-layer system architecture. Section 4 details the methodology, including memory factoring, consolidation, sync, and provenance. Section 5 covers prototype implementation and the milestone plan. Section 6 reports the planning-stage state, the six binding architectural decisions, and the four under-specified parts. Section 7 discusses limitations, the load-bearing scope principle, and future work. Section 8 concludes.

2. Related Work

2.1 LLMs as Operating Systems and Tiered Memory

MemGPT (Packer et al. 2023) introduces the LLM-as-operating-system framing: hierarchical memory with a "main context" tier and an "archival" tier, and the LLM uses function-call tools to self-edit memory. Hydra inherits the operating-system framing, extends the hierarchy to four cognitively grounded tiers, and explicitly separates the consolidation operation from in-band auto-compaction. Mem0 (Letta team 2024) ships a two-phase Extract / Update pipeline with ADD, UPDATE, DELETE, and NOOP operations. The Letta filesystem-agent variant (Wooders et al. 2026) treats conversation history as files retrievable by standard filesystem operations. Both report strong end-to-end retrieval scores on LoCoMo but, by their own admission, do not separately evaluate the faithfulness of the update operation itself.

2.2 Generative Agents and Reflection

Park et al. (2023) introduce a memory stream with periodic "reflection" passes that synthesise raw observations into higher-level conclusions. Reflection is the closest published analogue to Hydra's out-of-band consolidation, in that both produce distilled outputs that augment rather than replace the underlying memory stream. Hydra's consolidation is structurally distinct in three respects: it runs on a separate process from the live assistant, it produces structured wiki-shaped pages rather than free-form reflections, and it is gated by a faithfulness check before write.

2.3 Long-Horizon Conversation Benchmarks

LongMemEval (Du et al. 2024) is a 500-question benchmark covering five memory abilities (information extraction, multi-session reasoning, knowledge updates, temporal reasoning, abstention) on dialogues averaging approximately 9K tokens across multiple sessions. LoCoMo (Maharana et al. 2024) covers approximately 600 dialogues at roughly 30 sessions each, with explicit retrieval queries against multi-session memory. Both benchmarks evaluate end-to-end memory recall but do not isolate the update or consolidation step. Hydra's pre-registered evaluation plan augments both with consolidation-time faithfulness probes (Section 4.3).

2.4 CRDTs and Local-First Software

Yjs (Shapira et al. 2017) provides a production-grade CRDT framework for collaborative text and structured data with deterministic position-merge, set-union, and tombstone semantics. The Local-First Software manifesto (Kleppmann et al. 2019) articulates the principle that user data should remain on user-owned devices with peer-to-peer sync as the default rather than a third-party cloud service as the default. Hydra adopts both: Yjs for the deterministic-shape merge layer, local-first as the architectural register for sovereignty.

2.5 LLM Non-Determinism and Coordinated Consistency

CRDT eventual consistency requires the merge function to be commutative, associative, and idempotent. LLMs sample stochastically, and even greedy decoding has minor variation across inference engines, quantisations, and KV cache implementations (Karpathy 2026 surveys related determinism issues in LLM tooling). Two independent consolidation runs over the same source episodes will not produce byte-identical outputs. Pure-CRDT semantic merging is therefore unattainable for LLM-mediated rewrites. Hydra's response is to designate the home server as the only merger for LLM-shaped edits, with other instances submitting and waiting (Section 4.4). This trades full symmetric eventual consistency for coordinated eventual consistency, in line with quorum-coordinated systems such as Riak and Dynamo.

2.6 Permissioned Blockchain Provenance

Hyperledger Besu (Hyperledger 2024) is an EVM-compatible permissioned blockchain that supports Merkle commit anchoring, Solidity smart contracts, and configurable consensus. Hydra anchors a Merkle root of every consolidation pass on Besu, producing a signed, append-only, externally verifiable provenance log. The philosophical thinness is acknowledged in Section 4.5: a signed commit attests to who ran the consolidation, not to the truth of the resulting fact. Provenance is necessary but not sufficient for trust; faithfulness is the other half.

2.7 Multi-Modal Continuous Identity

ECAPA-TDNN (Desplanques et al. 2020) is the production-grade speaker embedding for voice biometrics, achieving state-of-the-art equal-error-rate on VoxCeleb. Behavioural fingerprints (typing cadence, navigation patterns) provide a second modality robust to voice unavailability. Hydra fuses both via Bayesian update to produce a continuous trust score rather than a one-shot login (Section 4.6), with different memory tiers gated at different thresholds.

3. System Architecture

Hydra factors into five horizontal layers, each with a distinct responsibility and clean interfaces to its neighbours. The factoring is non-arbitrary; each layer corresponds to a different class of hard problem, and removing any layer collapses a different property of the assistant.

3.1 Layer 1: Inference

Local LLM inference is the foundation. Hydra runs models on hardware the user owns: the iPhone for foreground conversational inference, a home server for heavy reasoning and consolidation. A multi-provider abstraction places Ollama, llama.cpp, MLX, and remote APIs behind one interface, so swapping the underlying model is a configuration change rather than a rewrite. Cloud APIs (Claude, GPT) are accepted only during the prototype phase, with explicit acknowledgement that this temporarily dilutes the sovereignty thesis until production hardware lands.

3.2 Layer 2: Memory

Memory is the technical heart of the architecture and the layer where most of the research contribution lives. Memory factors into four cognitive tiers, each with distinct update, retrieval, and decay semantics, mapped onto the human hippocampus-to-neocortex consolidation pathway. Section 4.2 details the factoring. The mnzrBrain wiki, an LLM-readable structured semantic-memory store maintained as a standalone artifact, is registered as the planned semantic-memory substrate; the coupling protocol between the wiki and Hydra (sync API versus direct file edits) is one of the under-specified parts (Section 6.3).

3.3 Layer 3: Sync

Hydra runs on multiple devices. Each device holds a partial replica of episodic and semantic memory. Cross-device sync uses Yjs CRDTs (Shapira et al. 2017) for deterministic-shape merges (text positions, set unions, tombstones) and a single designated merger on the home server for LLM-mediated semantic rewrites. Section 4.4 details the dispatch.

3.4 Layer 4: Provenance

Every commit to semantic memory is hashed, and the Merkle root is anchored to a private permissioned blockchain (Hyperledger Besu, EVM-compatible) via a Solidity contract. The provenance log is append-only, signed, and externally verifiable. Any future query against the semantic tier can prove that the content existed at a specific timestamp and was authored by a specific consolidation run. Section 4.5 details the operation and acknowledges the philosophical thinness.

3.5 Layer 5: Identity

Multi-modal continuous identity. Voice biometrics via ECAPA-TDNN speaker embeddings (Desplanques et al. 2020) plus behavioural fingerprints (typing cadence, navigation patterns) are fused via Bayesian update to maintain a continuous trust score. Different memory tiers require different trust thresholds: reading episodic memory requires low trust; modifying semantic memory or triggering a consolidation run requires high trust. The model fails gracefully under partial signal loss rather than locking the user out. Section 4.6 details the dispatch.

3.6 The Two-Instance LAN Topology

The thesis originally proposed three instances: mobile, laptop, and a cloud VM. The 2026-05-01 architecture review collapsed this to two instances for a single-user threat model. Table 1 summarises the comparison.

Table 1. Topology comparison: original three-instance plan against the revised two-instance LAN.

The iPhone is structurally a thin client, not interchangeable with a Linux laptop, because iOS background restrictions kill always-thinking daemons regardless of how the app is built. The home server is the heavy head: always-on, full episodic store, consolidation, CRDT merge coordinator, Besu anchor. Tailscale provides WAN reachability without port forwarding or public-IP cloud dependencies, and is registered as the one external trust dependency in an otherwise fully sovereign topology.

4. Methodology

4.1 Tiered Context Assembly Per Turn

From the user's point of view, the conversation never ends. From the system's point of view, every turn is a fresh contextual assembly drawn from tiered memory. The runtime loop embeds the user query, retrieves top-N episodic entries via recency-weighted vector search, retrieves top-M semantic pages via similarity plus graph walk to depth D following structured cross-references, pattern-matches a procedural template if applicable, and assembles a prompt in a fixed order before submitting to the LLM. The new turn is then appended to episodic. The pattern is the answer to the lost-in-the-middle problem at scale: targeted retrieval places only the most relevant content in context where the model uses it, rather than stuffing the full memory store every turn.

4.2 Four-Tier Memory Factoring

Memory splits into four cognitively grounded tiers rather than treating memory as a monolithic vector store. Table 2 summarises the factoring.

Table 2. Four-tier memory factoring with update, retrieval, and decay semantics.

The hippocampus-to-neocortex consolidation pathway in human memory research is the direct biological analogue: short-term traces in the hippocampus are gradually distilled into neocortical structures during slow-wave sleep; the hippocampus is not erased after consolidation but continues to hold autobiographical detail for years. Hydra's episodic-to-semantic consolidation mirrors this pattern algorithmically. The four-tier factoring is the smallest set that distinguishes context-window contents from stored memory, raw experience from distilled knowledge, and facts from procedures.

4.3 Out-of-Band Consolidation

Hydra's consolidation is structurally distinct from the in-band auto-compaction performed silently mid-turn by current chat products. It is scheduled (nightly), batched, runs on a separate process, and is gated by a faithfulness check before any write to canonical semantic memory. Original episodes are never modified or deleted. A consolidation pass selects relevant episodic entries for a topic, loads the existing semantic page if any, generates an updated wiki-shaped output via a frontier-grade model, evaluates every load-bearing claim against source episodes, writes only if faithfulness passes, and audit-logs the run with model, prompt version, and output hash.

The faithfulness target decomposes into six properties: recall (every load-bearing source fact appears in the output or is justifiably excluded), precision (every output claim traces to at least one source episode or to the prior semantic page), calibration (claim confidence matches source-evidence strength), contradiction handling (conflicting episodes surface as explicit callouts rather than silently resolved), cross-link integrity (every structured reference resolves), and versioning (updates preserve historical content). Hallucination is a violation of precision; information loss is a violation of recall; confident-but-wrong consolidation is a violation of calibration.

A web survey across the 2025 to 2026 personal-AI memory literature confirmed the gap that motivates this evaluation: existing systems (MemGPT, Mem0, Letta, Generative Agents, A-MEM, MemoryBank) implement consolidation-shaped operations but evaluate them only via downstream retrieval-and-generation accuracy. The leading benchmarks (LongMemEval, LoCoMo) likewise evaluate end-to-end memory recall, not the faithfulness of the update operation itself. The pre-registered evaluation plan augments LongMemEval (Du et al. 2024) and LoCoMo (Maharana et al. 2024) with consolidation-time faithfulness probes and reports metrics decomposed by the six properties above.

4.4 LLM-Coordinated CRDT Sync

Yjs CRDTs (Shapira et al. 2017) handle deterministic-shape merges: text-level position merges, set unions, and tombstones for deletions. They cannot handle LLM-mediated semantic rewrites. If two replicas independently consolidate the same topic, the resulting outputs will not be byte-identical, may not be semantically identical, and cannot be Yjs-merged into a coherent third page; the CRDT will produce a token-level interleaving that is probably nonsensical. Table 3 summarises the dispatch.

Table 3. Conflict-shape dispatch under coordinated eventual consistency.

The dispatch is by shape of edit, not by which instance produced it. The single-designated-merger pattern names the LLM non-determinism problem explicitly rather than pretending CRDT semantics are intact, and is the right answer for a single-user threat model where the home server naturally aligns with the heavy-head role and the merger queue is small.

4.5 Merkle-Anchored Provenance

Every consolidation pass produces an audit-log entry with inputs, model, prompt version, and output hash. The hash is anchored to a private permissioned blockchain (Hyperledger Besu, EVM-compatible) via a Solidity contract. The provenance log is append-only and externally verifiable: any party with read access to the chain can confirm that a given semantic-page hash existed at a specific block timestamp.

The philosophical thinness of LLM-authored provenance is acknowledged. A signed commit attests to who ran the consolidation and over which inputs, not to the truth of the resulting fact. A consolidation run that hallucinates "the user studied at MIT" produces a cryptographically signed lie. Provenance is necessary but not sufficient for trust; the faithfulness check (Section 4.3) is the other half. The two together produce a provenance-and-faithfulness story that neither alone delivers.

4.6 Multi-Modal Continuous Identity

The identity layer maintains a continuous trust score by Bayesian fusion of two modalities. Voice biometrics use ECAPA-TDNN speaker embeddings (Desplanques et al. 2020) computed from any speech captured by the iPhone. Behavioural fingerprints (typing cadence, navigation patterns, ambient interaction signatures) provide a second modality robust to voice unavailability or background noise. The fused score is updated continuously rather than on a one-shot login event.

Different memory tiers gate at different trust thresholds. Reading recent episodic memory requires low trust. Modifying a semantic page or triggering a consolidation run requires high trust. The model fails gracefully under partial signal loss; loss of one modality does not lock the user out, only constrains the operations available. Identity is scoped out of the M1 prototype and is registered as the second technical Achilles heel after consolidation faithfulness.

5. Prototype Implementation and Milestone Plan

5.1 The M1 Vertical-Slice Prototype

The M1 prototype runs on existing hardware: an HP Pavilion (i7-1065G7, 32GB RAM, MX250 with 2GB VRAM). The decision to start on the desk rather than on a new server is deliberate; the prototype proves the architecture rather than the hardware envelope. WSL2 plus Ubuntu provides the development substrate. Ollama runs llama3.2:3b-instruct-q4_K_M for the assistant and nomic-embed-text for the embedder, both on CPU at four to eight tokens per second, sufficient for interactive chat and weak for production-quality consolidation. A sqlite-vec vector store backs the episodic tier; a 100-page subset of the planned semantic-memory wiki seeds the semantic tier; a Python CLI of approximately 400 lines exercises the full retrieval-assembly-generation loop. Two concrete acceptance tests close M1: a query that requires retrieval from a wiki page, and a follow-up query that requires retrieval from a prior turn. Consolidation in the prototype routes to a frontier API (Claude or GPT) because the local 3B is too weak; the rewrite to a local 32B-plus model is accepted and deferred to M4.

5.2 Milestone Plan

The full thesis timeline is nine months across seven milestones, totalling approximately 27 weeks. The 2026-05-01 architecture review flagged that M5 and M7 are each thesis-sized on their own and recommended padding both up front. Table 4 summarises the milestone plan.

Table 4. Milestone plan with scope and duration.

5.3 Pre-Registered Evaluation Baselines

The thesis pre-registered three baselines for the M2 consolidation evaluation: a vector-RAG baseline (chunk, embed, retrieve, generate without a separate semantic tier), a MemGPT and Letta-class baseline (hierarchical memory with self-editing or filesystem-agent retrieval), and a no-memory baseline (zero context beyond the current turn). The faithfulness benchmark runs on LongMemEval (Du et al. 2024) and LoCoMo (Maharana et al. 2024), augmented with a custom personal-corpus-derived dataset constructed from the planned semantic-memory substrate. Pre-registration before the prototype is built is deliberate: it removes the temptation to retrofit metrics to a result.

6. Results

6.1 Planning-Stage State

This case study reports a planning-stage system rather than a shipped product. The deliverables to date are a 29-page formal thesis, a 2026-05-01 architecture-review session that produced six binding decisions and surfaced four under-specified parts, the planned semantic-memory substrate (a structured LLM-readable wiki maintained as a standalone artifact), and a v0 Python prototype scaffold authored during the review session. I treat the discipline of naming open questions before code is written as a strength rather than a deficiency: an integration system that composes five hard layers is precisely the kind of work where premature implementation produces commitments that survive past their useful life.

6.2 The Six Binding Architectural Decisions

The 2026-05-01 review produced six decisions that are now binding for M1 through M7. Table 5 summarises them.

Table 5. Six binding architectural decisions from the 2026-05-01 review.

6.3 The Four Under-Specified Parts

The review also identified four parts of the thesis that are real but under-specified, each of which will break the architecture if it ships in its current form. Each is registered with a target milestone for tightening. Table 6 summarises them.

Table 6. Four under-specified parts with target tightening milestones.

Two further questions surfaced as honest-accounting items rather than research blockers: the single-user trust model (the three-of-one majority assumption is suspicious) and the philosophical thinness of provenance for LLM-authored knowledge. Both are tracked for the next thesis revision.

6.4 Project Numbers as of 2026-05-02

Table 7 summarises the project numbers at the time of writing. The figures are reported to anchor the case study in the actual planning state, including parts that are designed but unbuilt.

Table 7. Project numbers at the planning-stage snapshot.

7. Discussion

7.1 Architecture-and-Decisions as the Appropriate Register

A reasonable objection to a research-paper-style case study on a planning-stage system is that there is no shipped artifact to evaluate. I address this directly. Hydra composes five layers (inference, memory, sync, provenance, identity) where each layer is a thesis topic on its own. The integration is the contribution. At this stage, the integration exists as a coherent set of architectural decisions, with each decision tied to an honest trade-off and a target tightening milestone. Reporting the architecture and the decisions before the build is not a deferral of evaluation; it is the form of evaluation appropriate to this stage. I treat the discipline of pre-registering open questions as a strength of the work rather than a deficiency.

7.2 The Iron Man Ethos and the Load-Bearing Scope Principle

Thesis §15 contains the load-bearing sentence for every scoping decision Hydra forces:

*the Iron Man fantasy is a north star. The shipped artifact is one carefully chosen slice of it, defended on its own merits.*

The sentence does three things in one breath. It honours the dream (Jarvis is a north star, not a delusion). It bounds the work (the artifact is one slice, not the whole). And it names the standard (defended on its own merits, not on the appeal of the larger fantasy). The sentence is the antidote to the well-documented failure mode of personal-AI projects: announce, demo, chase audience requests, sprawl, ship nothing defendable. The Ethos says the artifact under evaluation is not the silhouette but the slice. For Hydra-the-vision, the slice is the architecture and decisions reported here. For Hydra-the-FYP (if pursued), the slice is consolidation faithfulness on a real benchmark against published baselines. Each slice is defendable on its own merits.

7.3 Limitations

7.4 Future Work

8. Conclusion

I have presented Project Hydra, a sovereign personal AI architecture targeting the binding property of memory that does not forget. The system factors into five layers (Inference, Memory, Sync, Provenance, Identity) and four cognitively grounded memory tiers (working, episodic, semantic, procedural) mapped onto the human hippocampus-to-neocortex consolidation pathway. Cross-device synchronisation uses Yjs CRDTs for deterministic-shape merges and a single designated merger for LLM-mediated semantic rewrites, an honest answer to the LLM non-determinism problem. Every commit to semantic memory is anchored to a private permissioned blockchain (Hyperledger Besu, EVM-compatible) for verifiable provenance. Identity uses multi-modal continuous trust via ECAPA-TDNN voice embeddings and behavioural fingerprints fused by Bayesian update. The system is in the planning stage, supported by a 29-page formal thesis, six binding architectural decisions from a 2026-05-01 review, four explicitly under-specified parts registered for tightening, and a 9-month, 7-milestone build plan. I have argued that an architecture-and-decisions case study is the appropriate register for a research-grade integration system at this stage, that pre-registering open questions and evaluation baselines is a strength of the work, and that the Iron Man Ethos (the artifact is one carefully chosen slice of the fantasy, defended on its own merits) is the load-bearing scope principle that keeps the work shippable. The next paper will report the consolidation-faithfulness benchmark.

References

---

*The Iron Man fantasy is a north star. The shipped artifact is one carefully chosen slice of it, defended on its own merits.*