Local-first memory for AI agents: system design

Aletheia (npm package aletheia-mem) is a local-first memory store for AI coding agents. This post explains how it is designed: what runs where, which choices keep it from ever blocking your tool, and where the design stops short.

Four constraints

Four constraints drove every choice below.

1. A hook runs in the host tool’s hot path, so it must never block that tool.
2. Memory can hold secrets, so nothing is stored before it is redacted.
3. Memory has to serve several tools, so the store is tool-neutral.
4. A new session receives context every time, and it costs tokens every time, so the context must be small and predictable.

The pipeline in one picture

Capture runs in a short-lived aletheia hook <tool> <event> process that the host starts. Compression runs in a background worker inside aletheia-server, or when you run aletheia compress. Recall runs in the SessionStart hook, in the MCP server or in aletheia context. The hook opens hub.db itself and does not call the server, so memory capture works while the server is stopped.

Choice 1: the hook writes to SQLite, not to a server

A server in the capture path would add a failure mode to every prompt: if it is down, slow or starting, the hook has to wait or fail. Writing straight to SQLite removes that hop. The cost is that several short-lived processes can write to one file at once, so the hook waits at most 50 ms for a lock. The store itself opens with a 3,000 ms busy timeout (src/hub/store.mjs), and the hook lowers it to 50 ms after opening (PRAGMA busy_timeout = 50, src/hub/hooks/run.mjs). When the database is not available in time, the hook falls back to a spool.

Choice 2: fail open, then spool

The rules for a hook are short:

• It always exits 0. Any internal error is swallowed.
• A watchdog ends the process after 150 ms (setTimeout(() => process.exit(0), 150), bin/aletheia.mjs). It is a timer, so it can only fire while the event loop is free: it cannot interrupt a synchronous read or SQLite call.
• It makes no network call and no model call.
• If it cannot open or write the database, it appends the already-redacted item to spool/hook-spool.jsonl in the data directory, with file mode 0600. The next hook that reaches the database replays the spool first.

During an outage the effect is delay, not loss, for items that reach the spool: the memory appears in search after the next hook that reaches a healthy database. If the watchdog fires while a write is in flight, that item is lost. The reference documentation says so, and I did not trigger it in a test.

Choice 3: redact in a plain function, before the write

Redaction is a pure function (src/hub/redact.mjs) that runs in a fixed order:

1. Text between <private> and </private> becomes [private].
2. Known secret shapes become a marker such as [redacted:github-token]: Anthropic, OpenAI-style and Stripe keys, GitLab, npm and GitHub tokens, JSON Web Tokens, AWS key ids, Slack tokens, Google API keys and PEM private keys.
3. Passwords inside URLs, Authorization: Bearer values of 12 or more characters, and password, passwd, secret, token or api_key assignments with a value of six or more characters become [redacted].

The three groups are what matter. In the code the URL-password rule runs before the GitHub and AWS rules.

I chose patterns over a classifier because a pattern is predictable, fast and testable inside a 150 ms budget. The price is coverage. The documentation lists what slips through:

• A secret under a JSON key, because the quote sits between the key and the colon.
• Variable names with a prefix, such as MY_API_KEY= or DB_PASSWORD=.
• Values shorter than six characters, personal data, and an unclosed <private> tag.

Redaction also does not run on every write path. Hooks, aletheia save, aletheia capture, POST /v1/events, the MCP save_memory tool, transcript import and aletheia watch redact. The REST route POST /v1/memories, the dashboard note box, edits, bundle imports and the claude-mem import store text exactly as sent. In a test on 2026-10-29 the first two stored a fake key unchanged.

Choice 4: compress later, in a queue with leases

Raw prompts and tool events are cheap and noisy. Turning a finished session into facts and a summary is slower, so it never runs inside a hook.

1. The Stop hook adds the session to a small queue and extracts nothing.
2. The worker in aletheia-server ticks every 2 seconds and turns each entry into a job row of type extract_session.
3. It claims one job at a time with a 60 second lease (LEASE_MS = 60_000). If the process dies, the lease expires and the job is claimable again. The job lives in a row, so a killed process loses nothing.
4. A failed job retries with exponential backoff for at most five attempts (MAX_ATTEMPTS = 5). The waits are 1, 2, 4 and 8 seconds. The formula caps at 60 seconds, which five attempts never reach.
5. A job reads the summary and note memories that were not extracted yet, splits them into windows of 20 items and 12,000 characters by default, extracts facts, builds a session summary and a project summary, and merges near-duplicates at a similarity of 0.92 or higher.

The default extractor is rule-based and makes no network call. A model provider is optional. One negative result is worth knowing: a first local extraction run with a 1B model returned HTTP 500 after 4 minutes 41 seconds and produced no facts. The rule extractor added 1,672 facts and moved Recall@5 from 57.3% to 57.7%, while the benchmark’s own facts added 7.3 points. The rule extractor’s move is about 0.4 points (57.3% to 57.7% in the README text; its own table shows the baseline as 57.4%). Source: README, run of 2026-10-26.

Choice 5: recall is a bounded, labelled index, not a dump

Each section has a cap: up to 50 pinned memories, 20 recent ones by default (100 at most), up to five recent session requests, up to ten active facts and every enabled rule that is not limited to other tools. By default (context.facts set to auto) the facts that qualify exclude rule-extracted facts, which are stored with a confidence of 0.45. A title is cut at 100 characters and the whole block is cut at 6,000 characters, from the end, so a long block loses its legend, hint and economics line first.

Three design details matter more than the caps.

• Progressive disclosure. The block is an index of one line per memory. The legend and the fetch hint teach the agent the pattern search, then timeline, then get_observations, so full text is fetched only when needed.
• Data, not instructions. The block sits inside <aletheia-memory> tags with a notice that the content is data. A copy of the tag inside a stored memory is made inert, so a stored note cannot close the block early.
• An honest cost line. Aletheia estimates tokens as characters divided by four, and says so. In the demo above the block is about 194 tokens against about 132 for the text of the listed items, so it is larger. A real saving needs memories with long narratives.

Choice 6: search is keyword search plus facts, merged by rank

A query becomes content words. A word found in more than 20% of all memories is dropped when a rarer word remains, and at most 12 of the rarest words are kept. SQLite FTS5 ranks matches with BM25 over title, subtitle, narrative, facts and concepts, with the tokenizer porter unicode61 remove_diacritics 2, so “retries” matches “retry”. Facts have their own index and come back newest first. The CLI merges the two lists with reciprocal rank fusion, weight 1 for facts and 0.9 for memories (FUSION in src/hub/search.mjs, where the memory weight is named episodeWeight). Filters apply before ranking, and recency, pins and importance scores do not change the order of memories. Semantic search with a local embedding model is optional, off by default and available in the CLI only.

The data model in one table

Every row has a project and a scope, project by default or global after promotion. Dismiss hides a row without deleting it, and aletheia erase deletes on purpose and gives a signed receipt.

What fails, and what happens

Numbers

All numbers are measured by the author and carry their source.

• At 50,000 synthetic rows, keyword search p95 was 32.7 ms project-scoped and 58.6 ms across all projects (Linux, 12 cores, 2026-10-27, bun bench/scale.mjs 50000 20000 --index tuned --queries 200 --seed 1).
• On the LoCoMo questions, raw turns reached Recall@5 of 57.4% at 3.72 ms per query. That row is the hybrid search (keywords plus embeddings), not the keyword-only search described above. Keyword-only was 55.3% at 2.00 ms. One log file shows 44.7% for raw turns, and the benchmark documentation says the cause is unresolved. All of it is retrieval, not answer accuracy.

What I checked, and what I did not

I read the constants in this post in the code on 2026-10-31: the 150 ms watchdog in bin/aletheia.mjs, the 50 ms busy timeout in src/hub/hooks/run.mjs, the lease, the attempt limit and the backoff in src/hub/queue.mjs, the 0.92 merge threshold in src/hub/dedupe.mjs, the fusion weights in src/hub/search.mjs and the 6,000-character default in src/hub/store.mjs. The terminal outputs ran with aletheia-mem 0.1.10 in a throwaway folder on 2026-10-30.

I did not trigger the watchdog mid-write, so “that item is lost” is the documentation’s statement, not mine. Hooks have run in real sessions of Claude Code, Command Code and Codex CLI, on one machine and for one user; no hook has run inside Cursor, Gemini CLI, Windsurf or OpenCode. Answer accuracy is not measured. Sync across two real machines is not tested.

What I would do differently

1. Redact on every write path from the start. Today the REST route, the dashboard note box, edits and imports store text as sent.
2. Make the settings match the code. The Settings screen lists “Redact before storing” and “Excluded projects”, and the capture code reads neither value. Redaction is always on in its paths, and exclusion works through ALETHEIA_EXCLUDED_PROJECTS. I would remove the two controls or wire them, so the screen cannot say something the code does not do.
3. Fuse results the same way everywhere. The CLI fuses facts and memories; REST and MCP do not. One ranking would be easier to reason about.

This post was drafted with AI assistance. I reviewed every claim and ran the commands shown.

About the project: Aletheia.

Mohamed Ashraf Elsaed
Technical Lead at Convertedin, Cairo. I build tools for AI coding agents and write about Laravel, Node and engineering.