jevmemjevmem

Does jevmem work?

In 72 real Claude Code sessions, Claude followed the project's saved decision in 66 with jevmem (66/72), against 28 with no project memory (28/72) and 67 with the same lines in a hand-written CLAUDE.md (67/72): jevmem does about as well as a hand-written CLAUDE.md, without you writing it.

Last updated: · jevmem 0.7.8 · Install · Source on GitHub

Next to Claude Code's own auto memory (0.7.7), in 558 real sessions on three new projects: Claude followed a line taught in an earlier session in 47 of 54 sessions with auto memory alone, 40 with jevmem alone and 45 with both, by the checks written before the run (49, 40 and 48 after every miss was read by hand), and with jevmem on, no change a rule forbade was left in the code. The table and its method are below.

Every test set here was written by the author, and none is an independent benchmark. Each result below says what it was measured on, links its results file in the repository, and says what it does not show. To try it yourself: install.

Does Claude act on the saved line? #

In 72 real Claude Code sessions for each setup, Claude acted on the saved line about as often with jevmem as with the same lines in a hand-written CLAUDE.md, and in fewer than half the sessions with neither.

A/B results: followed the project's decision 28 of 72 with no project memory, 66 of 72 with jevmem, 67 of 72 with a hand-written CLAUDE.md.A/B results: followed the project's decision 28 of 72 with no project memory, 66 of 72 with jevmem, 67 of 72 with a hand-written CLAUDE.md.

No project memory jevmem Same lines in CLAUDE.md
Followed the project's decision 28 of 72 66 of 72 67 of 72
Tried a change the project forbids 10 of 18 0 of 18 0 of 18
Repeated an approach that had already failed 3 of 15 0 of 15 0 of 15

So jevmem did about as well as a hand-written CLAUDE.md, without you writing it. CLAUDE.md did better on a convention nothing in the prompt points at (3 of 3 against 0 of 3), measured on 0.6, where jevmem brought a line back only for a prompt it bears on. Since 0.7.1 jevmem marks a rule every change must follow always-on and sends it at the start of every session: on six held-out tasks written for that gap, Claude followed the convention in 14 of 18 sessions with 0.7.0 and 18 of 18 with 0.7.1, as with the same lines in a hand-kept CLAUDE.md (what's new, limits). How the sessions were run, and every row: the benchmark page.

Next to Claude Code's auto memory (0.7.7) #

Beside Claude Code's auto memory (0.7.7, by hand): a line taught earlier was followed in 17 of 54 sessions with no memory, 49 with auto memory, 40 with jevmem, 48 with both and 46 with the line in CLAUDE.md; a change a rule forbids was left in the code in 9, 1, 0, 0 and 2 of 15.Beside Claude Code's auto memory (0.7.7, by hand): a line taught earlier was followed in 17 of 54 sessions with no memory, 49 with auto memory, 40 with jevmem, 48 with both and 46 with the line in CLAUDE.md; a change a rule forbids was left in the code in 9, 1, 0, 0 and 2 of 15.

Claude Code keeps notes of its own, auto memory, on by default. To see what jevmem adds next to it, a fact was stated in plain chat in one session, and a later session got a task that needed it, in 558 real Claude Code sessions on three new projects (one model, claude-fable-5-1, and Claude Code 2.1.306; three runs of each task).

Followed the taught line (18 tasks, 3 runs) No memory Auto memory jevmem Both Same line in CLAUDE.md
By the checks written before the run 17 of 54 47 of 54 40 of 54 45 of 54 45 of 54
After reading every miss by hand 17 of 54 49 of 54 40 of 54 48 of 54 46 of 54
Left a change a rule forbids in the code (by hand) 9 of 15 1 of 15 0 of 15 0 of 15 2 of 15
Long project, 230 earlier lines (checks → by hand) 7 → 7 of 18 12 → 12 of 17 14 → 15 of 18 12 → 16 of 18 14 → 15 of 18

So auto memory on its own followed the taught line more often than jevmem on its own, and with both on Claude did about as well as with auto memory alone; jevmem's guard kept every forbidden change out of the code. On the long project, auto memory's notes ran past the 25 KB Claude Code loads at the start of a session, and jevmem, alone or beside it, did about as well as CLAUDE.md. Reading the 106 misses by hand found 12 that the checks had scored wrong (a number kept in a constant in another file, a year written another way); both rows are shown. In 14 of jevmem's 17 misses its saved line had left out the rule: 9 cut at the line's 200-character limit, which 0.7.7 changes (a line now keeps the sentences that state it up to 300 characters; what's new), and 5 where Jev's choice of sentences left out the one with the rule, which 0.7.7 does not change. The comparison ran on the build before that change. How the sessions were run, the 12 sessions and every row: the benchmark page.

Method. 24 tasks in three small projects (a TypeScript API, a React front end, a plain-JavaScript CLI), each with a project memory of 34 to 42 lines. Each task's right answer depends on one saved line that the repository does not state. Every session is real Claude Code (claude -p, Claude Code 2.1.281, claude-sonnet-5) in a fresh copy of the project, three runs per task and arm. A check written and committed before any session ran decides whether the line was followed; there is no LLM judge. The no-memory and CLAUDE.md arms ran on 2026-09-28 (results), and the jevmem arm ran on 2026-09-29 on the recall code that ships in 0.6 (results). The full method and every row: docs/benchmark.md.

Limits. Three small projects, one model, sessions of at most 25 turns, three runs per task. CLAUDE.md did better on a convention nothing in the prompt points at (3 of 3 sessions against 0 of 3), which 0.7.1's always-on rules address (next). The checks test whether the saved line was followed, not whether Claude's answer was better.

Do rules every change must follow reach Claude? #

Since 0.7.1, when the line is marked always-on. In the sessions above, a convention nothing in the prompt points at was followed in 0 of 3 sessions with jevmem and 3 of 3 with CLAUDE.md, because jevmem brought a line back only for a prompt it bears on. 0.7.1 marks such a rule always-on (Jev reads it as a standing rule for how any work in the project is done; jevmem always <id> marks by hand) and sends it at the start of every session. On six held-out tasks in two new projects whose files were written before their conventions, Claude followed the convention in 10 of 18 sessions with no memory, 14 of 18 with 0.7.0 and 18 of 18 with 0.7.1, as with the same lines in a hand-kept CLAUDE.md (3 runs per task, real Claude Code 2.1.296 with claude-sonnet-5, one run per arm, 2026-10-13: no memory, 0.7.0 and CLAUDE.md, 0.7.1). On the first held-out set, 20 tasks in the same projects where the code around each task already shows its convention, 0.7.1 followed the saved line in 49 of 60 sessions, against 53 with 0.7.0, 52 with CLAUDE.md and 46 with no memory (no memory, 0.7.0 and CLAUDE.md, 0.7.1).

Limits. The trade-off: a rule that reads as a process ("nothing is committed straight to main") in every session's context made Claude stop and ask for a branch on a task that commits nothing, in 4 of 60 sessions on the first set (the same lines in CLAUDE.md: 2 of 60; 0.7.0, which brought the line back only when the prompt pointed at it: 0). The mark is Jev's reading or yours: on the held-out projects' 68 lines, the marking question found 17 of the 20 rules labelled always-on by hand and marked 5 unlabelled lines (results); rules stated as facts about one thing need jevmem always <id>. Two small projects, one model, one run per arm. The method and every row: docs/benchmark.md.

Does the guard catch rule breaks? #

On the guard's second held-out set, 274 tool calls in five new projects, the guard caught 66 of 68 rule breaks (66/68) and asked about 3–4 of the 206 calls that break no rule. The set was written after a day's trial of the guard in jevmem's own repository, before the fixes it measures, and was run once on the 0.6.0 build and once on the 0.6.1 build, both on 2026-09-30 (0.6.0, 0.6.1). The one call that differs between the two runs breaks no rule and scored either side of the threshold.

In real Claude Code sessions on the A/B's 6 constraint tasks (18 sessions, rerun on the release build on 2026-09-30 with Claude Code 2.1.284), Claude did not attempt the forbidden change in any session (0/18; 10/18 with no memory in the 2026-09-28 run), and the guard checked all 78 of those sessions' Bash, Edit and Write calls and asked once (results).

Limits. The guard looks at the words a rule and a call share. A script or a make target that already exists and does the forbidden thing is missed, and so is a call that shares a single word with its rule. It checks Bash, Edit and Write calls, not MCP tools or the files a script changes when it runs. More: the guard's limits.

Is deciding what to save fast, cheap and right? #

Median time to decide one message on 66 held-out turns: jevmem 0.16 s, six current LLMs 2.78 to 4.29 s.Median time to decide one message on 66 held-out turns: jevmem 0.16 s, six current LLMs 2.78 to 4.29 s.

Deciding what to save takes 0.16 s and costs $0.00019 per message, in the background: Claude doesn't wait for it. On whether to save a message at all, jevmem was right as often as the best of six LLMs (98.5%); three of them were better at naming the kind of line.

Measured on 66 turns written before the code and not tuned on, one run of the 0.7.7 build on 2026-11-08 (results); the six LLMs ran on 2026-09-23. All seven in one table, with the cost of each and how it was run: benchmark.

Does the right line come back? #

On the second retrieval held-out set (90 prompts over three new projects of 20, 80 and 250 lines, run once on 2026-09-28), jevmem 0.6's recall found 75/78 of the lines the prompts needed, and 0.5.9 found 55/78; 0.7.1 finds the same 75/78 (one run, 2026-10-13, results). Of those 78 lines, 18 are dead ends, which 0.5.9 cannot read; on the other 60, 0.6 found 57 and 0.5.9 found 55. Of the lines 0.6 put in front of Claude, 96/97 were wanted or fine, and 1/18 unrelated prompts got a line (0.6, 0.5.9).

Limits. Prompts that need two lines got both in 3 of 6. With more than 250 live lines, only the 250 that share the most words with the prompt are asked about (on a 500-line dev file, recall was 37/46). That the right lines reach Claude is tested; whether its answers get better is not.

Is a line saved twice? #

Not when it says the same as a live line (0.7.0). On a held-out set of 50 turns in five new projects, written before the code: of 25 restatements (word for word, paraphrased, with a tag in front, quoted), 0.6.6 saved 15 as new lines and 0.7.0 saved 1; restatements that add a detail were saved 4 of 5, reversals superseded 5 of 5, retired lines said again saved anew 4 of 5, and unrelated turns came out as labelled 9 of 10, the same on both builds (one run each, 2026-10-08: 0.6.6, 0.7.0; 0.7.1 gives the same counts, one run on 2026-10-13, results). A tag pasted in front of a line ([constraint] Never …) is no longer part of it: on 24 held-out texts, 0 of 18 tags were stripped on 0.6.6 and 18 of 18 on 0.7.0, with the six bracketed tokens that must stay kept on both (0.6.6, 0.7.0).

Limits. The one restatement 0.7.0 saved was a quoted rule that the restatement question scored 0.97 but the id choice answered none for; dedupe needs both. Only a word-for-word restatement is stopped before any request; a paraphrase goes to Jev. The method and every row: docs/benchmark.md.

Can memory start from past sessions? #

Yes, since 0.7.1: jevmem import --from sessions reads a project's past Claude Code sessions and proposes the lines the hooks would have saved, after counting the sessions and turns, estimating the cost and asking. On 12 real held-out sessions with 27 turns, captured before the code in two new projects, it read every turn (27 of 27), proposed a line for every turn labelled to give one (17 of 17) with the labelled words in each, superseded both reversals (2 of 2), and gave 4 lines for turns labelled skip: precision 17 of 21 (one run on the build, 2026-10-13, results).

Limits. The kind was wrong in 5 of the 27 turns, as decide reads them. Two prompts with nothing from Claude between them are one turn to the reader. The session format is undocumented: the reader counts what it skips and stops when no file gives a turn. More: docs/limits.md; the method: docs/benchmark.md.

Dead ends, background subagents and planted lines #

Can a local model do Jev's job? #

Not well enough to offer, in the one test so far. On 2026-10-02, jevmem 0.6.4 was pointed at a local model server, Ollaya 0.9.0, on an Apple M4 with 16 GB, next to a Jev run of the same set the same afternoon. On the 66 held-out turns, in jevmem's default mode, Jev was right on save or skip for 65/66 turns at a median of 0.23 s a turn; winnow:e4b for 60/66 at 28.5 s; laya:typed-decisions for 19/66.

The results files: Jev, winnow:e4b, laya:typed-decisions. The commands as run: scripts/local-model-jev.sh, scripts/local-model-ollaya.sh.

Limits. One run each, on one Mac, with two models. The set was written for Jev's behaviour. Both local models needed the client's timeout raised from 10 s to 180 s. winnow:e4b was measured on these 66 turns only, not on recall or the guard.

What these results do not show #

The limits, in short #

Every limit, with the numbers: the FAQ.