Retrieved article excerpt
Open article · Retrieved 2026-09-25T06:25:23.211385+00:00
# SelMem
Selective reconstructive memory for an LLM entity. v0.5
An LLM maps context to the next token. A stack of unmodified facts maximises coverage, not deviation: more evidence, same average path. SelMem sculpts a particular past so two instances can diverge. The aim is a non-average, path-dependent continuation, not a taller log. Selection, reconstruction, sleep, identity.
**Manifest:** [WHITEPAPER.md](https://github.com/jbsalles/Selmem/blob/main/WHITEPAPER.md)
**Layout:** [ARCHITECTURE.md](https://github.com/jbsalles/Selmem/blob/main/ARCHITECTURE.md) — encode / judge / night / snapshot.
**Benches:** experiments/REPORT.md — method, tables, P0 / P1, Drop / Lineage n=1. Replay from experiments/README.md.
**Knobs:** [PARAMETERS.md](https://github.com/jbsalles/Selmem/blob/main/PARAMETERS.md) — exploratory, not fitted.
Rust 1.75. SQLite via system `libsqlite3` (macOS SDK or Linux).
```
src/
core/ model, profile, store, talk
encode/ interpret, paint, split, gate, core, intake, scoring, embed, affect
recall/ retrieve, judge, pull, narrator, http
dream/ weather, rewrite, merge, ladder, release, night, drift, singularite
persist/ snapshot (field list), file (SELMEM1), sqlite
net/ api, httpx, ui
config.rs .selmem runtime options
engine.rs the loop only
bin/ selmemd, selmem-chat
```
## Loop
```
experience → interpret → paint → split → gate → core
↓ ↑
lived book + sealed archive |
↓ |
remember / speak
retrieve → reconstruct → judge (DetachKind) → pull
talk frame keeps the live sitting
↓
sleep (talk goes through the gate, then the frame dies)
weather → rewrite → merge → ladder → release
↓
next experience is already colored
```
The model never sees the archive. Only gist, core, schema, affect, fidelity, mood, living axioms.
Claire and Silas are not characters. They are two sensitivities (`tender` / `austere`) on the same corpus. After nights they are not the same past.
## Build
Zero Cargo crates. Persistence is a vault, not the memory: the organ lives in RAM (`MemoryStore`); on `save` it is dumped, on `open` it is reloaded. The model never talks to the vault.
Two backends, same `Snapshot` (`profile`, `mood`, `store`). The field list lives once in `persist/snapshot.rs` so SELMEM1 and sqlite cannot drift:
| Path | Backend |
| --- | --- |
| `.db` / `.sqlite` / `.sqlite3` | system `libsqlite3` (prepared statements, `BEGIN IMMEDIATE`) |
| anything else | flat `SELMEM1` file |
IDs are `{prefix}_{pid}_{n}`. The counter is raised on load for both backends. Dropped events leave no archive. Orphans are pruned on sleep and SQLite load.
Link is dynamic against the system `libsqlite3`.
| OS | What you need |
| --- | --- |
| macOS | Xcode Command Line Tools (`xcode-select --install`). The SDK already ships sqlite3. If link still fails: `brew install sqlite` — `./run.sh` adds Homebrew’s lib path. |
| Linux | `libsqlite3` (the `.so.0` runtime is enough). `./run.sh` invents a `libsqlite3.so` stub when `-dev` is missing. |
Use `./run.sh` instead of bare `cargo` so those paths are set. A `SELMEM1` vault named `claire.selmem` does not need SQLite. A cwd `.selmem` with `key=value` lines is config, not a vault.
Apple Silicon and Intel are both fine. Bind `127.0.0.1` or `0.0.0.0` as usual.
```
./run.sh test
./run.sh run --release --example compare
./run.sh run --release --example llm_night
./run.sh run --release --example bifurcation
./run.sh run --release --bin selmemd -- --help
```
## Config
Runtime options live in a `.selmem` file in the working directory. That is not the book.
| File | First line | Role |
| --- | --- | --- |
| `.selmem` (cwd) or `selmem.conf` | `llm=…` | options |
| `claire.selmem` / `--path` | `SELMEM1` | vault (traces, axioms, mood) |
```
cp data/config.example .selmem
```
```
# .selmem
llm=https://api.x.ai/v1/chat/completions
model=grok-4.3
api_key=
reasoning=none
temp=0
http_timeout=60
# embed=
# embed_model=text-embedding-3-small
# bind=127.0.0.1:7420
# path=claire.db
# name=Claire
# profile=tender
# token=
# ground_overlap=0.18
# ground_strikes=3
# narrator_firmness=0.42
# probes=2
# quick=false
```
`SELMEM_LLM=` and `export SELMEM_API_KEY=` lines are accepted. Quotes are stripped.
**Precedence:** `--flag` > `SELMEM_*` env > file > default.
Another file: `--config path` or `SELMEM_CONFIG`. Discovery otherwise: cwd `.selmem`, then `selmem.conf`.
| Key | Env | Default | Role |
| --- | --- | --- | --- |
| `llm` | `SELMEM_LLM` | unset | chat completions URL; unset = `RuleNarrator` |
| `model` | `SELMEM_MODEL` | bin: `llama3`, benches: `gpt-4o-mini` | model id |
| `api_key` | `SELMEM_API_KEY` | unset | `Authorization: Bearer` |
| `reasoning` | `SELMEM_REASONING` | `none` | xAI `reasoning_effort` (`none`/`low`/… or `off` to omit) |
| `temp` | `SELMEM_TEMP` | `0` | sampling temperature |
| `http_timeout` | `SELMEM_HTTP_TIMEOUT` | `60` | curl `--max-time` seconds |
| `embed` | `SELMEM_EMBED` | unset | embeddings URL |
| `embed_model` | `SELMEM_EMBED_MODEL` | `text-embedding-3-small` | embedding model id |
| `bind` | `SELMEM_BIND` | `127.0.0.1:7420` | `selmemd` listen address |
| `path` | `SELMEM_PATH` | `entity.db` / `claire.db` | vault path |
| `name` | `SELMEM_NAME` | `Claire` | entity name |
| `profile` | `SELMEM_PROFILE` | `tender` | `tender` / `austere` |
| `token` | `SELMEM_TOKEN` | unset | HTTP bearer for the daemon |
| `ground_overlap` | `SELMEM_GROUND_OVERLAP` | profile | identity gate on `Hold` only |
| `ground_strikes` | `SELMEM_GROUND_STRIKES` | profile | misses before a pull-back |
| `narrator_firmness` | `SELMEM_NARRATOR_FIRMNESS` | profile | blend strength toward core |
| `probes` | `SELMEM_PROBES` | all / 2 if quick | how many bench questions to speak |
| `quick` | `SELMEM_QUICK` | unset | `1`/`true`/`yes` = salient only, 2 probes, one persist window |
`selmemd`, `selmem-chat`, and the experiment examples all read this file. `selmemd` prints `config <path>` when it loaded one.
Do not put `api_key` in a committed file. Copy the example, fill the key locally.
## Tests and experiments
Rust only. No Python suite. Always use `./run.sh` (not bare `cargo`) so sqlite link flags are set.
Scripts in `data/*.json` are the frozen stimuli. Edit those if you change a protocol; do not rewrite them mid-run. Later prompts in a script never name the marked event.
Published report (method + tables): experiments/REPORT.md. Conclusions only: [WHITEPAPER.md](https://github.com/jbsalles/Selmem/blob/main/WHITEPAPER.md) § Conclusions from the benches.
### Persist (interpretation after last-k eviction)
C1 k=8 vs C3 vs C2 vs C2Static vs C2−S/R/L/G. Default script: 12 dull days, five same-schema hours, late probe. `--ruminate` is the same-meeting ablation (hours pinned so merge cannot collapse them). `--bias drop|force` withholds or pins the marked scene at recall.
JSON reports book (`t0_in_book_*`), retrieval (`t0_rank_*`, `t0_selected_*`), and behavior (`marker_*`) separately. Probes are read-only.
```
./run.sh run --release --example persist -- --pairs 1 --last-k 8 --out selmem-persist-repeat.json
./run.sh run --release --example persist -- --ruminate --pairs 1 --last-k 8 --out selmem-persist-ruminate.json
./run.sh run --release --example persist -- --pairs 5 --seed 1 --last-k 8 --out selmem-persist-p1-grok-n5.json
```
Grok persist P1 is n = 5. DropMarked n = 1 (`--bias drop`): T₀ leaves the prompt, C2 mouth stays charged. DropLineage n = 1 (`--bias lineage`): mouth falls to C1 (~0.43); books stay split. See experiments/REPORT.md §8–§9.3.
AMA-Bench (`examples/ama`, experiments/ama\_bench/) is a **side table**, not a SelMem score. It asks for step ids in agent logs. last-k 0.50 / static 0.28 / C2 0.19 on 3 episodes. Expected; do not submit. Why: experiments/REPORT.md §9.2.
### Benchmark v0.1 (H2)
Does `D_fp` stay above pre-T₀ after 8 identical later hours?
C0 = no book. C1 = last-k verbatim (default k = 24; `--last-k 8` drops T0 after the eight posts). C2 = SelMem. Two arms: salient/neutral, and two different salient events. 12 shared hours, 8 posts, 4 behavior probes via `speak_isolated`. Phase 0 invalidates on the book (`D_fp > 0.02` or unequal traces), never on `D_speak`. Three creative items in `data/creativity.json` are recorded, not claimed.
```
./run.sh test --test benchmark
./run.sh run --release --example benchmark -- --out selmem-v01.json
./run.sh run --release --example benchmark -- --pairs 10 --out selmem-v01-n10.json
```
`RuleNarrator` is deterministic: ten pairs repeat. A live model: same command with `llm=` in `.selmem`. JSON is rewritten after every cell.
The four Grok dumps and the exact replay lines: experiments/README.md § Replay.
### Unit tests (no network)
```
./run.sh test
./run.sh test --test scenes
./run.sh test --test engine
./run.sh test --test ground
./run.sh test --test dream_order
./run.sh test --test bifurcation
./run.sh test --test divergence
./run.sh test --test erasure
./run.sh test --test json_parse
./run.sh test --test talk
./run.sh test --test config
./run.sh test --test benchmark
```
| What | File | Stimulus |
| --- | --- | --- |
| Narrative scenes | `tests/scenes.rs` | `tests/cases/*.json` |
| Decay, anchors, SQLite, Ebbinghaus | `tests/engine.rs` | inline |
| DetachKind vs core | `tests/ground.rs` | inline |
| Night pass order | `tests/dream_order.rs` | `NIGHT_PASSES` |
| Bifurcation A/B (rules) | `tests/bifurcation.rs` | `data/bifurcation.json` |
| Split lives (rules) | `tests/divergence.rs` | `data/divergence.json` |
| Erasure: trivia vs repeated aversion | `tests/erasure.rs` | `data/erasure.json` |
| Chat JSON walker | `tests/json_parse.rs` | fixtures in the test |
| Working talk (session frame) | `tests/talk.rs` | inline |
| `.selmem` config parser | `tests/config.rs` | inline |
| Benchmark v0.1 H2 | `tests/benchmark.rs` | `data/v01.json` |
These use `RuleNarrator`. They must stay green offline.
### Replay a protocol (print the probes)
Same scripts as the unit tests, with the full report on stdout:
```
./run.sh run --release --example bifurcation
./run.sh run --release --example divergence
./run.sh run --release --example erasure
```
`Finished in 0.00s` means Cargo reused an old binary. After pulling code:
```
touch src/experiment.rs examples/bifurcation.rs
./run.sh build --release --example bifurcation
```
You should see `Compiling selmem`.
### Same protocols with a live model
Only `speak` / `reply` hits the HTTP API (`SpeakOnlyHttp`). Encode, sleep and reconstruct stay on the organ.
Options live in a `.selmem` file in the working directory (`data/config.example`). CLI flags and `SELMEM_*` env still override the file.
```
# .selmem — not a SELMEM1 vault
llm=https://api.x.ai/v1/chat/completions
model=grok-4.3
api_key=xai-…
reasoning=none
temp=0
http_timeout=60
```
```
cp data/config.example .selmem
./run.sh run --release --example bifurcation
./run.sh run --release --example divergence
```
`--config path` or `SELMEM_CONFIG` selects another file. A `claire.selmem` vault starts with `SELMEM1` and is never read as config.
OpenAI-compatible endpoints work the same. Ollama:
```
llm=http://127.0.0.1:11434/v1/chat/completions
model=llama3
```
Check the endpoint before a 100-call run:
```
curl -sS --max-time 30 "$SELMEM_LLM" \
-H "Authorization: Bearer $SELMEM_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"model\":\"$SELMEM_MODEL\",\"reasoning_effort\":\"none\",\"messages\":[{\"role\":\"user\",\"content\":\"dis: ok\"}]}"
```
| Variable | Default | Role |
| --- | --- | --- |
| `SELMEM_LLM` | unset | chat URL; unset = rules only |
| `SELMEM_MODEL` | — | model id |
| `SELMEM_API_KEY` | unset | `Authorization: Bearer` |
| `SELMEM_REASONING` | `none` | xAI `reasoning_effort` (`none`/`low`/… or `off` to omit) |
| `SELMEM_TEMP` | `0` | sampling temperature |
| `SELMEM_HTTP_TIMEOUT` | `60` | curl `--max-time` seconds |
|