Memorilla: learned memory tokens for frozen LLMs

state: significantheat: lowuncertainty: mediumrelevance: highllm-memory semantic-memoryStanford SNAP

What is this?

Memorilla is a trainable memory module from Stanford-linked researchers, including Jure Leskovec, described in an October 15, 2026 arXiv paper and released with code, data, and checkpoints on GitHub and Hugging Face under Apache 2.0 for the code. It encodes document collections into reusable embeddings and, for each query, produces a fixed set of learned memory tokens for a frozen LLM. The release reports results on eight benchmarks, including stronger average performance than methods such as Mem0 and MemoRAG, and a FactKG result of 84.33% versus 80.43% for full context. These are the authors’ reported evaluations, not independently confirmed here. The supplied material also describes a TextWorld experiment; sources differ on the agent model size (Qwen3-8B in the repository’s general setup versus Qwen3-4B in the web result).

Why it matters

A learned, query-specific memory could let an LLM use long histories or large collections without adding their full contents to every prompt—potentially reducing latency and token costs while enabling persistent personalization or agents that learn from experience. The reported results are promising, but depend on trained modules, substantial GPU resources, and benchmark evidence; they do not establish reliable performance on arbitrary collections. Personalization also raises a consequential question: which past details should a system retrieve and surface in a new context?

Current assessment

Stanford-linked researchers published Memorilla in an arXiv paper dated October 15, 2026, and released code, checkpoints, and benchmark materials; the code is Apache-2.0 licensed. The system encodes a document collection and gives a frozen LLM a fixed set of query-dependent memory tokens. The project reports strong benchmark results, including 84.33% versus 80.43% for full context on FactKG using 108 times fewer tokens, and improved TextWorld agent success; these are author-reported results, not independent confirmation. Training and reproduction require substantial GPU resources. Generalization to arbitrary collections, reliability, privacy controls, and real-world adoption remain open questions. The newer Hacker News discussion adds interest and skeptical debate, but no material new implementation or validation evidence; attention has cooled from its peak.

Measured heat

now 0 pts/hpeak 20 pts/hcomments 0/hpeers p0momentum: steady2 platformsage 749h

points/hour across evidence · reading as of 2026-11-15 05:24:09.021861+11:00 · deterministic, not a model opinion

How the heat travelled

10-14 13:00	⭐ origin echo-reconstructed	The earliest primary artifact located is the arXiv paper, submitted October 15, 2026 (v1). Its abstract says: “Here we introduce Memorilla, Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli, Nikita Mounier, Shirley Wu, Shayan Pardis, and Jure Leskovec on paper (echo) · attributed from hn.story.50391235	—
11-06 11:50	first on hacker news · published · +550.8h	Memorilla: Latent Semantic Memory for LLMs tomislavpet	—
11-06 11:50	amplified on hacker news	hn.story.50391235 tomislavpet	peak 6 · 1 comments · 5% of case engagement
11-06 18:03	amplified on hacker news 👑	hn.story.50395919 handfuloflight	peak 125 · 13 comments · 95% of case engagement
11-06 12:20	our radar first saw it · +551.4h	discovery anchor: hn.story.50391235	—

pace: p64 vs 553 stories at the 720h mark (now 749h old) — ahead of n2-lookahead-ssd-moe-apple-silicon (1.0x), behind jeff-code-agent-loop-acceleration (1.0x)

Evidence (3) — ⭐ canonical anchor

source	object	author	score	comments
🟧 hn	Memorilla: Latent Semantic Memory for LLMs Retrieved article excerpt Open article · Retrieved 2026-11-06T12:22:10.928694+00:00 # Memorilla: Latent Semantic Memory for LLMs [Paper](https://arxiv.org/abs/2610.21375) [Website](https://memory.stanford.edu/) [Hugging Face](https://huggingface.co/memorilla) [License: Apache-2.0](https://github.com/snap-stanford/memorilla/blob/main/LICENSE) This repository provides code for training and evaluating Memorilla, a memory system that gathers information from an entire document collection into a fixed number of query-dependent memory tokens for a frozen LLM. The code is based on our paper [Memorilla: Latent Semantic Memory for LLMs](https://arxiv.org/abs/2610.21375). A frozen LLM can only use what fits in its context window, so Memorilla reads an entire document collection, such as a user's past chats or a long book, and hands the LLM just 16 memory tokens that carry what each question needs. With this small budget, it achieves the best average performance against methods such as Mem0 and MemoRAG, and on FactKG it is even more accurate than giving the LLM every document (84.33% vs. 80.43%) while using 108× fewer tokens. Furthermore, Memorilla can also help an agent keep learning from its own history, raising its average success rate on TextWorld games from 18.6% to 24.2%. [Memorilla architecture](https://github.com/snap-stanford/memorilla/blob/main/assets/architecture.png) ## Contents - [Installation](https://github.com/snap-stanford/memorilla#installation) - [Getting started](https://github.com/snap-stanford/memorilla#getting-started) - [Training and Evaluation](https://github.com/snap-stanford/memorilla#training-and-evaluation) - [Checkpoints](https://github.com/snap-stanford/memorilla#checkpoints) - [Using your own data](https://github.com/snap-stanford/memorilla#using-your-own-data) - [PersonalizationV4](https://github.com/snap-stanford/memorilla#personalizationv4) - [Reinforcement learning on TextWorld](https://github.com/snap-stanford/memorilla#reinforcement-learning-on-textworld) - [Documentation](https://github.com/snap-stanford/memorilla#documentation) - [Development](https://github.com/snap-stanford/memorilla#development) - [Citation](https://github.com/snap-stanford/memorilla#citation) - [License](https://github.com/snap-stanford/memorilla#license) ## Installation Memorilla needs Linux with CUDA GPUs and Python 3.11; the recipes were run on NVIDIA H100 80GB GPUs. Install it with [uv](https://docs.astral.sh/uv/): ``` $ git clone https://github.com/snap-stanford/memorilla.git && cd memorilla $ uv venv --python 3.11 && source .venv/bin/activate $ uv pip install -e . $ uv pip install --no-build-isolation flash-attn==2.7.4.post1 ``` FlashAttention-2 and optional extras The last command installs FlashAttention-2, which training uses by default. Evaluation does not need it, and training can use PyTorch's attention instead: ``` $ scripts/train.sh configs/stage1_enwiki.yaml --attn_implementation sdpa ``` Optional extras: ``` $ uv pip install -e ".[dev]" # tests, linters and pre-commit $ uv pip install -e ".[wandb]" # Weights & Biases logging $ uv pip install -e ".[personalizationv4]" # OpenAI client for the PersonalizationV4 pipeline $ uv pip install -e ".[rl]" # TextWorld, for the RL data tools ``` ## Getting started Here is an example from the PersonaMem-v2 test set. The user has had 107 earlier conversations with an assistant, about 32,000 tokens in total, and now asks a new question. Memorilla reads all of the conversations and hands them to a frozen Qwen3-8B as just 16 memory tokens, about 2,000 times fewer. You need one 80 GB GPU, and the first run downloads about 25 GB of models and the 37 MB test set. ``` from memorilla.benchmarks import PERSONA_PROMPT from memorilla.data import ensure_data, load_split from memorilla.vllm import MemoryVLLM row = load_split("pmv2", "test", ensure_data("pmv2", ["test"]))[73] # one user's question and past conversations model = MemoryVLLM.from_pretrained("memorilla/Memorilla-Qwen3-8B", "pmv2") # Qwen3-8B, the memory module and the encoder answer = model.answer(row["question"], row["documents"], system_prompt=PERSONA_PROMPT) print(f"Question: {row['question']}\nMemorilla: {answer}") ``` Output: > Question: What are some easy breakfast ideas that will keep me full all morning? > > Memorilla: Since you’ve got a family history of type 2 diabetes, you might enjoy breakfasts that balance > protein, healthy fats, and fiber to keep you full without spiking blood sugar—like scrambled eggs with avocado and a > side of roasted veggies, or Greek yogurt with chia seeds and berries. These options give you steady energy while > keeping your blood sugar in check. Note The user never says that diabetes runs in their family. The only clue is in one of the 107 conversations, where they asked whether diabetes on the mother's side of the family raises a person's risk more than on the father's side. Memorilla connects that clue to a breakfast question that has nothing to do with it, while Qwen3-8B without the memory answers with a generic list of breakfasts. This is just one example. Because the LLM always receives the same 16 memory tokens, no matter how many documents are stored, the approach can potentially scale to millions of tokens. In the paper, Memorilla compresses books and movie scripts from NarrativeQA by more than 4,000x. On one H100, growing a collection from 1 to 512 documents (about 128K tokens) leaves Memorilla's response time at 0.74 seconds, while putting all the documents in the prompt slows the model down from 0.75 to 19.9 seconds, 27x slower. The other checkpoints listed under [Checkpoints](https://github.com/snap-stanford/memorilla#checkpoints) load the same way. ## Training and Evaluation The recipes below train the memory module from scratch, and evaluates it on benchmarks. The data and its precomputed embeddings are downloaded from [`memorilla/Memorilla-Data`](https://huggingface.co/datasets/memorilla/Memorilla-Data) on first use. ``` # Train the three stages with the Qwen3-8B decoder on 4 GPUs. $ scripts/train.sh configs/stage1_enwiki.yaml $ scripts/train.sh configs/stage2_mixture.yaml $ scripts/train.sh configs/stage3_multitask.yaml # Continue on the personalisation benchmarks. $ scripts/train.sh configs/personalization_pv4.yaml $ scripts/train.sh configs/personalization_pmv2.yaml # Evaluate a checkpoint on every benchmark (or name one, e.g. factkg). $ scripts/evaluate.sh runs/stage3/epoch-04 all $ scripts/evaluate.sh runs/personalization_pv4/epoch-04 pv4 # Text-only baselines: closed book, RAG over the top-5 documents, full context. $ scripts/baselines.sh closed_book all $ scripts/baselines.sh rag all --top_k 5 $ scripts/baselines.sh full_context triviaqa ``` On 4 H100 80GB GPUs, the three stages take about 19 hours and download about 310 GiB of data and embeddings, and evaluating a checkpoint on all eight benchmarks takes about 15 minutes on one GPU. The [training](https://github.com/snap-stanford/memorilla/blob/main/docs/training.md) and [evaluation](https://github.com/snap-stanford/memorilla/blob/main/docs/evaluation.md) guides describe the main options, and each script's `--help` lists all of them. ## Checkpoints The trained memory modules are on Hugging Face at [`memorilla/Memorilla-Qwen3-8B`](https://huggingface.co/memorilla/Memorilla-Qwen3-8B). All were trained with the Qwen3-8B decoder and the Qwen3-Embedding-4B encoder, the defaults of `train.py` and `evaluate.py`. \| Checkpoint \| Folder \| Recipe \| \| --- \| --- \| --- \| \| Stage 1 \| `stage1/` \| `configs/stage1_enwiki.yaml` \| \| Stage 2 \| `stage2/` \| `configs/stage2_mixture.yaml` \| \| Stage 3 (`epoch-00`) \| `stage3/` \| `configs/stage3_multitask.yaml` \| \| PersonalizationV4, from Stage 1 \| `pv4/` \| `configs/single_task.yaml` \| \| PersonaMem-v2, from Stage 1 \| `pmv2/` \| `configs/single_task.yaml` \| ``` $ hf download memorilla/Memorilla-Qwen3-8B --local-dir checkpoints $ scripts/evaluate.sh checkpoints/pv4 pv4 # evaluate a checkpoint $ scripts/train.sh configs/stage3_multitask.yaml --init_memory checkpoints/stage2 # continue from Stage 2 ``` ``` from memorilla import MemoryModule memory = MemoryModule.from_pretrained("checkpoints/stage3") ``` ## Using your own data Any collection of documents with questions and answers can be trained and evaluated without code changes. Write one parquet file per split, named like `train-00000-of-00001.parquet`, under `my_data/data/my_task/`, with one row per question and the columns `collection_id`, `question`, `answer` and `documents`, then pass the directory as the data repository. The embeddings are computed on first use. ``` $ python train.py --config configs/single_task.yaml --data_repo my_data --datasets my_task --output_dir runs/my_task $ python evaluate.py --data_repo my_data --config my_task --split test --scoring generation --checkpoint runs/my_task/epoch-04 ``` The `single_task` recipe warm-starts from the Stage 2 checkpoint; pass `--init_memory ""` to train from scratch. The [data guide](https://github.com/snap-stanford/memorilla/blob/main/docs/data.md#using-your-own-data) covers the file layout, document chunking and scoring options. ## PersonalizationV4 PersonalizationV4 (PV4) is a synthetic personalisation benchmark released with Memorilla. Each of its 149 fictional users has had 200 short conversations with an AI assistant, and each question places the user in a new scenario and asks what they would most likely do or prefer. The questions are written to require combining at least two facts about the user, which the conversations show but never state. The data is on [Hugging Face](https://huggingface.co/datasets/memorilla/PersonalizationV4), and [`personalizationv4/`](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/README.md) documents the task, the data format and the generation pipeline. ## Reinforcement learning on TextWorld Memorilla can also serve as the memory of an agent and learn from reward alone. [`rl/`](https://github.com/snap-stanford/memorilla/blob/main/rl/README.md) trains the memory module with GRPO on [TextWorld](https://github.com/microsoft/TextWorld) text adventures, using [SkyRL-Memorilla](https://github.com/Adibvafa/SkyRL-Memorilla), a fork of [SkyRL](https://github.com/NovaSky-AI/SkyRL), and documents its installation, data, training and evaluation. [Replay of a held-out TextWorld game with and without Memorilla](https://github.com/snap-stanford/memorilla/blob/main/assets/textworld_replay.gif) A schematic replay of one held-out game (routes simplified, observations paraphrased). With only its full history in context, the agent revisits explored rooms until the 50-turn limit; with 8 trained memory tokens added, it reaches the goal. ## Documentation \| Guide \| Contents \| \| --- \| --- \| \| [Data](https://github.com/snap-stanford/memorilla/blob/main/docs/data.md) \| Released configs, row format, embedding layout, storage locations, using your own data \| \| [Training](https://github.com/snap-stanford/memorilla/blob/main/docs/training.md) \| Options, recipes and stages, outputs and resuming, multiple GPUs and SLURM, single-task training \| \| [Evaluation](https://github.com/snap-stanford/memorilla/blob/main/docs/evaluation.md) \| Benchmarks, metrics, outputs, and the closed-book, RAG and full-context baselines \| \| [PersonalizationV4](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/README.md) \| Task and scoring, data format, generation pipeline, regeneration, license \| \| [Reinforcement learning](https://github.com/snap-stanford/memorilla/blob/main/rl/README.md) \| TextWorld setup, installation, data, training and evaluation \| \| [Troubleshooting](https://github.com/snap-stanford/memorilla/blob/main/docs/troubleshooting.md) \| Multi-GPU stalls, slow evaluation, offline machines \| ## Development ``` $ uv pip install -e ".[dev]" $ pytest # unit tests (CPU only, no network)	tomislavpet	6	1
🟧 echo.paper ⭐	The earliest primary artifact located is the arXiv paper, submitted October 15, 2026 (v1). Its abstract says: “Here we introduce Memorilla,	Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli, Nikita Mounier, Shirley Wu, Shayan Pardis, and Jure Leskovec	—	—
🟧 hn	Memorilla: Latent Semantic Memory for LLMs	handfuloflight	125	13

source

object

author

score

comments

🟧 hn

Memorilla: Latent Semantic Memory for LLMs

Retrieved article excerpt

Open article · Retrieved 2026-11-06T12:22:10.928694+00:00

# Memorilla: Latent Semantic Memory for LLMs

[Paper](https://arxiv.org/abs/2610.21375)
[Website](https://memory.stanford.edu/)
[Hugging Face](https://huggingface.co/memorilla)
[License: Apache-2.0](https://github.com/snap-stanford/memorilla/blob/main/LICENSE)

This repository provides code for training and evaluating Memorilla, a memory system that gathers information from an
entire document collection into a fixed number of query-dependent memory tokens for a frozen LLM. The code is based on
our paper *[Memorilla: Latent Semantic Memory for LLMs](https://arxiv.org/abs/2610.21375)*.

A frozen LLM can only use what fits in its context window, so Memorilla reads an entire document collection, such as a
user's past chats or a long book, and hands the LLM just 16 memory tokens that carry what each question needs. With this
small budget, it achieves the best average performance against methods such as Mem0 and MemoRAG, and on FactKG it is
even more accurate than giving the LLM every document (84.33% vs. 80.43%) while using 108× fewer tokens. Furthermore, Memorilla can also help an agent keep learning from its own history, raising its average
success rate on TextWorld games from 18.6% to 24.2%.

[Memorilla architecture](https://github.com/snap-stanford/memorilla/blob/main/assets/architecture.png)

## Contents

- [Installation](https://github.com/snap-stanford/memorilla#installation)
- [Getting started](https://github.com/snap-stanford/memorilla#getting-started)
- [Training and Evaluation](https://github.com/snap-stanford/memorilla#training-and-evaluation)
- [Checkpoints](https://github.com/snap-stanford/memorilla#checkpoints)
- [Using your own data](https://github.com/snap-stanford/memorilla#using-your-own-data)
- [PersonalizationV4](https://github.com/snap-stanford/memorilla#personalizationv4)
- [Reinforcement learning on TextWorld](https://github.com/snap-stanford/memorilla#reinforcement-learning-on-textworld)
- [Documentation](https://github.com/snap-stanford/memorilla#documentation)
- [Development](https://github.com/snap-stanford/memorilla#development)
- [Citation](https://github.com/snap-stanford/memorilla#citation)
- [License](https://github.com/snap-stanford/memorilla#license)

## Installation

Memorilla needs Linux with CUDA GPUs and Python 3.11; the recipes were run on NVIDIA H100 80GB GPUs. Install it with
[uv](https://docs.astral.sh/uv/):

```
$ git clone https://github.com/snap-stanford/memorilla.git && cd memorilla
$ uv venv --python 3.11 && source .venv/bin/activate
$ uv pip install -e .
$ uv pip install --no-build-isolation flash-attn==2.7.4.post1
```

FlashAttention-2 and optional extras

The last command installs FlashAttention-2, which training uses by default. Evaluation does not need it, and training
can use PyTorch's attention instead:

```
$ scripts/train.sh configs/stage1_enwiki.yaml --attn_implementation sdpa
```

Optional extras:

```
$ uv pip install -e ".[dev]"                 # tests, linters and pre-commit
$ uv pip install -e ".[wandb]"               # Weights & Biases logging
$ uv pip install -e ".[personalizationv4]"   # OpenAI client for the PersonalizationV4 pipeline
$ uv pip install -e ".[rl]"                  # TextWorld, for the RL data tools
```

## Getting started

Here is an example from the PersonaMem-v2 test set. The user has had 107 earlier conversations with an assistant,
about 32,000 tokens in total, and now asks a new question. Memorilla reads all of the conversations and hands them to
a frozen Qwen3-8B as just 16 memory tokens, about 2,000 times fewer. You need one 80 GB GPU, and the first run
downloads about 25 GB of models and the 37 MB test set.

```
from memorilla.benchmarks import PERSONA_PROMPT
from memorilla.data import ensure_data, load_split
from memorilla.vllm import MemoryVLLM

row = load_split("pmv2", "test", ensure_data("pmv2", ["test"]))[73]  # one user's question and past conversations

model = MemoryVLLM.from_pretrained("memorilla/Memorilla-Qwen3-8B", "pmv2")  # Qwen3-8B, the memory module and the encoder
answer = model.answer(row["question"], row["documents"], system_prompt=PERSONA_PROMPT)
print(f"Question: {row['question']}\nMemorilla: {answer}")
```

Output:

> **Question:** What are some easy breakfast ideas that will keep me full all morning?
>
> **Memorilla:** Since you’ve got a family history of type 2 diabetes, you might enjoy breakfasts that balance
> protein, healthy fats, and fiber to keep you full without spiking blood sugar—like scrambled eggs with avocado and a
> side of roasted veggies, or Greek yogurt with chia seeds and berries. These options give you steady energy while
> keeping your blood sugar in check.

Note

The user never says that diabetes runs in their family. The only clue is in one of the 107 conversations, where they
asked whether diabetes on the mother's side of the family raises a person's risk more than on the father's side.
Memorilla connects that clue to a breakfast question that has nothing to do with it, while Qwen3-8B without the
memory answers with a generic list of breakfasts.

This is just one example. Because the LLM always receives the same **16 memory tokens**, no matter how many documents
are stored, the approach can potentially scale to millions of tokens. In the paper, Memorilla compresses books and
movie scripts from NarrativeQA by **more than 4,000x**. On one H100, growing a collection from 1 to **512 documents
(about 128K tokens)** leaves Memorilla's response time at **0.74 seconds**, while putting all the documents in the
prompt slows the model down from 0.75 to **19.9 seconds, 27x slower**.

The other checkpoints listed under [Checkpoints](https://github.com/snap-stanford/memorilla#checkpoints) load the same way.

## Training and Evaluation

The recipes below train the memory module from scratch, and evaluates it on benchmarks. The data and its precomputed embeddings are downloaded from
[`memorilla/Memorilla-Data`](https://huggingface.co/datasets/memorilla/Memorilla-Data) on first use.

```
# Train the three stages with the Qwen3-8B decoder on 4 GPUs.
$ scripts/train.sh configs/stage1_enwiki.yaml
$ scripts/train.sh configs/stage2_mixture.yaml
$ scripts/train.sh configs/stage3_multitask.yaml

# Continue on the personalisation benchmarks.
$ scripts/train.sh configs/personalization_pv4.yaml
$ scripts/train.sh configs/personalization_pmv2.yaml

# Evaluate a checkpoint on every benchmark (or name one, e.g. factkg).
$ scripts/evaluate.sh runs/stage3/epoch-04 all
$ scripts/evaluate.sh runs/personalization_pv4/epoch-04 pv4

# Text-only baselines: closed book, RAG over the top-5 documents, full context.
$ scripts/baselines.sh closed_book all
$ scripts/baselines.sh rag all --top_k 5
$ scripts/baselines.sh full_context triviaqa
```

On 4 H100 80GB GPUs, the three stages take about 19 hours and download about 310 GiB of data and embeddings, and
evaluating a checkpoint on all eight benchmarks takes about 15 minutes on one GPU. The [training](https://github.com/snap-stanford/memorilla/blob/main/docs/training.md) and
[evaluation](https://github.com/snap-stanford/memorilla/blob/main/docs/evaluation.md) guides describe the main options, and each script's `--help` lists all of them.

## Checkpoints

The trained memory modules are on Hugging Face at
[`memorilla/Memorilla-Qwen3-8B`](https://huggingface.co/memorilla/Memorilla-Qwen3-8B). All were trained with the
Qwen3-8B decoder and the Qwen3-Embedding-4B encoder, the defaults of `train.py` and `evaluate.py`.

| Checkpoint | Folder | Recipe |
| --- | --- | --- |
| Stage 1 | `stage1/` | `configs/stage1_enwiki.yaml` |
| Stage 2 | `stage2/` | `configs/stage2_mixture.yaml` |
| Stage 3 (`epoch-00`) | `stage3/` | `configs/stage3_multitask.yaml` |
| PersonalizationV4, from Stage 1 | `pv4/` | `configs/single_task.yaml` |
| PersonaMem-v2, from Stage 1 | `pmv2/` | `configs/single_task.yaml` |

```
$ hf download memorilla/Memorilla-Qwen3-8B --local-dir checkpoints
$ scripts/evaluate.sh checkpoints/pv4 pv4                                          # evaluate a checkpoint
$ scripts/train.sh configs/stage3_multitask.yaml --init_memory checkpoints/stage2  # continue from Stage 2
```

```
from memorilla import MemoryModule

memory = MemoryModule.from_pretrained("checkpoints/stage3")
```

## Using your own data

Any collection of documents with questions and answers can be trained and evaluated without code changes. Write one
parquet file per split, named like `train-00000-of-00001.parquet`, under `my_data/data/my_task/`, with one row per
question and the columns `collection_id`, `question`, `answer` and `documents`, then pass the directory as the data
repository. The embeddings are computed on first use.

```
$ python train.py --config configs/single_task.yaml --data_repo my_data --datasets my_task --output_dir runs/my_task
$ python evaluate.py --data_repo my_data --config my_task --split test --scoring generation --checkpoint runs/my_task/epoch-04
```

The `single_task` recipe warm-starts from the Stage 2 checkpoint; pass `--init_memory ""` to train from scratch. The
[data guide](https://github.com/snap-stanford/memorilla/blob/main/docs/data.md#using-your-own-data) covers the file layout, document chunking and scoring options.

## PersonalizationV4

PersonalizationV4 (PV4) is a synthetic personalisation benchmark released with Memorilla. Each of its 149 fictional
users has had 200 short conversations with an AI assistant, and each question places the user in a new scenario and
asks what they would most likely do or prefer. The questions are written to require combining at least two facts about
the user, which the conversations show but never state. The data is on
[Hugging Face](https://huggingface.co/datasets/memorilla/PersonalizationV4), and
[`personalizationv4/`](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/README.md) documents the task, the data format and the generation pipeline.

## Reinforcement learning on TextWorld

Memorilla can also serve as the memory of an agent and learn from reward alone. [`rl/`](https://github.com/snap-stanford/memorilla/blob/main/rl/README.md) trains the
memory module with GRPO on [TextWorld](https://github.com/microsoft/TextWorld) text adventures, using
[SkyRL-Memorilla](https://github.com/Adibvafa/SkyRL-Memorilla), a fork of [SkyRL](https://github.com/NovaSky-AI/SkyRL),
and documents its installation, data, training and evaluation.

[Replay of a held-out TextWorld game with and without Memorilla](https://github.com/snap-stanford/memorilla/blob/main/assets/textworld_replay.gif)

A schematic replay of one held-out game (routes simplified, observations paraphrased). With only its full history in
context, the agent revisits explored rooms until the 50-turn limit; with 8 trained memory tokens added, it reaches the
goal.

## Documentation

| Guide | Contents |
| --- | --- |
| [Data](https://github.com/snap-stanford/memorilla/blob/main/docs/data.md) | Released configs, row format, embedding layout, storage locations, using your own data |
| [Training](https://github.com/snap-stanford/memorilla/blob/main/docs/training.md) | Options, recipes and stages, outputs and resuming, multiple GPUs and SLURM, single-task training |
| [Evaluation](https://github.com/snap-stanford/memorilla/blob/main/docs/evaluation.md) | Benchmarks, metrics, outputs, and the closed-book, RAG and full-context baselines |
| [PersonalizationV4](https://github.com/snap-stanford/memorilla/blob/main/personalizationv4/README.md) | Task and scoring, data format, generation pipeline, regeneration, license |
| [Reinforcement learning](https://github.com/snap-stanford/memorilla/blob/main/rl/README.md) | TextWorld setup, installation, data, training and evaluation |
| [Troubleshooting](https://github.com/snap-stanford/memorilla/blob/main/docs/troubleshooting.md) | Multi-GPU stalls, slow evaluation, offline machines |

## Development

```
$ uv pip install -e ".[dev]"
$ pytest                             # unit tests (CPU only, no network)

tomislavpet

🟧 echo.paper ⭐

The earliest primary artifact located is the arXiv paper, submitted October 15, 2026 (v1). Its abstract says: “Here we introduce Memorilla,

Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli, Nikita Mounier, Shirley Wu, Shayan Pardis, and Jure Leskovec

—

🟧 hn

Memorilla: Latent Semantic Memory for LLMs

handfuloflight

125

Interpretation history

2026-11-09T04:26:17Z
The additional attention is discussion around the existing release, not evidence that the method has been independently validated or adopted. Memorilla remains a consequential open-source research result with promising author-reported efficiency and benchmark performance, while practical generality and reproduction remain unsettled.
2026-11-06T18:24:28Z
The additional Hacker News post repeats the existing Memorilla release and adds no independent validation or implementation result. The research remains consequential for learned, query-specific memory, but attention has cooled and the key limits remain unchanged.
2026-11-06T12:22:44Z
audience grounding: high — A learned, query-specific memory could let an LLM use long histories or large collections without adding their full contents to every prompt—potentially reducing latency and token costs while enabling persistent personalization or agents that learn from experience. The reported results are promising, but depend on trained modules, substantial GPU resources, and benchmark evidence; they do not establish reliable performance on arbitrary collections. Personalization also raises a consequential question: which past details should a system retrieve and surface in a new context?

Decision trace

11-09 15:26repriceThe additional attention is discussion around the existing release, not evidence that the method has been independently validated or adopted. Memorilla remains a consequential open-source research res
11-09 15:26jev_reprice_gatechanges_anything noul=0.15 would_skip=False population=cold_direct
11-07 06:25sensor_dirtyvelocity_spike
11-07 05:24repriceThe additional Hacker News post repeats the existing Memorilla release and adds no independent validation or implementation result. The research remains consequential for learned, query-specific memor
11-07 05:22attention_candidateattach
11-07 05:22attachThis is additional coverage of the already tracked Memorilla release, not a separate development.
11-07 05:21propose_attachThis is additional coverage of the already tracked Memorilla release, not a separate development.
11-06 23:24attention_candidatecreate
11-06 23:24promote_anchororigin walk conf 0.95
11-06 23:22groundA learned, query-specific memory could let an LLM use long histories or large collections without adding their full contents to every prompt—potentially reducing latency and token costs while enabling
11-06 23:22createThe released code and checkpoints make a concrete memory architecture available, and its substantial but author-reported benchmark claims are worth assessing without treating them as independently val