How Sapience implements Complementary Learning Systems
The problem, in one paragraph
A neural network stores everything in shared weights, so there is no such thing as writing one memory without touching others: gradient descent on new data marches straight through the representations the old knowledge depends on. This is the stability–plasticity dilemma, and it is structural, not an engineering bug. The field's mitigations each concede part of it — regularization saturates after a few tasks; naive rehearsal means retraining on everything forever; adapters used as fact containers sidestep integration; bigger context windows avoid learning entirely, and collapse beyond the window (we measured this directly). The brain hit the same wall evolutionarily, and its answer was two systems with different learning rates and a managed interface. The 1995 CLS paper is explicit that the hippocampus exists because of catastrophic interference.
The blueprint: eight CLS mechanisms and their implementations
| Brain mechanism | What it does | Sapience implementation |
|---|---|---|
| Sleep replay | Hippocampus re-fires the day's episodes at ~20× speed, teaching cortex offline; silence the ripples and consolidation fails (Girardeau 2009) | Consolidation over the store; replay corpora rendered from the store for weight adaptation (measured below) |
| Memory typing (H.M.) | Episodic, semantic, procedural memory are separable systems, visible in one patient | Typed knowledge objects; episodic and semantic tiers with different decay |
| Consolidation gradient | Memories migrate from hippocampal dependence to cortical storage over time (Ribot's law) | Knowledge moves from raw capture to consolidated, schema-linked objects |
| Reconsolidation | A retrieved memory becomes rewritable before re-storage — update, not duplicate | Supersession edges: what replaced what, tracked at write time |
| Salience gating | The amygdala tags what's worth keeping; most of experience is deliberately not kept | The write gate. Selection is the intelligence, not a compromise |
| Pattern separation / completion | Similar episodes stored as distinct codes; whole memories retrieved from partial cues | Distinct typed objects + hybrid associative retrieval |
| Schema-dependent consolidation | New information that fits existing structure consolidates far faster (Tse 2007) | Structured store accelerates rather than archives |
| Working memory vs. the stores | Prefrontal cortex holds a few items at high metabolic cost; the fix for its limit is routing, not enlargement | The context window is working memory; scaling it is the wrong axis |
The implementation, measured
Reading: the store carries long-context reasoning
With identical model weights, a frontier reader answers 3-hop questions at 34.4% from a raw ~1M-token window and at 66.7% from ~700 tokens selected by the store (n=90 item-paired, five judges from three vendors agree on every row). The same system holds a flat ~82% band from 2M to 10M tokens of stored history — the reader's working set stays bounded while history grows 2,500-fold — while at the 10M tier the same reader over its raw window scores 0/20 with the answer verified present in what it reads. Beyond the window the mechanism is physical: truncated serving drops gold-passage availability to 2–3%, while iterative retrieval holds it at ~34%.
Updating: supersession beats relevance
On real Wikipedia revision histories, where facts change and the current value is defined by the edit stream, the store's supersession structure beats naive relevance retrieval by +40.4 points — the baseline retrieved both the stale and the current value and could not tell which was current. The reconsolidation analogue at the weight level: restricting replay to store-current memories removed every observed stale response (0/81 superseded probes) at non-inferior retention, and a volume-matched control does not reproduce the effect — it follows from which content is excluded, not from replaying less.
Learning: sleep replay in the weights
The base model stays frozen — it is the neocortex. Learning happens in a LoRA adapter, and the training data for new knowledge is interleaved with replay rows rendered from the episodic store: the store decides what gets rehearsed, as the hippocampus decides what the cortex hears at night. Sequential fine-tuning forgets ~85% of previously-correct items; store-generated replay cuts that to ~10% at formally matched acquisition — pooled protection of 47–75 points less forgetting, replicated across a second model family, a second domain pair, and a real git-history corpus. The controls make it CLS rather than generic rehearsal: matching the training dose or the optimizer steps without the replayed content restores full forgetting. Remove the replay, consolidation fails — Girardeau's experiment, run in the weights.
What ships today, what's in the lab
The store, the write gate, hybrid retrieval, supersession, and consolidation run in production over a frozen, swappable reader — the same system has run on frontier models and on an 8B open model. The weight-adaptation layer is measured at the mechanism level (≤8B open models under LoRA, with a qualified 72B extension) and is not yet in the shipped product. The paper reports negative results and falsified predictions alongside the positives, and every headline number passed an adversarial audit protocol before publication; scope statements travel with each claim.