How to Give an
AI Agent
a Memory
5 layers of memory. Episodic recall. Self-critique. Semantic search. A system that learns from every decision.
I used to hold human brains in my hands.
Three pounds. Wrinkled like a walnut left in water. Oxford anatomy lab, two in the morning. Just me, a cadaver and questions that wouldn't stop circling.
I'd trace a single neuron from the cortex all the way down to the fingertip. An unbroken thread. Thought to touch.
The brain isn't a computer. Every metaphor you've absorbed — files and folders, storage and retrieval, input and output — they got us far. But they're not the whole story. The brain is a jungle. Dense. Pulsing with things growing and dying and competing for sunlight.
Donald Hebb figured this out in 1949. Seven words: neurons that fire together wire together.
I taught this at Oxford. And now — seventy-six years later — we can build it.
Then I met a community building brains in code.
Reuven Cohen — 82 Rust crates. Each one a neuron. RuVector for synapses that strengthen when they fire together. AgentDB for episodic memory that captures everything. ReasoningBank for the thalamic relay that filters signal from noise. SAFLA for the brainstem — the safety layer that cannot fail.
He mapped neuroscience to software architecture. And gave it away. Open source.
I joined the Agentics Foundation. London chapter. A community of practitioners building the next layer of AI infrastructure. Not consultants. Builders.
Inspired by this work, I built my own.
The architecture of a brain
that learns.
LLM Reasoning Layer
The slow, deliberate thinker. Language, planning, abstract thought. The LLM handles what the cortex handles — conscious, expensive reasoning.
analyst.evaluate(context, indicators)
Episodic Memory
Records everything. Doesn't judge importance. Damage it and you're frozen in an eternal present. Every decision captured with full context.
entry_reasoning, exit_reasoning, context, confidence_at_entry, outcome_tags
Reflexion Engine
The relay station. Good outputs reinforce pathways. Bad outputs trigger correction. Post-decision critique via LLM.
reflexion: "Entry timing premature — divergence not confirmed" lesson_type: "timing"
Semantic Search (FAISS)
Paths that fire together wire together. Three namespaced indices with staleness decay. The system sculpts itself around use.
namespaces: [reasoning, wiki, raw] staleness: score *= 1/(1 + 0.01 × days)
Safety & Guardrails
Ancient. The systems that cannot fail. Hard limits, emergency stops, automatic rollback. The autonomic layer that keeps the system alive — even when the cortex hallucinates.
max_exposure: 5% daily_loss_limit: -2% → circuit_break blackout: [high_impact_events]
Cohen built a brain.
I built one that evolves.
Skill Library
Procedural memory. Motor patterns that run without conscious thought. Patterns auto-promote at 5+ observations with >60% success rate. The agent develops muscle memory.
pattern: "momentum_divergence" success_rate: 0.73 sample_size: 12 status: "active"
Provenance Tracking
Which data sources actually predicted well? Credit the winners, discount the losers. Not all information is equal. The brain learns who to trust.
source: "economic_calendar" accuracy: 0.71 predictions: 38
Regime Snapshots
When the environment returns to something seen before, reload the priors. Like how a smell triggers a childhood memory. Snapshot on regime change, reload on regime return.
regime: "risk_off"
snapshot: { skills: [...], provenance: [...] }
restored_from: "2026-03-15"When a decision closes, seven things happen automatically.
Input arrives — cortex analyses
Decision executes — hippocampus records
Outcome observed — thalamus critiques
Patterns extracted — cerebellum updates
FAISS re-indexes — synapses strengthen
Provenance scores update — dopamine fires
Loop. The brain improves while it runs.
This is what learning looks like. Not a training run. Not a fine-tune. Continuous. Emergent. Alive.
What nature already knew.
Episodic memory isn't a database pattern — it's the hippocampus.
We didn't invent "capture everything with context." Biology did. We just finally have the compute to implement it. I held this structure in my hands at Oxford. Now it's a Python class.
Self-critique isn't a prompting trick — it's the thalamus.
Every sensory signal gets filtered before reaching consciousness. Good signals reinforce. Bad ones correct. Cohen calls it "bounded reasoning." Neuroscientists called it the thalamic relay 150 years ago.
Forgetting is a feature, not a bug.
Staleness decay in FAISS mirrors synaptic pruning — the brain actively discards what isn't reinforced. We had to build this deliberately. The brain does it automatically.
Muscle memory is real in software.
The skill library auto-promoting patterns after enough observations — that's the cerebellum. Procedural knowledge that runs without conscious thought. The agent develops intuition. Not because we programmed it, but because the architecture produces it as an emergent property.
The scalpel's in your hand.
Not replacing the human. Making the human unstoppable.