What Should an AI System Remember?

It is Tuesday morning and you return to a project after several days of other work. You remember the broad question and can find the documents. An AI can produce a fresh summary in seconds, yet you still have to work out which option the team ruled out, whether an open question was resolved, and which document describes the current decision rather than an earlier possibility.

The difficulty is familiar even without AI. Work is interrupted; priorities move; important reasoning is scattered between meetings, messages and drafts. AI lowers the cost of producing more material around that work. It does not automatically preserve the state of the work itself.

I began building a memory architecture because I was tired of losing this thread. A thought might begin on my reMarkable, continue in a conversation, become a note in Mem and then change a live project. Connecting these tools helped with retrieval. It did not, by itself, answer the more important question: what should be retained, for whom, and with what degree of trust?

Why retrieval fails at handover

When people talk about AI memory, they often imagine a large store of documents and conversations from which the model retrieves useful fragments. Such a store can be valuable. It can return a source, surface an earlier idea or remind us of a decision made months ago.

It may still fail the Tuesday-morning test. A knowledge base can tell us a great deal about a subject without knowing where a particular project stands today. It may contain the minutes of a meeting in which two options were discussed, while leaving the final choice in a later chat. Retrieval gives the agent material. Someone still has to establish which material is authoritative and what should happen next.

For active projects, I now keep a small continuity layer alongside the ordinary working files. It records the current objective, the decisions that still constrain the work, lessons worth carrying forward and questions that remain open. A short entry point tells a returning person or agent where to begin. The layer stays small so that it can serve as a useful handover.

The test is practical: could a capable colleague return tomorrow, avoid reopening settled questions and take the next sensible step? If the answer is no, the project may be well documented while still being difficult to continue.

Different memories do different jobs

I have found it useful to separate five kinds of memory. They interact, but treating them as one undifferentiated store creates confusion.

Source records preserve evidence: the original document, transcript, link, attachment or project artefact. A summary can help us work with a source, but it cannot replace the source when a claim needs checking or the context changes.

Curated knowledge contains ideas and reflections that remain useful beyond the task that produced them. In my own setup, Mem holds much of this material. An article, meeting or experiment does not enter this layer merely because an AI has summarised it. It belongs there when it changes or sharpens something I expect to use again.

Mental models help interpret new situations. A model should expose a mechanism or improve a question, rather than give an attractive label to every new note. If a technically capable AI pilot fails to produce value, for example, the model might direct attention to workflow ownership, data quality or the passage from experiment to routine operations. The model does not supply the answer. It helps us test a better explanation.

Operational memory helps an AI work consistently. It may hold a review sequence, a recurring correction, the current status of a task or a rule about which sources it may use. Much of this should be visible and easy to change. Some of it should expire when the task is over. A one-off instruction should not quietly become a permanent preference.

Project continuity memory describes where live work stands. It answers what we are trying to do, what has been decided, what remains uncertain and what should happen next. This is the layer a returning agent should read before it searches the wider knowledge base for interesting material.

The boundaries protect judgement. An agent can need operational context without absorbing every note I have ever kept. A useful idea can enter curated knowledge without becoming a rule for every future project. A project decision can remain local unless experience shows that it should change the wider way of working.

Promotion into memory needs a decision

The hardest part of a memory system is deciding what crosses those boundaries. AI can capture, summarise, connect and suggest. It can make the case that a meeting revealed a recurring failure or that a new article challenges an established view. Similarity between two notes is a useful prompt for thought, but a weak reason on its own to treat them as connected knowledge.

In my own practice, I try to restate a proposed connection in my own words, check what supports or contradicts it, and ask whether it changes a decision. That review can be short. Its value is that a human has made the promotion from captured material to accepted understanding.

The same principle applies when an agent updates a project. At a meaningful stopping point, it can propose a memory checkpoint. Did the current objective change? Was a durable decision made? Did an experiment teach us something that should affect the next attempt? Has an uncertainty been resolved? The agent can update the relevant file visibly, with a reason, rather than appending a summary of everything that happened.

Most conversations do not need to update every memory container. A system that does so will create a more orderly archive of noise.

Start with one interrupted project

A serious objection is that this architecture could become documentation theatre. More files, more categories and more maintenance can make a system feel responsible while slowing the work it was meant to support.

I would therefore start with one project that people repeatedly leave and return to. Give it a short brief, a current-state note and a decision log. Add a separate list of open questions only if uncertainty is being lost. At the end of a substantial work session, ask the AI to propose only the changes a returning colleague would need. Review those changes before accepting them.

After a few interruptions, test the result. How long does it take to recover the thread? Are settled choices still being reopened? Can someone trace an important claim to its source? Does the agent carry forward a correction appropriately, and can it forget one that was temporary? If the system does not improve those moments, simplify it.

As agents become more capable, this discipline becomes more important. A system that remembers isolated preferences may feel convenient. One that quietly accumulates years of opaque assumptions can become difficult to correct and too easy to trust. Memory should help an agent work from a reliable starting point while leaving a person able to inspect the evidence, revise the interpretation and decide what deserves to endure.