What Your AI Should Remember, and What It Should Not

Three kinds of memory

• Session: what has been said in this conversation. Essential and uncontroversial.
• Profile: stable facts about this user — their account, their role, their preferences.
• Historical: what happened in past conversations. Useful and the most sensitive.

Session memory needs managing

Long conversations exceed what can be sent economically. Summarise older turns while keeping recent ones verbatim, and keep any decisions or commitments in the summary.

The failure mode is a summary that drops the specific detail the user just gave you. Keep entities and numbers verbatim; compress the discussion around them.

Profile memory should come from your systems

Do not have the assistant remember the customer's account number. Look it up from the CRM at the start of the conversation, from an authenticated identity.

Anything the system already knows should be fetched rather than remembered, because fetched data is current and remembered data drifts.

Historical memory: helpful and hazardous

1. Say plainly that past conversations are retained
2. Let the user see what is stored about them
3. Let them delete it, and honour that properly
4. Set a retention period and enforce it automatically
5. Never surface one user's history to another

The recognition trade-off

“Last time you asked about X” is helpful from a service you use daily and unsettling from one you contacted once. Match the memory to the relationship.

When in doubt, remember less and fetch more. Fetched context feels like good record-keeping; remembered context can feel like surveillance.

Frequently asked questions