Separating the two memory layers of the OpenClaw agent and what changed

When building a multi-turn agent for the first time — I did this too — most people just prepend the entire history to the front of the prompt. The first few turns work fine. But past 30 or 50 turns, the context window overflows, the agent forgets what it said earlier, or token costs spike with every request. You end up endlessly patching with ad-hoc trimming, wondering "where should I cut?"

The root of this problem is using the context window as storage. As the Mem0 team put it, the context window is RAM, not disk. RAM is fast but small and volatile. Treating it like persistent disk storage will inevitably break things.

The solution is to clearly separate two layers. The short-term buffer serves as working memory for the current session; the long-term vector store serves as persistent memory that survives session boundaries. The agent loop is designed to read from and write to both layers in order, every turn. This article walks through implementing that separation inside an OpenClaw agent loop.

Why the Two Layers Have Different Roles

Short-term memory only needs to know about the current task

Short-term memory holds the conversation from the current session, the state of work in progress, and the system prompt. It offers sub-millisecond access but disappears when the session ends. OpenClaw's State object or a Python dictionary is sufficient. Attaching Redis or DragonflyDB allows sharing across multiple agent instances and enables crash recovery.

Long-term memory handles what needs to be known across past sessions

Long-term memory consists of past facts, summaries, and procedural knowledge indexed by embeddings. It is stored in vector databases like Qdrant, Pinecone, or Chroma, and retrieved at runtime via semantic similarity search. Even months after a session, you can ask "what was it that this user said they preferred last time?"

That is why the loop matters

Simply using two stores is different from having the agent loop coordinate both layers in order. The key to in-loop separation is that every turn explicitly follows this sequence:

Without this pipeline, the two stores are just independent databases — not a memory layer. The code examples ahead follow this sequence directly. The logic for deciding "what to store in which layer" is covered separately in a later section.

When to Introduce Three Layers

In practice, going deeper reveals that long-term memory itself contains things of different character. There is a trend in academia and industry moving beyond a two-layer model toward a three-layer model, dividing into episodic, semantic, and procedural layers.

The tool-use agent research (arXiv 2512.07287) fleshed out a hybrid architecture that stores episodic and procedural memory separately, then retrieves procedural memory when executing a new task and incorporates it into the execution plan.

However, introducing three layers is not free. Storage schemas, retrieval pipelines, and success/failure labeling logic all multiply. I recommend crossing over to three layers only after meeting the following criteria:

• Episodic separation: Sessions span multiple days, and there is a real need to query by time axis — "what happened and when."
• Procedural separation: Tool-call sequences repeat, and the same failure patterns start becoming visible. Store successful cases in a separate store and inject them into the next planning phase.

Until then, two layers are sufficient. The code examples in this article focus on two layers.

Attaching This to the OpenClaw Loop in Practice

The code below is a conceptual example combining Mem0 and the Anthropic SDK. Adaptation is required to fit the OpenClaw agent structure; parts specific to OpenClaw's native API are marked as conceptual examples.

Initializing the Memory Client

from mem0 import MemoryClient
from anthropic import Anthropic
 
# Long-term memory — Mem0 (supports ADD/UPDATE/DELETE semantics)
mem0_client = MemoryClient(api_key="your-mem0-key")
 
# Short-term buffer — in-process (session-scoped)
# Conceptual example: managed via OpenClaw's State object
class ShortTermBuffer:
    # Notes on the max_turns default:
    # Tune this so that (average message tokens × 2 × max_turns) + system prompt
    # + long-term retrieval results fit within your target input token budget,
    # taking into account model context size, average message length,
    # and per-turn token cost. The value below is just an example.
    def __init__(self, max_turns: int = 20):
        self.turns = []
        self.max_turns = max_turns
 
    def append(self, role: str, content: str):
        self.turns.append({"role": role, "content": content})
        if len(self.turns) > self.max_turns:
            self.turns = self.turns[-self.max_turns:]
 
    def to_messages(self) -> list:
        return self.turns

Assembling Context Inside the Loop

This is the core logic that reads from both layers each turn to assemble context. The reason long-term memory retrieval results are also returned to the caller is covered in the observability section below.

def build_context(
    user_input: str,
    short_term: ShortTermBuffer,
    user_id: str,
    top_k: int = 5,
):
    """Combines the short-term buffer and long-term vector search results to build LLM input."""
 
    long_term_results = mem0_client.search(
        query=user_input,
        user_id=user_id,
        limit=top_k,
    )
    long_term_context = "\n".join(
        f"- {r['memory']}" for r in long_term_results
    )
 
    system_prompt = "You are a helpful agent."
    if long_term_context:
        system_prompt += (
            f"\n\n[Relevant information retrieved from long-term memory]\n{long_term_context}"
        )
 
    messages = list(short_term.to_messages())
    messages.append({"role": "user", "content": user_input})
 
    return system_prompt, messages, long_term_results

Running the Loop and Deciding What to Persist

The Anthropic SDK is used as-is. The model ID should be pinned to the version actually in use to get reproducible results (specify the ID you are actually using at the time of deployment as an example).

The key thing to notice in agent_turn is the persistence decision. Without translating the "decide what to store" step from the earlier diagram into code, idle chatter and confirmatory responses accumulate in the vector store. Here, a simple heuristic combined with LLM tagging is applied.

llm_client = Anthropic()
MODEL_ID = "claude-sonnet-5-YYYYMMDD"  # Conceptual example: replace with the actual pinned date suffix
 
def should_persist(user_input: str, assistant_reply: str) -> bool:
    """Classifies whether a turn is worth writing to the long-term store."""
    # Minimum length filter
    if len(user_input.strip()) < 8 or len(assistant_reply.strip()) < 8:
        return False
 
    # Persistence tagging — ask the LLM briefly
    tagging = llm_client.messages.create(
        model=MODEL_ID,
        max_tokens=8,
        messages=[{
            "role": "user",
            "content": (
                "If the following conversation turn contains a user preference, fact, decision, "
                "or plan, answer YES; otherwise answer only NO.\n\n"
                f"USER: {user_input}\nASSISTANT: {assistant_reply}"
            ),
        }],
    )
    return "YES" in tagging.content[0].text.upper()
 
 
def agent_turn(
    user_input: str,
    short_term: ShortTermBuffer,
    user_id: str,
):
    system_prompt, messages, retrieved = build_context(
        user_input, short_term, user_id
    )
 
    response = llm_client.messages.create(
        model=MODEL_ID,
        max_tokens=1024,
        system=system_prompt,
        messages=messages,
    )
    assistant_reply = response.content[0].text
 
    # Short-term buffer — always update (maintain current session context)
    short_term.append("user", user_input)
    short_term.append("assistant", assistant_reply)
 
    # Long-term store — write only when worth persisting
    persisted = False
    if should_persist(user_input, assistant_reply):
        mem0_client.add(
            messages=[
                {"role": "user", "content": user_input},
                {"role": "assistant", "content": assistant_reply},
            ],
            user_id=user_id,
        )
        persisted = True
 
    return assistant_reply, retrieved, persisted

Mem0's add() is not a simple insert — it compares against existing memories and automatically decides ADD/UPDATE/DELETE/NOOP. If a memory "the user prefers Python" already exists and you write it again, it becomes an update rather than a duplicate. Layering an upstream persistence-value judgment on top means there is a double filter: one before entries enter the vector store, and one that cleans up what is already inside.

Full Loop Flow

Pitfalls That Come Up Frequently in Practice

Hallucinated Memory

The problem of an agent trusting incorrectly stored memories as fact. If "the user likes A" is recorded by mistake, all subsequent responses become biased in that direction. Without a separate pipeline to evaluate memory freshness and detect conflicts, the problem compounds as memories accumulate.

Not Designing for Forgetting

Without a policy for when to delete or update information, the store becomes polluted. Mem0's ADD/UPDATE/DELETE/NOOP semantics are a representative solution to this problem. If you are using raw vector insertion, you must explicitly design a periodic cleanup policy.

Long-Term Memory Design That Ignores Token Costs

Appending long-term memory retrieval results directly to the prompt adds more input tokens than you might expect. arXiv 2603.13017 proposes structuring and distilling personalized agent memory as compressed factual units rather than raw text, and reports a substantial reduction in prompt tokens under personalization benchmark conditions. Before storing full raw text in the vector store, consider first extracting and storing only the key facts.

Vector Search Alone Cannot Capture Temporal Order

Finding "what was said three conversations ago" is difficult with vector similarity search. This is why vector + graph DB combinations come up frequently in production discussions as of 2026 when temporal order or logical relationships matter.

Design Observability In From the Start

The real operational burden of this architecture is how hard it is to trace how the memory layer influences agent behavior. Without logging which memories were retrieved and how they were reflected in the response, debugging becomes extremely painful.

This is why agent_turn was designed to return retrieved and persisted. Re-invoking retrieval inside the logging layer doubles API costs, and if the index state changes between the two calls, the log diverges from the actual inference input. You must pass through the exact same results that were actually used.

import logging
 
def agent_turn_with_observability(
    user_input: str,
    short_term: ShortTermBuffer,
    user_id: str,
) -> str:
    reply, retrieved, persisted = agent_turn(user_input, short_term, user_id)
 
    logging.info(
        "memory_retrieval",
        extra={
            "user_id": user_id,
            "query": user_input,
            "retrieved": [r["memory"] for r in retrieved],
            "scores": [r.get("score") for r in retrieved],
        },
    )
    logging.info(
        "memory_write",
        extra={"user_id": user_id, "persisted": persisted},
    )
    return reply

You Can Defer the Vector DB Selection Conversation

At first I spent a long time deliberating over which vector DB to choose. But in practice, embedding model quality, chunking strategy, and whether re-ranking is applied matter far more than the choice of vector DB. With the same data, if the embedding model does not fit the domain, retrieval quality will be poor regardless of which DB you use; and if chunking units are too large, retrieval results will always be coarse.

After that, here is a comparison to reference when selecting a DB:

Start prototyping with Chroma, and if response latency matters in production, it is also worth exploring approaches like the in-process vector index pattern (arXiv 2607.05690), which searches within the agent process itself without external network calls. But to re-emphasize: get embeddings, chunking, and re-ranking right first, then choose the DB — that order minimizes trial and error.

Where You Can Start Tomorrow

You do not need to build everything described in this article from scratch. If you have an agent running today, I recommend attaching these pieces one at a time in the following order:

1. Formalize the loop sequence: In your current code, clarify function boundaries so that "short-term query → long-term search → inference → layer-specific writes" is visible in order. This alone makes all future improvements much easier.
2. Insert a persistence decision function: Place a thin filter like should_persist right before each vector write. The rate at which noise accumulates in the store drops noticeably.
3. Observability logging: Log retrieved and persisted every turn. The time spent tracing the cause of a wrong answer decreases significantly.
4. Decide on a forgetting policy: Document at least one of TTL, freshness-based re-evaluation, or conflict-update rules. Without a policy, the store silently becomes polluted.
5. Revisit the need for three layers: After the above four are stable, check whether the conditions actually calling for episodic or procedural layers have been observed.

Cramming everything into the context window only works when the agent is simple. Separating into two layers adds a bit of structure, but past 30 turns, having this structure is what creates room to solve the next problem.

References

• Building Memory for AI Agents: Context Windows, Vector Search, PostgreSQL, and Long-Term Recall — Medium
• Memory vs Context Window for LLM and AI Agents — Mem0
• State of AI Agent Memory 2026: Benchmarks & Trends Report — Mem0
• Memory in the Loop: In-Process Retrieval as Extended Working Memory for Language Agents — arXiv 2607.05690
• Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memory — arXiv 2512.07287
• Episodic-Semantic Memory Architecture for Long-Horizon Scientific Agents — arXiv 2605.17725
• Structured Distillation for Personalized Agent Memory — arXiv 2603.13017
• Context Engineering - LLM Memory and Retrieval for AI Agents — Weaviate
• Building Long-Term Memory in AI Agents with LangGraph and Mem0 — DigitalOcean
• Long-Term Memory Architectures for AI Agents — Redis
• Agent Memory Architectures: Vector vs Graph vs Episodic — DigitalApplied
• Top 5 Vector Databases 2026 — Deepak Gupta