Block: Modern AI — Topic: Long-term Memory. Target audience: interested adults. Style: factual, comprehensible.
Time allocation:
In research on artificial intelligence, the term "memory" is usually used by analogy to human cognition, but the technical realizations differ in some respects from biological systems. Two common categories are short-term (or working/context) memory and long-term memory. Short-term memory in technical systems practically refers mainly to the information that a model has immediately available for an ongoing computation — typically the context window of a Transformer model or internal states of a recurrent network. Long-term memory, by contrast, means persistently stored information that is preserved across individual sessions, user interactions, or system restarts.
To illustrate: imagine a presenter who places notes on a clipboard during a session (short-term) and transfers important findings into a notebook or archive (long-term). The clipboard corresponds to the context window, the notebook corresponds to a persistent database or parameterized model weights.
Main distinguishing features can be summarized: persistence (long-term remains across sessions), capacity and scalability (long-term storage is externally scaled, short-term is often limited by model design), access methods (direct read/write vs. embedding into input context), updatability (how easily something can be added or forgotten) and consistency/reliability of the stored information.
Technically, three general implementations of "memory" shape current practice: (1) information encoded in the model's own weights (parametric memory), (2) explicit external persistent storage such as databases, document stores or vector indices (external/long-term memory), and (3) temporary context stores (non-persistent tokens / context window). The Transformer architecture made the context window efficient, but at the same time created new challenges for long-term remembering because it is not persistent by itself (Vaswani et al., 2017).
A short example: a chat system that remembers a user's earlier preferences can either load these preferences into the context window for each request (short-term) or store them in a persistent profile in a database and retrieve them as needed (long-term). The choice affects performance, cost, privacy and error susceptibility.
Important is the difference between "knowing" and "remembering": models "know" information in their weights when it is relevant for generation; they "remember" most reliably, however, when a clean external representation exists that can be retrieved on demand (see retrieval approaches, Lewis et al., 2020).
Terms that we introduce and distinguish here are: parametric vs. non‑parametric memory, episodic vs. semantic memory (by analogy to cognitive psychology), continual or lifelong learning, indexing/embeddings, retrieval systems and mechanisms for selective storage and forgetting.
Parametric memory denotes information encoded in the model parameters (weights). It is robust against simple deletion but hard to change in a targeted way; fine‑tuning or targeted weight adjustments can add or correct knowledge, but are costly and may disturb other stored capabilities. Non‑parametric, external memory uses data structures outside the model: document databases, knowledge graphs, vector indices for embeddings. Such stores allow targeted updates, deletion and access control and are suitable for long-term persistence (Lewis et al., 2020; Johnson et al., 2017).
From a cognitive perspective, one often distinguishes episodic memory (concrete events, sessions, interactions) from semantic memory (general knowledge). In technical systems these functions are often implemented separately: episodic databases store logs and personalized events; semantic knowledge is maintained in curated knowledge bases or in the parameters (Baddeley & Hitch; Atkinson & Shiffrin — classical concepts from psychology on working and long-term storage).
Architectures with explicit long-term storage have historically taken various forms. Memory Networks and later Differentiable Neural Computers integrated external, addressable memory that the model could access in a differentiable way (Weston et al., 2014; Graves et al., 2016). Modern systems often combine a strongly parameterized model (Transformer) with external retrieval and vector indices for fast semantic access (Vaswani et al., 2017; Lewis et al., 2020; Johnson et al., 2017).
Key components of current solutions are embeddings (dense vector representations of text or other modalities), vector indices (for nearest‑neighbor search) and retrieval mechanisms that feed relevant entries into the model's context. FAISS is a widespread tool for such vector indices and demonstrates in practice how large quantities of representations can be searched efficiently (Johnson, Douze & Jégou, 2017).
To control remembering there are several technical levers: selective indexing (only certain data are stored at all), metadata‑based retention (timestamps, categories), role- and access management, audit logs and algorithmic deletion mechanisms. In addition there are methods from continual learning that aim to incorporate new information without overwriting existing knowledge (the catastrophic forgetting problem). The main classes of solution strategies are replay mechanisms (retraining on old examples), regularization (preserving important weights) and modular architectures (extend rather than overwrite). These strategies are discussed in reviews on continual learning (Parisi et al., 2019).
On the user-control level, mechanisms such as explicit opt‑in/opt‑out for persistence, togglable personalization and the ability to request deletion are central. Legal requirements (see below) make clear processes for data deletion and logging necessary.
Technical limits and uncertainties: there is no universally accepted optimal trade-off between latency, cost, persistence and privacy. Research into efficient and privacy-friendly long-term memories is active; many questions about robustness against manipulation, attribution of errors and reliable deletion remain open or are only partially standardized (see, e.g., reports and technical guidelines from major providers; the GPT‑4 Technical Report as an example of systemic limitations and error types).
Concrete applications that benefit from long-term memory include personalized assistant systems (remembering preferences over long periods), knowledge management in companies (persistent, searchable knowledge bases), medical decision support (longitudinal patient data with consent) and research databases (cumulative collection of results). In all cases the combination of good index design, clear data governance rules and transparent access controls is decisive.
Limitations: persistent stores can become outdated, become inconsistent, or preserve false information for a long time. Models can produce errors ("hallucinations") when retrieving from external sources if retrieval relevance and fusion logic are insufficiently designed; reliable verification of external content therefore remains necessary (cf. retrieval-and-generation approaches, Lewis et al., 2020). In addition, large persistent collections increase attack surfaces: data leaks in vector indices or metadata can pose security and privacy risks (see legal classification below).
Data protection issues — central aspects and obligations: in the EU the General Data Protection Regulation (Regulation (EU) 2016/679, "GDPR") regulates the processing of personal data; core principles are lawfulness, purpose limitation, data minimization, storage limitation and the right to erasure ("right to be forgotten"). For AI systems with long-term storage this means: personal data may only be stored with a legal basis (e.g. consent, contract performance); users must be informed; deletion and correction requests must be made technically feasible (Regulation (EU) 2016/679). National data protection authorities and supervisory bodies publish additional guidance for implementation in AI applications (example: ICO Guidance on AI and Data Protection for practical advice).
Technical means to meet data protection requirements include pseudonymization, encryption of data at rest and in transit, fine-grained access control, auditing and procedures for secure deletion from vector indices. Research and practice also recommend privacy-by-design and privacy-enhancing technologies such as differential privacy or federated learning as complementary safeguards; their use often carries costs in model performance and complexity (this is the subject of current research and not fully solved).
Small thought exercises for deepening (briefly reflect on each): 1) Assume an assistant system stores a user preference permanently. How would you combine technical and organizational measures to ensure both fast availability and legally compliant deletion? 2) Consider which information absolutely must be preserved long-term if the goal is medical course recommendations; which information must not be stored permanently? 3) A company wants to store all user interactions for product improvement. Name three reasons why that can be legally and ethically problematic, and three technical precautions that would reduce the risk.
In all considerations transparency toward users is central: which data are stored, for how long, for what purpose, and how deletion can be requested. Cooperation with data protection officers, security teams and users is part of standard good practice.