Anamnesis and the Next Stage of Virtual Persona Population Simulation

🎥 The YouTube version is also available:

What Is Emergent Intelligence? | The Future of AI Beyond Models

Books: Al Coordination Engineering A Practical Guide: Designing Coordination, Autonomy, and Trust in the Multi-Agent Era

Introduction

The effort to model human and social behavior in advance in order to examine the effects of policies and interventions is not new.

Agent-based modeling tools such as NetLogo, for example, have long assigned relatively simple behavioral rules to individual vehicles, pedestrians, residents, patients, or firms. By observing their interactions, researchers and practitioners have explored population-level phenomena such as traffic congestion, the spread of infectious disease, evacuation behavior, resource allocation, and market dynamics.

The strength of this approach is that it does not assume the behavior of a single “average person.” Instead, it makes it possible to explore how unexpected outcomes can emerge from the interactions of many actors. At the same time, conventional models have found it difficult to represent, in fine detail, context-dependent aspects of human judgment: personal experience, values, how people interpret language, or whether they feel convinced or concerned by an explanation.

LLM-based virtual persona populations are an attempt to address this limitation. Rather than focusing primarily on physical or behavioral state changes, such as traffic volume or infection probability, they seek to work at the level of meaning and narrative: how people interpret a survey question, which part of an institutional explanation makes them uneasy, and why they express support or opposition.

Of course, LLMs do not reproduce real people as they are. But just as traditional agent-based simulations have served as experimental environments for thinking about social phenomena, virtual persona populations can become a new kind of experimental environment for examining surveys, products, policies, and messages before they are delivered to people.

One notable development in this area is Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation, accepted to the EMNLP 2026 Demo track. Anamnesis is an open-source platform that enables researchers and non-technical users to prototype and stress-test survey instruments with virtual populations without writing code. According to its arXiv paper, it supports open-ended responses, probabilistic demographic resampling, and multimodal surveys involving images and audio.

The important point is not to understand this merely as a tool for asking AI to complete surveys in place of people. More fundamentally, it should be positioned as a simulation environment for discovering diverse response hypotheses and potential weaknesses in the design of surveys, products, and policies before they are presented to people.

Why Virtual Persona Populations Matter Now

Conventional user research remains indispensable. Yet recruiting participants, fielding surveys, and analyzing results require time and cost. At early stages, this can mean that a survey or initiative is launched before there has been enough opportunity to test ambiguity in wording, bias in answer choices, misunderstandings among particular segments, or negative reactions to a message.

LLM-based virtual persona populations make it possible to investigate questions such as the following through rapid iterations:

• Will people with different ages, occupations, locations, and values interpret this question in the same way?
• Which answer choices may be steering the response distribution unnaturally?
• Which groups may feel anxiety or friction in response to an explanation of a policy or service?
• Do open-ended responses reveal concerns or vocabulary that quantitative choices fail to capture?
• Which hypotheses should be prioritized in the next study with human participants?

The value, therefore, lies less in “replacing answers cheaply” than in accelerating learning before field research begins.

Major Approaches to Building Virtual Persona Populations

Even when they are all called “virtual users,” their properties vary greatly according to what conditions are used to generate LLM responses. The principal approaches can be organized as follows.

Classical user modeling has centered on statistically estimating preferences and behavior from observed data. LLMs are new in that they can generate contextually rich open-ended responses and dialogue with relatively low setup cost. At the same time, as a review of LLM-based conversational simulation notes, diversity in human behavior, bias, harmful content, and the validation of fidelity remain central challenges. Ni et al., 2026

Where Anamnesis Fits: A Population with Context, Not Just Attributes

The Anthology and Alterity frameworks underlying Anamnesis condition LLM responses not with a short list of attributes—such as “a 45-year-old office worker living in a city”—but with structured backstories.

The purpose is not to derive an answer mechanically from each attribute. It is to bring real-world heterogeneity into the simulation: people with the same demographic attributes may still make different judgments because of their experiences, family situations, work constraints, values, and past events.

The Anthology research released a collection of approximately 10,000 backstories and demonstrated a method for sampling from them to match a target population composition. In experiments approximating Pew Research Center’s American Trends Panel, it reported improved approximation of both response distributions and response consistency compared with standard persona conditioning. For underrepresented groups, it reported improvements of up to 18% in Wasserstein Distance and 27% in consistency. Moon et al., 2024

Anamnesis turns this approach into a practical survey environment on the web. Its paper evaluates the system against Pew surveys on political typology and biomedical issues, as well as human preferences in the New Yorker Caption Contest, and reports that it more closely approximates real-world opinion distributions than standard persona-prompting baselines. Yu et al., 2026

Its value is not simply that its answers appear plausible.

1. Reproducibility: It can retain the backstories, model, prompts, and sampling conditions used to generate a result.
2. Population design: It makes it possible to specify a desired population composition and resample accordingly.
3. Rich exploration: It supports observation of open-ended responses as well as reactions to images and audio, not only multiple-choice outcomes.
4. Accessibility: Researchers who do not write code can begin prototyping and comparing survey instruments.

What Should Be Compared in a Benchmark?

For virtual persona populations, the simple question “Does the LLM seem human-like?” is insufficient. The object of evaluation should be separated according to use case.

Most importantly, population-level distributional fit must not be conflated with individual-level predictive power. A model can approximate aggregate answer proportions while failing to reproduce the judgment of individual people. The reverse can also be true.

A 2026 cross-domain benchmark examined this issue rigorously using the General Social Survey and the World Values Survey. Under the demographic-prompting protocols it tested, no LLM outperformed strong non-LLM baselines at the individual level, and the models tended to overestimate how strongly demographics explain attitudes. Its decision-impact analysis reported that models could exaggerate segment differences by a factor of two to four and lead to incorrect targeting choices. Chen, Zhu, and Zheng, 2026

This result does not invalidate Anamnesis. Rather, it shows why richer context than short attribute prompts, distributional calibration, comparison with reference data, and a verifiable experimental design are needed.

A Practical Design for Comparative Benchmarking

When comparing Anamnesis with other approaches, the following sequence is useful.

1. Fix the intended use case.
   Discovering ambiguity in a survey instrument, initial screening of an intervention, demand forecasting, estimating policy opinion, and testing dialogue cannot be evaluated with one identical accuracy metric.
2. Prepare held-out human reference data.
   Use existing public surveys or a small original survey with human participants for evaluation, separately from simulation design and calibration.
3. Include a minimum set of baselines.
   Compare an unconditioned LLM, attribute prompting, backstory conditioning, and a statistical baseline calibrated on real data using the same questions and sample design.
4. Inspect both the overall population and subgroups.
   An overall average can appear accurate while errors are large for minorities, regions, value groups, or people with different levels of experience.
5. Repeat across random seeds, models, and question phrasings.
   Do not treat a single result as definitive “public opinion.” Examine how stable the conclusion remains.
6. Design the connection to human research.
   Treat simulation results not as a final decision, but as a way to prioritize hypotheses for validation with people.

The Benefits of Using Virtual Persona Populations

When used with appropriate purposes and boundaries, virtual persona populations offer clear benefits.

1. They accelerate hypothesis exploration

Before conducting research with people, teams can compare multiple versions of question wording, explanations, answer choices, and user journeys. They can eliminate obviously unsuitable or hard-to-understand options before consuming participants’ time.

2. They bring overlooked perspectives into the design process

Population composition can be resampled for a particular purpose, making it possible to explore perspectives that may be buried in an overall average. This does not grant authority to “speak for” a minority’s real views. It is a means of finding blind spots and prioritizing validation with people.

3. They support early exploration of open-ended and multimodal responses

They can surface possible reasons, objections, language, and reactions to images or audio that cannot be seen from choice proportions alone. Anamnesis’s support for multimodal surveys may be particularly useful for early reviews of service explanations, advertising, educational materials, and user interfaces.

4. They make experimental traces easier to retain

If the persona population, conditions, questions, model, execution time, and results are recorded, teams can trace the basis on which they selected the next human study or design change. This also connects to Decision Trace when AI is incorporated into decision-making.

5. They can become an open research foundation

When organizations rely only on closed services, it is difficult to examine how personas are generated, how distributions are calibrated, and how a model change affects results. The open-source nature of Anamnesis is important because it enables reproduction, comparison, criticism, and improvement.

But Virtual Persona Populations Are Not Replacements for People

The most important caution is not to confuse calling a virtual persona population a “synthetic population” with claiming that it represents real people.

LLMs reconstruct social biases, existing discourse, and dominant values embedded in their training data. If attributes are emphasized too strongly, they can produce answers that are more obedient to demographic labels than real people are, thereby exaggerating differences between groups. Results may also change with the wording of a question or an update to the underlying model.

The following boundaries should therefore be made explicit:

• Use them actively for pre-fieldwork prototyping, survey stress-testing, and hypothesis generation.
• Do not use them alone as evidence for decisions with major consequences for individuals or groups, including hiring, credit, insurance, healthcare, or public benefits.
• When publishing conclusions, disclose the virtual population’s construction, the model, prompts, evaluation method, and uncertainty.
• Combine them with human research, field observation, expert review, and ethical and legal judgment in final decisions.

Conclusion: Not a Technology for Replacing People, but for Enabling Better Validation

What Anamnesis demonstrates is that the value of virtual persona populations does not arise from simply giving an LLM demographic labels.

It comes from combining population design that attends to representativeness, backstories that give individuals context, benchmarking against real data, reproducible experiments, and clear boundaries that connect simulation to human research.

Only then can virtual persona populations evolve from a mechanism that generates plausible answers into a research and design foundation for improving the quality of surveys and decisions.

The acceptance of Anamnesis marks an important step: it suggests that this field is moving not only toward competition among closed products, but toward an open and verifiable research foundation.

References

• Song-Ze Yu et al., Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation, 2026.
• S. Moon et al., Virtual Personas for Language Models via an Anthology of Backstories, EMNLP 2024.
• Zihan Chen, Di Zhu, and Lei Nico Zheng, When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses, 2026. Preprint under review.
• Bo Ni et al., A Survey on LLM-based Conversational User Simulation, EACL 2026.
• Zhengyu Hu et al., Population-Aligned Persona Generation for LLM-based Social Simulation, 2025.

Chinoba

Chinoba

Intelligence as Relationship

Research Platform

founded by

Masao Watanabe

AI Systems Architecture
Decision Trace
Human–AI Coordination
Algorithmic Governance