# iMario vs Base LLMs: Solving Mode Collapse and Identity Drift in Synthetic Individuals

> A data-driven benchmark comparing base LLMs (GPT, Claude, Doubao, DeepSeek) against iMario. See how we solve mode collapse and maintain 96% identity consistency when simulating tens of thousands of synthetic users.

Source: https://imario.ai/blog/imario-vs-llms
Published: 2026-03-23
Author: iMario Team

![](https://cdn.imario.ai/blog/imario-vs-raw-llms.png)

If you try to use ChatGPT, Claude, or Doubao to run a qualitative user interview, the illusion usually shatters pretty quickly. The first few questions might sound convincing. But if you ask the model to simulate tens of thousands of different people, or try to hold a deep 30-minute conversation, you will find: the personas start sounding identical, they blindly agree with everything you say, and they eventually forget their own backstory. 

This is known as **Persona Collapse** or the **Artificial Hivemind Effect**. While LLMs are incredible general-purpose reasoning engines, their raw output is fundamentally bad at maintaining distinct, diverse human identities over time. 

In recent evaluations, we compared standard LLMs against iMario's dedicated synthetic users platform to see how they perform when generating and interviewing [synthetic individuals](https://imario.ai/synthetic-individuals) at scale. Here is what the data shows.

## The Human Layer Benchmark

Based on recent academic frameworks evaluating "Pluralistic Alignment" and "Persona Gyms," we measure synthetic users across two critical dimensions:

1. **Scale & Representation:** Can the system generate 10,000+ distinct personas that accurately reflect real-world demographic and psychological diversity?

2. **Long-Term Consistency:** Can each synthetic individual maintain its specific identity, beliefs, and behaviors across a long, multi-turn interview without breaking character?

### 1. Scale & Representation (Overcoming the Hivemind)

When you ask a base LLM to generate 10,000+ different user profiles, it suffers from severe mode collapse. Studies show that base models tend to over-represent majority viewpoints and generate stereotypical attributes. They default to a "helpful assistant" tone, regardless of the prompt.

iMario, on the other hand, is [built specifically to construct populations](https://imario.ai/blog/beyond-average-how-imario-is-building-a-million-synthetic-individuals). When you need 10,000 or even more unique "Gen Z gamers from the Midwest," iMario generates 10,000 statistically accurate, distinct individuals whose socioeconomic backgrounds, quirks, and nuanced opinions perfectly match real-world sociological distributions.

| Metric (Scale: N=10,000) | GPT-5.3 | Claude 4.6 Opus | DeepSeek V3 | Doubao 2.0 | iMario |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **Linguistic Variance** | Low | Medium | Low | Low | **High** |
| **Demographic Parity** | Skewed | Skewed | Highly Skewed | Skewed | **Real-world Parity** |
| **Mode Collapse** | ~55% | ~50% | ~65% | ~60% | **< 5%** |
| **Sycophancy**| High | Medium-High | Medium-High | High | **Low (Maintains opinion)** |

###### ***Evaluation Methodology**: Tested generating 10,000 synthetic individuals based on US Census criteria. Variance and homogenization computed using embeddings distance metrics and qualitative review.*

Base models are trained to be polite and helpful. iMario personas are designed to act like real humans—which means they will disagree, express frustration, or hold unpopular opinions if it aligns with their profile.

### 2. Long-Term Consistency (The Interview Endurance Test)

In a 60-minute qualitative interview, context is everything. Standard LLMs rely entirely on their context window. As the conversation grows, their attention dilutes. A major academic study on *Identity Drift in LLM Agents* revealed a startling fact: **larger, more advanced models actually experience greater identity drift than smaller models over time.** Around the 10th or 15th question, a persona that started as an "impatient 50-year-old executive" will slowly revert back to a standard AI chatbot tone.

iMario is designed to bypass this limitation. Even after extended back-and-forth interactions, follow-up questions, and topic shifts, the synthetic individual remains flawlessly in character.

### Identity Consistency

Tracking persona attribute adherence, tone retention, and memory recall over a 40-turn interview.

- iMario System 96%
- Claude 4.6 Opus 62%
- GPT-5.3 58%
- Doubao V2.0 52%
- DeepSeek V3 45%

###### ***Evaluation Methodology:** Tested via an internal multi-turn automated validation pipeline measuring context retention, trait persistence, and hallucination rates across 10,000 distinct synthetic individuals. 40 consecutive interview interactions per individual. Baseline metrics adapted from academic identity drift frameworks (e.g., PersonaGym, 2024).*

## iMario Delivers Production-Ready Research Capabilities

By utilizing dynamic persona fabric techniques and continuous state management, iMario drops mode collapse to under 5% and pushes long-term identity consistency to 96%. When it comes to conducting rigorous qualitative interviews with thousands of synthetic individuals simultaneously, iMario provides the [specialized infrastructure](https://imario.ai/blog/harness-engineering-is-the-missing-layer-between-models-and-products) that off-the-shelf LLMs simply lack. You can inspect how we measure this on our [public benchmark](https://imario.ai/synthetic-audience-benchmark).

## Frequently Asked Questions

### Why not just use ChatGPT or Claude directly for user research?

Raw LLMs work for the first few questions of a single interview. Past that, three failure modes show up. Sycophancy: the model agrees with whatever the interviewer suggests, which destroys the value of probing. Identity drift: somewhere around the 10th to 15th question the persona reverts toward generic AI assistant tone, regardless of how detailed the original prompt was. Mode collapse: if you ask the same model to generate 1,000 distinct personas, 50 to 65 percent of them sound functionally identical, with the same vocabulary, the same hedges, and the same opinions. These are not prompt engineering bugs. They are properties of how the underlying model was trained to be helpful. iMario does not replace the LLM. It builds the constraints, memory, and identity scaffolding around it that raw chat interfaces lack.

### What is persona collapse and how is it different from mode collapse?

Both are diversity failures, at different time scales. Mode collapse happens at generation time: when a model is asked to produce a large population, the outputs cluster around a small set of high-probability templates. The 1,000 Gen Z gamers from the Midwest you wanted end up sounding like 30 versions of the same person. Persona collapse, also called the Artificial Hivemind Effect, happens during a conversation: the persona starts distinct but gradually converges toward the model's default voice as the context window fills. Independent academic work on identity drift in LLM agents found a counterintuitive result: larger and more advanced base models often experience more drift than smaller models, because their fluency makes the regression toward the mean smoother and harder to detect. Both failures look the same from the outside but require different fixes.

### How does iMario actually maintain identity consistency across a long interview?

Three mechanisms working together. First, a structured persona representation with seed attributes, behavioral stances, and preference vectors that gets re-injected at each turn instead of relying on the model's attention to a long opening prompt. Second, a memory layer that keeps track of what the persona has already said in the interview, so earlier answers constrain later ones. Third, governance checks that catch tone reversion, off-profile claims, and I-am-an-AI defaults before they reach the user. In benchmark testing across 40-turn interviews, iMario maintains 96 percent identity consistency, compared to 62 percent for Claude Opus, 58 percent for GPT, 52 percent for Doubao, and 45 percent for DeepSeek when used as raw chat. The gap widens as interviews get longer. At 60 turns, raw LLMs are essentially their default assistant voice with a thin costume.

### Can I get diverse synthetic populations from a base LLM with better prompting?

Marginally, but the ceiling is low. Prompt engineering can push base-model mode collapse from 65 percent down to maybe 40 percent with techniques like explicit demographic conditioning, temperature variation, and seeded persona templates. iMario's measured mode collapse rate is under 5 percent. The gap is structural, not about prompt quality. Raw LLMs are trained on internet text dominated by a few mainstream voices, and their decoding distribution reflects that. Generating 10,000 statistically distinct personas requires sampling logic that lives outside the model: distribution-aware seed generation, diversity validation passes, demographic parity checks against real population data, and rejection sampling for outputs that cluster too tightly. None of that is exposed in a chat API. If your use case is fewer than 50 personas with manual review, prompting can get you there. Beyond that, dedicated infrastructure is the only reliable path.

### What does sycophancy look like in synthetic user interviews?

Sycophancy is when a synthetic user agrees with whatever the interviewer suggests, including contradictory suggestions in the same conversation. In an interview, it shows up as the user loving your concept then loving the opposite concept five minutes later, accepting every value-prop you propose without pushing back, and softening objections that should match a 50-year-old skeptical executive's profile into mild concerns from a friendly assistant. We measure it by feeding synthetic users a controlled adversarial probe set: claims that contradict their profile, leading questions with the wrong answer baked in, and pressure to abandon a stated preference. A persona built to be skeptical of cloud migration should reject a leading but-cloud-is-clearly-better prompt at least 80 percent of the time. Below that threshold, the model has reverted to assistant mode regardless of what the surface output sounds like.