Methods & Evidence

Interpret Evidence

Distinguish source data, modelled answers, respondent agreement and real-world validation.

此页译文尚未提供,当前显示英文原文。

A useful finding makes its evidence visible. In synthetic research, that means separating what was observed from what was inferred and what the current study simulated.

Identify the source

EvidenceWhat it supportsWhat it does not establish by itself
Demographic statisticsPopulation composition within the source's scopeAttitudes about a new product
Real survey answersResponses to the original questions, in that sample and periodThe answer to a materially different question today
Customer recordsRecorded characteristics and eventsUnrecorded motives or preferences
Interview or feedback textThe language and experiences in that sourceHow common those views are in the whole market
Modelled answersA simulated response to the supplied question and contextAn observed action or guaranteed future outcome

An audience can combine several evidence types. Inspect its definition and sources, then inspect the material and question in your run.

Distinguish recorded source facts, simulated responses, interpretations, and observed outcomes.
Explanatory diagram · click to enlarge

Agreement is not accuracy

Respondents agreeing with each other is a pattern within the simulation. It does not independently verify the answer. Increasing the panel size may improve coverage of represented subgroups but does not correct a shared modelling error.

Likewise, a high score on a benchmark is measured on that benchmark's datasets and tasks. It is not a probability that any new finding is correct.

Read the denominator

Distinguish people, answers, coded mentions and selected subgroups. One person may mention several themes; percentages of mentions need not equal percentages of respondents. Filters change the group being described.

Do not treat missing, suppressed or unavailable values as zero. When comparing runs, check question wording, source population, timing and stimulus.

Make the claim match the evidence

Prefer “this modelled audience raised these concerns under this stimulus” over “customers will reject the product”. For a precise commercial claim, test against observed outcomes with real validation.

The public benchmark describes measured performance. About the benchmark explains how to read different metrics without combining them into one universal accuracy claim.

Interpret Evidence | iMario