Every agency has shipped a creative that the team loved and the audience ignored. Real A/B tests on the ad platforms answer the question properly, but they cost money and days, and you can only run a few at once. Simulated audience testing is a way to narrow down the options before that point.
This guide explains what it is, how to use it well, and — just as important — what it cannot do.
What a simulated audience is
A simulated audience is a panel of fictional people, generated to match a description of who a brand sells to: age range, country, language, interests, attitudes. A language model plays each persona, looks at the real creative — the image, the carousel, the video frames, the copy — and answers questions about it in a fixed format.
The result is a simulated reaction: how a panel built from your description responds to what you show it. It is not a forecast of click-through rate, conversions or sales. Treat any tool that presents it as a prediction with care.
What it is good for
- Comparing options. Which of three headlines is clearer? Which image draws attention to the product rather than the background? Comparisons between variants shown to the same panel are where this method is most useful.
- Catching obvious problems early. Copy that is confusing, a claim that sounds too good to be true, a visual that does not fit the brand.
- Getting written feedback. The reasons behind a score are often more useful than the score itself.
- Speed. Minutes instead of days, before any media budget is spent.
What it is not good for
- Predicting numbers. A higher simulated score does not mean a specific CTR or ROAS.
- Replacing real customers. Personas are built from a description. If the description is wrong, the panel is wrong.
- Niche or very new audiences. The less written material exists about a group of people, the less a language model knows about how they think.
- Final decisions on big budgets. Use it to choose what to test for real, not to skip the real test.
How to run a fair test
Build the panel with skeptics in it
A panel that likes everything tells you nothing. Make sure a real share of the personas are skeptical by design: people who distrust advertising, are price-sensitive or are simply busy. If every variant scores well, your panel is too friendly.
Show every variant to the same people
Comparing variant A shown to one panel with variant B shown to another mixes two differences at once. Pair the comparison: the same personas see both.
Fix the panel and record it
If the panel changes between tests, scores are not comparable. Keep the panel stable (the same seed gives the same people), and record which version of the panel, the scoring weights and the model a test used.
Look at the spread, not only the average
An average of 72 against 70 may be noise. A proper comparison looks at the uncertainty: if the confidence interval for the difference includes zero, call it a tie. When you compare more than two variants, correct for the extra comparisons, or chance alone will hand you a "winner".
Read the comments
Scores tell you which variant won; written feedback tells you why. Often the most useful outcome is a sentence like "I can't tell what the product is" from several personas at once.
Check it against reality
The honest way to use simulated testing is to measure how well it works for each brand. When a tested creative later runs as a real ad, compare its simulated score with its real click-through rate. Over a number of creatives, you will see whether the two move together for that brand — and how much weight to give the simulation next time. If they do not agree, trust the real results.
How Reachio handles this
Reachio's Test Lab follows the approach above, and labels every result "simulated audience reaction", never a prediction:
- You describe the audience and Reachio builds a panel of 10–50 personas. Their traits are sampled in code with a fixed seed, so the same audience gives the same panel; at least one in five is a skeptic by design. Panels are versioned.
- Test kinds: a single score, a survey, an A/B test where the same personas see both subjects, and experiments with 2–6 subjects. A winner is named only when the confidence interval excludes zero, with a correction for more than two subjects.
- Each persona sees the real creative and answers in a strict format. Six dimensions — attention, relevance, emotional resonance, persuasion, brand fit and clarity — are combined in code, with weights per platform, into a 0–100 score and a band, always next to the written feedback.
- Improve makes new variants aimed at the weakest dimensions and re-tests them on the same panel.
- The Lab board compares Lab scores with the real click-through rate of creatives that ran as ads, so you can see how far to trust the Lab for each brand.
- The cost is estimated before a test runs and comes from the plan’s AI allowance.
More in Test Lab: a simulated audience.