AEO Prompt Testing: Build a Repeatable Dataset Before Claiming Visibility

Most AEO reporting starts at the wrong end. Someone asks an assistant a question, sees the brand named, screenshots it, and a visibility claim enters a deck. Run the same prompt an hour later on a different account and the answer often differs.

The fix is not a better tool. It is a dataset that records enough about each run to be comparable to the next one.

Start with a prompt library, not a keyword list

Buyers ask assistants in sentences, and the sentence changes with the stage. Build a library that covers the journey rather than a list of head terms.

Keep the library small enough to run every cycle without shortcuts. A library you actually rerun beats a larger one you sample from.

Reproducibility fields for every row

Each run is one row. Without these fields a row is an anecdote, not data.

• Assistant and model version, exactly as reported.
• Date and time of the run.
• Region or locale setting used.
• Account state: signed in or out, memory or personalization on or off, any custom instructions.
• The prompt text, verbatim and unedited.
• The full response text, saved rather than summarized.
• Run number, so repeats of the same prompt are grouped.
• The tester's name.

Score dimensions separately

Collapsing everything into one visibility score is where AEO reporting loses its meaning. Keep four columns apart.

1. Mention. Was the brand named in the answer text, and in what position within it.
2. Citation. Was a page attributed as a source, and which URL.
3. Accuracy. Is what the assistant said about you correct, including services, location and claims.
4. Competitive context. Which other providers appeared in the same answer.

Accuracy is the dimension teams skip and the one with the clearest business consequence. A confident, wrong description of your service is worse than no mention.

A six-step test protocol

1. Freeze the prompt library and the run schedule.
2. Set the account state deliberately and record it, rather than testing from whatever session happens to be open.
3. Run each prompt the agreed number of times per cycle.
4. Save the full response before scoring anything.
5. Score the four dimensions from the saved text, not from memory.
6. Review the variation across runs before writing any conclusion about change.

Bridging to Search Console

Google says its generative AI performance report completed worldwide rollout on August 31, 2026. It covers AI Overviews and AI Mode link impressions broken out by page, country, date and device. It does not include a query dimension, so it cannot tell you which prompt produced which impression.

That shapes what the bridge can do. Use the report to see which pages accumulate AI-surface impressions over time, and use the prompt dataset to see how you are described and cited. Together they support statements about direction on specific pages. Neither supports a claim about share of recommendations across a market.

Honest reporting language

Why we wrote this one

Between August 11 and September 7, 2026, Google Search Console recorded 189 impressions for "aeo services" at an average position of 15.56 with zero clicks for our own site. One conversational Vancouver query recorded 20 impressions at an average position of 1.00 in the same window. That sample is far too small to support a durable ranking claim, and we are reporting it as an observation only. These are ThinkProfits visibility observations, not market search volume.

Our AEO services page describes the work behind these tests, reporting dashboards shows how the dataset reaches a monthly pack, an SEO audit covers the crawlable foundations these answers depend on, digital marketing strategy is where the prompt library gets prioritized, and our SEO content analyzer helps keep the answer pages themselves tight.