How to build an AI visibility question set that measures something

A question set is the fixed list of buyer questions an AI visibility measurement asks. Each question is written in the words a buyer would use, contains no brand name, and stays unchanged between runs, so that a change in the answers can be read as a change in the market rather than a change in the wording.

This is lesson 2 of Learn GEO. Lesson 1, what answer engine optimization means, ended on the point this lesson starts from: the phrasing of a question changes which brands appear next to you. That makes the question set the single input with the most leverage over the result, and the one most measurement write-ups treat as an afterthought.

Our own rules are published below, with the thresholds taken from the code that enforces them rather than from a summary of it. You do not need our product to apply them; a spreadsheet and a column for the intent label will do. What the rules protect is described in the metric definitions on our methodology page, and the commercial pipeline that runs them is on AI visibility tracking.

The unit is a question, not a keyword

A keyword is a fragment that a ranking system matches. A question is a whole sentence that a person types, and an assistant answers the sentence it was given. Our generator is instructed to build each question from three parts: who the buyer is, what they need, and a constraint — budget, location, pain point or timeline.

Here is what that produced in a real run. On 2026-09-20 we ran one measurement through our own pipeline using the ChatGPT API with web search. The category was online whiteboard and visual collaboration tools, the market was US, and the brand watched was Miro. The pipeline generated three questions and asked each of them once:

Question generated	The constraint it carries
"Can you recommend online whiteboard tools that are best for remote teams in the US with a budget under $20 per user per month?"	A price ceiling
"Which visual collaboration platforms offer the best integration with popular project management software for small businesses?"	An existing stack, and company size
"What are the top online whiteboard vendors known for strong security features suitable for healthcare organizations?"	A regulated industry

Miro was named in all three answers, which is what we expected from a category leader and is mainly a check that the brand matching works. The rest of the list is where the constraints show up:

Brand named alongside Miro	Answers naming it, out of 3
Lucidspark	3
Mural	2
FigJam	2
Figma	2
Stormboard	2

One detail is worth more than the counts. In the answer to the budget question, the first vendor recommended was Mural, not Miro. Same category, same day, same engine, same brand present in every answer — and a different name at the top as soon as the constraint changed. A measurement built on one framing is a measurement of that framing.

This run asked each question once. It says nothing about how much a single question varies when you ask it again, and we are not going to imply otherwise. The repeat count is a separate dial, covered in our methodology page.

Never put the brand in the question

This is the rule everything else depends on, and it is usually stated as a warning rather than explained. The reason it holds is mechanical.

A retrieval-backed assistant turns your question into search queries before it writes anything. A question containing "Miro" produces queries containing "Miro", which retrieve Miro's own pages and pages written about Miro. The name is then sitting in the context the model writes from, so it appears in the answer. Your mention detector finds it and records a mention — of a name that you supplied.

Even with retrieval switched off, the effect survives. A question that names an entity asks the model to write about that entity, not to choose among entities. The answer names it because that is what the sentence requested.

The consequence is not that the number is inflated by some amount you could subtract. It is that the number stops varying. A brand-containing question returns a mention close to every time, whether the brand is the category leader or has been in the market for a week, so the metric loses the ability to distinguish the two. You can still watch the number, but it cannot move for any reason you care about.

Our pipeline enforces this by checking every question against the brand name and its confirmed aliases, lowercased, as a substring. A question that contains any of them is rejected unless it is explicitly labelled as a vendor check.

The vendor check is the deliberate exception. Those questions name the brand on purpose, to see what an engine says when asked about it directly, and they are excluded from the headline mention rate and reported on their own row. That separation is part of the metric definitions, not an optional courtesy: mixing one vendor-check answer into a rate computed over brand-free questions contaminates the denominator with a trial that could only go one way.

The intent mix: two numbers that are not the same number

Two proportions govern the intent of a set, and they are easy to run together because both are written as a percentage of the questions. They differ in the threshold and in which intent labels they count.

	What it is	Threshold	Which intents count
Generator instruction	What the model writing the questions is told to aim for	At least 80%	recommend and compare
Gate rejection	What the check applies to the finished set before anything is asked of an engine	Below 70% fails	recommend, compare and vendor_check

So the target is the higher of the two, and it is measured over a narrower group of labels. The gap is not academic. A set made entirely of vendor checks scores 100% against the gate, because vendor checks count toward that threshold, and 0% against the instruction, which counts only recommendations and comparisons. A set that is 70% recommend-and-compare clears the gate without having met the instruction. Report both numbers separately when you build your own set, because a set that repeatedly lands between them is telling you something about your generator.

Why buyer intent at all: an answer to "what is an online whiteboard" is a paragraph of explanation, and explanations do not list vendors. Questions that do not ask for a recommendation produce answers with nothing in them for a mention rate to count, so a set drifting toward definitions quietly shrinks its own denominator of usable answers.

The full gate, as implemented

Every generated set passes these checks before a single engine is queried. The thresholds below are read from the function that runs them.

Check	The set is rejected when
Buyer intent	Fewer than 70% of questions are labelled recommend, compare or vendor_check
Definitions	More than one question in the set is labelled definition
Brand leakage	Any question not labelled vendor_check contains a brand term as a case-insensitive substring. Terms of a single character are skipped
Length	Any question is shorter than 8 characters or longer than 200 characters
Near duplicates	Any pair of questions scores 0.85 or above on character-set Jaccard similarity
Empty	The set has no questions at all

A failing set is regenerated, with the first five failure reasons handed back to the generator as feedback, for up to three attempts in total. If the third attempt still fails, the offending questions are dropped individually and the rest are kept, rather than the whole set being thrown away. That last step re-checks each surviving question on its own, which means a set can come out of a bad generation smaller than requested — worth watching, because a set that quietly shrank has a smaller denominator than you think.

The duplicate check deserves a note on what it actually compares. It scores the set of characters a question uses, not the set of words. The three questions from the run above score 0.75, 0.76 and 0.75 against each other. The pair "Which online whiteboard tools are best for remote teams?" and "Which online whiteboard tool is best for a remote team?" scores 1.00, because both sentences draw on exactly the same alphabet. It catches paraphrases reliably. It is also coarse enough that genuinely different long questions about one category already sit close to the threshold, as the 0.75 figures show, so treat a near-duplicate rejection as a prompt to change the constraint rather than to reword.

Bad questions and why

The question	What is wrong	The fix
"Is Miro the best online whiteboard?"	Brand leakage, so the answer names the brand because the question did. It is also a yes/no question, which produces a verdict rather than a list	"Which online whiteboard tools work best for distributed product teams?"
"What is an online whiteboard?"	Definition intent. The answer explains a category and names no vendors, so it contributes an answer to the denominator and no possible mention	"Which online whiteboard tools do remote teams use to run workshops?"
"Tools?"	Six characters, so it fails the length check, and it carries no persona, job or constraint. Whatever comes back describes the model's default assumptions, not your market	"Which visual collaboration tools suit a 20-person agency running client workshops?"
"Which online whiteboard and visual collaboration platform is the best choice for a 340-person healthcare provider in the United States that needs SOC 2 Type II and HIPAA coverage, single sign-on, Jira and Salesforce integrations, and a rollout finished before the end of the quarter?"	283 characters, so it fails the length check. It is also over-constrained: only one vendor can satisfy all of it at once, so the answer is settled before the engine sees it	Split it. One question per constraint a buyer would actually lead with
"Which online whiteboard tools are best for remote teams?" next to "Which online whiteboard tool is best for a remote team?"	Scores 1.00 on the duplicate check. Two spellings of one question, using two of your answer budget	Keep one, and spend the other on a different constraint: security, pricing, a specific integration
"Which whiteboard tools are the best alternatives to Mural?"	Passes our gate, because the gate only checks your brand terms. It still measures the wrong thing: the answer is framed around a rival, and you appear, if at all, as an alternative to them	"Which online whiteboard tools are worth comparing for a security-conscious team?"

The last row is the one people get wrong after they have learned the first. Keeping your own name out of the question is not the whole rule. Any brand in the question sets the frame, and a set full of competitor names measures your competitors' position in the answer, with you as a footnote to it.

How many questions

Set size is not a matter of taste, because it multiplies straight into the denominator of every rate you report. The number of answers behind a mention rate is:

valid answers = questions × engines × repeats per question

Our free check is three questions on one engine, asked once each: three answers. At three trials, an observed 3 out of 3 carries a Wilson 95% interval of 43.8% to 100%, and 0 out of 3 runs from 0% to 56.2%. Those two rows are on our methodology page with the arithmetic, and they are the reason we call a free check a smoke test rather than a baseline. Paid monitoring asks each question on three engines, three times each, so ten questions produce 90 answers and an interval you can put a trend line under.

Two dials, two different jobs, and they do not substitute for each other:

• More questions buys coverage. It adds constraints and phrasings you were not asking about, which is what surfaces the competitors you did not know you had.
• More repeats of the same question buys precision. It narrows the interval on a question you already ask, which is what lets you tell this week's movement from noise.

A set that is wide and shallow will name many rivals and be unable to prove any change. A set that is narrow and deep will measure one framing with confidence. Decide which problem you have before you add anything.

We do not publish an optimal set size, because we do not have data that would support one. What we can tell you is the arithmetic above, our own configuration, and one consequence of changing the set: a question has to have been live for at least four weekly runs before it enters a trend, so questions added this month do not retroactively improve the history. Adding to a set is cheap; replacing one restarts its clock.

What you have at the end of this lesson

A list of questions, each one a sentence combining a persona, a job and a constraint, written in the language of the market you sell to. Next to each, the intent label you assigned it. Every question free of your brand name except the vendor checks you marked as such, none under 8 or over 200 characters, no two of them paraphrases, and at least 70% of them asking for a recommendation or a comparison.

That list is the input to every measurement that follows, and it is also the thing to hand to any vendor that wants to measure this for you. A supplier who cannot tell you which questions produced a number has given you a number about nothing. What we measure, and what we do not, is set out on AI visibility tracking.

Questions

How many questions should I track? Enough that the answer count behind your mention rate gives you an interval you can act on. Work backwards from the arithmetic: questions multiplied by engines multiplied by repeats is your denominator. Three questions asked once on one engine is a smoke test. We do not publish a recommended number because we have no data to support one.

Can I ever put my brand name in a question? Yes, in a question labelled as a vendor check, whose purpose is to see what an engine says about you when asked directly. It has to be excluded from the headline mention rate and reported separately, because it is a trial that can only come out one way.

What is the difference between a keyword and a question here? A keyword is matched by a ranking system; a question is answered by a model. The model answers the sentence in front of it, so the persona and the constraint inside the sentence change the answer. Pasting a keyword list into a monitoring tool measures keywords nobody typed.

Should my competitors' names appear in the questions? Our gate does not block them, because it only checks your own brand terms. It still changes what you are measuring: a question naming a rival returns an answer framed around that rival. Use category language instead, and let the engine tell you who the competitors are.

What language should the questions be written in? The language of the market being measured, not a translation of an English set. Our generator writes in the requested language. On 2026-09-20 we verified German and Japanese end to end on ChatGPT: the generated questions were idiomatic, brand matching hit 3 of 3, and competitor extraction returned sensible names. Gemini and Perplexity have not been verified in those two languages.

How often should I change the question set? As rarely as the business allows. A question needs at least four weekly runs before it enters a trend, so a replaced question restarts its own history. Add questions when you enter a new segment; replace them only when the old one no longer describes anything a buyer asks.

Do these rules apply to Google AI Overviews? AI Overviews and Google's AI Mode are outside what we measure, so we have no measurement of our own to offer about how question phrasing behaves there. The rules above come from what we observe on ChatGPT, Gemini and Perplexity through their official APIs.

The raw output of the run quoted here, including the questions as generated and the first 600 characters of each answer, is kept with the article so any figure above can be traced back to the answer it was counted from.