How we test what AI says about you
What the test is
We ask AI assistants the questions your buyers ask: some with your name ("How much does Fakturo cost per month?") and some without it ("Which invoicing software should a freelancer choose?"). We compare every answer with the facts you confirmed on your own pages: a price, a trial, an integration, the company behind the product.
The AI never sees your facts and does not know it is a test. It only gets the question, in the language of your market, with one neutral instruction: "Answer as an assistant helping a buyer. Use web search." A question that names you ends with "Source:" and your website address.
Which AI we ask, and how
- OpenAI: the model gpt-6-luna through the OpenAI API with web search. This is not a test of the ChatGPT app.
- Anthropic: the model claude-haiku-5.5 through the Anthropic API with web search and page fetch. This is not a test of the Claude app.
- Google Gemini and Perplexity are asked for our own records only. Their answers are not scored and never shown.
Every call goes through an API. We never drive the consumer apps: their terms forbid automated use, and an API answer can differ from the answer the same provider gives in its app. The approximate country of your market is sent to OpenAI and Anthropic, so the answer is the one a buyer there would get.
A person changes the models, never a script. When a model changes, a comparison between a test before and after the change is marked as not comparable instead of being shown as a result.
Runs and verdicts
Each question is asked three times per assistant, in every test. "Right in 4 of 6 runs" counts the completed runs of both assistants whose answer matched the information you confirmed. A provider error or an empty answer is retried once and is not counted.
- Right: the answer states your fact with the same value and condition. A fact is right for one assistant when at least 2 of its 3 runs are right.
- Wrong: the answer states a different value. Only an owner who confirmed the fact can see "wrong"; a public report says "not on your page".
- Missing: the answer does not state the fact, or only says it cannot find it.
- Varies: the runs disagree, with no 2 of 3 on one side.
- For a question without your name, each run is judged on one thing: did the answer name you.
An evaluation model judges every run against every relevant fact with one question: does this answer state the fact as this value? A plain text match of the value is stored next to it and decides only when the judge is unavailable. "Facts AI gets right, N of M" counts your confirmed facts that both assistants get right.
What AI said instead is shown only in its own words: a wrong value appears when its quote is word for word in the answer, and the products AI recommended instead of you appear only when their names are in the answer. Nothing is paraphrased or guessed.
AI answers change
AI answers change. We measure several times. The same question can get a different answer on another day, so one run proves little: that is why every test has three runs per assistant, and why Autopilot asks again every week.
A change to your page counts as a fix when the right runs grow by at least two. A rise of one is shown as "probably helped", never claimed as a fix. Before and after are compared only with the same facts, the same questions, the same models and the same location.
Independent, and an API test
Check Your Website is independent and is not endorsed by OpenAI, Google, Anthropic or Perplexity. Every answer we show is an AI-generated answer from an API test. It may differ from answers in the provider's consumer app. We show no provider logos.
The owner of a website is responsible for the facts they confirm. A claim we know to be untrue is refused before Autopilot writes it into a page.
What you see on Home
For every question: the verdict, one finding in plain words, the runs that got it right over both assistants, the products AI named instead of you, and the sites AI read. Under it, one line says which providers were asked, when, in which language, and that it was an API test.