Why AI answers change every time you ask

Short answer. AI answers change because the model writes each answer fresh, choosing its words with some randomness; because servers add small variations even with that randomness turned off; and because the web searches behind the answer return different pages. Exact lists rarely repeat, but whether a brand is named at all is far steadier.

Ask an AI assistant to recommend a product, then open a new chat and ask again. There is a good chance the list is different: a name added, a name dropped, the order shuffled. It isn't a fault in your test. It is how these systems work, and it changes how you should measure them.

The short answer

Three things move an AI answer between one run and the next:

1. The model picks its words with some randomness. It doesn't look up a fixed answer; it writes one, word by word.
2. Even with the randomness turned off, the servers add variation. How requests are grouped on the hardware can change the result.
3. When the assistant searches the web, the search itself varies. It writes its own search queries, gets back different pages, and builds the answer from those.

Each is small on its own. Together they mean a recommendation list rarely repeats exactly.

Words are chosen, not retrieved

A language model writes by repeatedly predicting the next piece of text. At each step it has a spread of likely options, and it usually samples from that spread rather than always taking the single most likely one. That is what makes the writing sound natural instead of repetitive. It also means two runs can take a different turn early on, such as naming a different brand first, and then follow that turn to a different answer.

The chat apps people use don't let you control this setting. So any single answer you see is one draw from a range of answers the model could have given.

Even "no randomness" isn't fully repeatable

You might expect that turning the randomness to zero would give the same answer every time. In practice it often doesn't.

In September 2025 the research lab Thinking Machines tested this directly. They asked one open model the same question 1,000 times with randomness set to zero. They got 80 different completions. The answers matched for the first 102 tokens and then split. The cause was not the model's maths but how the server batches requests: the batch size depends on how busy the server is at that moment, and that changes the numbers slightly. When they rewrote the calculations so the batch size couldn't matter, all 1,000 answers came out identical.

You can't control any of this as a user of ChatGPT or Claude. It is simply part of why the same question at a different moment can come back different.

Search adds a second moving part

For "who should I use for X" questions, the assistants usually search the web first. That adds another source of change.

The model decides whether to search at all. Google's Gemini documentation says the model judges for each prompt whether a search would improve the answer, and if so writes one or more search queries itself. Google's AI Mode goes further and runs several related searches at once, then combines the results. Different queries bring back different pages, and different pages lead to different recommendations.

The cited pages churn quickly. Ahrefs tracked AI Overviews for more than 43,000 keywords over about a month and found the content changed between observations 70% of the time, with an answer lasting 2.15 days on average. Only 54.5% of cited URLs carried over from one observation to the next. The meaning of the answers stayed very similar. The wording and the sources did not.

What the research says about recommendations

Two studies looked specifically at brand recommendations, and together they give a useful picture.

Exact lists almost never repeat. SparkToro and Gumshoe ran 12 recommendation prompts 2,961 times across ChatGPT, Claude and Google's AI, with 60 to 100 runs per prompt per platform. ChatGPT and Google's AI returned the same list of brands in under 1% of repeated runs, and the same list in the same order in under 0.1% (Search Engine Journal, January 2026). Note that Gumshoe sells AI tracking, and the study wasn't peer-reviewed.

But whether a brand appears is much steadier. In the same study, for headphone prompts, the leading brands each appeared in between 55% and 77% of 994 responses. The authors' conclusion was that a position in the list is unreliable, but how often a brand appears across many runs is a steadier signal.

A separate analysis by Analyze AI of 22,295 answers to 460 prompts found that when a brand was mentioned in one observation, it was mentioned in the next about 83% of the time, across ChatGPT, Perplexity and Google's AI Mode (State of AI Search Persistence, August 2026). Brands that were missing tended to stay missing, 88% to 90% of the time. This is also vendor research.

The pattern across all of them: the order shuffles constantly, the sources churn, but who gets named at all is fairly stable.

What this means for measuring

If you are tracking how AI assistants talk about your business, a few rules follow directly.

Don't trust one answer. A screenshot of ChatGPT naming you first is one draw. So is a screenshot of it ignoring you. Neither tells you much on its own.

Report a rate, not a rank. "Named in 4 of 5 runs" holds up from one week to the next. "Ranked #1" may not survive the next refresh. Any tool or report that shows a single position from a single run is presenting noise as a number.

Keep everything else fixed. Same questions, same wording, same setup, same time of day, so the only thing that changes is the answer. Otherwise you can't tell a real shift from a change in how you asked.

Look at sources over a longer window. Because cited pages churn faster than brand names, a daily list of sources jumps around. Pooled over a week, it shows which pages the engines keep coming back to, and those are the pages worth improving or getting into.

Keep the full answers. When a number moves, you will want to read what the AI actually said, word for word. A tally without the text can't tell you why.

The takeaway

AI answers change because the model samples its words, the servers add small variations, and the web searches behind the answer vary. None of that is going away. The useful response is to stop treating any one answer as the truth and measure the way the variation demands: several runs, a fixed method, a rate rather than a rank, and the full text kept so you can check it.

That is how aipozi reports results. If you want to do it by hand first, here is how to check whether ChatGPT recommends your business.

Frequently asked questions

Will ChatGPT give the same answer if I ask the same question twice?

Often not. In a study of 2,961 runs by SparkToro and Gumshoe, ChatGPT and Google's AI returned the same list of brands in under 1% of repeated runs, and the same list in the same order in under 0.1%.

Does turning the randomness (temperature) to zero make answers identical?

Not reliably. Thinking Machines asked one model the same question 1,000 times at zero temperature and got 80 different completions, because the way servers batch requests changes the arithmetic slightly.

Do the sources an AI cites change as well?

Yes, and faster than the brand names. Ahrefs found that only 54.5% of the URLs cited in Google's AI Overviews carried over from one observation to the next.

How should I measure AI visibility if the answers keep changing?

Ask the same questions several times with a fixed method and report a rate, such as named in 4 of 5 runs, rather than a single rank. Keep the full answers so you can see why a number moved.