By Carson Cobb. Data collected November 2026.
Most AI visibility checks use short questions, the kind you’d type into Google: “best weighted blanket”, “best creatine brands”. Real shoppers don’t talk to AI that way. They describe themselves: “I have anxiety and trouble falling asleep, and I sleep hot. I’m thinking about a weighted blanket that won’t make me overheat. Which one should I get?”
Does the wording change the answer? I took 24 short buyer questions and rewrote each one the way a real shopper would ask it, in the language customers use in reviews. Then I asked both versions on ChatGPT and Google AI Mode, in fresh sessions, and compared the brands.
The short version
- The top pick changed in most questions. ChatGPT kept the same first brand in 9 of 24 questions. AI Mode in 8 of 24.
- Half the list changed. ChatGPT named only 50% of its short answer brands again for the long version. AI Mode 38%.
- That’s far more change than randomness. Asking ChatGPT the same short question again kept the top pick 98% of the time and 77% of brands.
- Long questions get shorter lists. Median brands per answer fell from 4 to 3 on ChatGPT and from 5 to 3 on AI Mode.
- Growing Shopify brands lost a third of their mentions. From 25 to 16 on ChatGPT and from 33 to 20 on AI Mode.
How much the answers changed
| ChatGPT | Google AI Mode | |
|---|---|---|
| Questions compared | 24 | 24 |
| Same top pick | 9 | 8 |
| Short answer’s top pick named anywhere in the long answer | 17 | 12 |
| Short answer brands named again | 50% | 38% |
| Long answer brands that were new | 27% | 38% |
| Median brands, short question | 4 | 5 |
| Median brands, long question | 3 | 3 |
| Long answer named fewer brands | 16 of 24 | 22 of 24 |
On AI Mode, the brand ranked first for the short question didn’t appear at all in the long answer half the time.
Wording beats randomness
AI answers vary from run to run, so some change is expected. To separate wording from randomness, I compared these results with repeat runs of the same short questions on ChatGPT. See how often AI recommendations change.
| ChatGPT | Asked again the same way | Asked in a shopper’s words |
|---|---|---|
| Top pick kept | 98% | 38% (9 of 24) |
| Brands named again | 77% | 50% |
Asking again changes the end of the list. Rewording the question changes the top of it. If your checks use short keyword questions, you may be measuring a shortlist your customers never see.
Why long questions change the answer
A short question asks for the most recommended brands in general. A long question asks for the right product for one person, with constraints. The AI then has to find evidence that a product meets them: sensitive skin, sleeping hot, wide feet, nut free for school, third party tested for heavy metals, fits a car cup holder.
That changes which sources matter and which brands qualify:
- General favorites drop out when nothing on their pages or in their reviews addresses the constraint.
- Specialists come in when a review, test or product page says plainly that the product is made for that need.
- The list gets shorter because fewer products clear the bar, and the AI spends more of its answer explaining fit than listing options.
Long answers weren’t shorter overall. ChatGPT’s long answers averaged about 440 words against 340 for short ones. More words went to explaining why a product suits the shopper, and fewer brands made the cut.
What happened to growing Shopify brands
I flagged brands with their own Shopify store from my dataset of growing US brands.
| ChatGPT short | ChatGPT long | AI Mode short | AI Mode long | |
|---|---|---|---|---|
| Mentions of brands from my dataset | 25 | 16 | 33 | 20 |
| Answers naming at least one | 14 | 11 | 16 | 15 |
It’s common to assume smaller brands win on specific questions. In this test, specific questions shrank the list, and growing brands lost more mentions than they gained. The ones that held on were the brands whose product information clearly matched the need in the question. A brand that’s known as a good option, but whose page never says who it’s for, is exactly the kind that drops out.
What to do with this
- Write your test questions the way your customers talk. Pull the wording from your reviews, support tickets and post purchase surveys. Include the situation, the constraint and the doubt.
- Keep a short version too, but don’t rely on it. It tells you about general reputation. The long version tells you about the shopper who’s ready to buy.
- Make your product pages answer the constraint. Who it’s for, who it isn’t for, sizing, ingredients, testing and how it compares. See do your product pages answer what shoppers ask AI?
- Track the same questions over time. Same wording, same number of runs, fresh sessions. Change the wording and you’ve changed the test.
Building buyer questions from your customers’ own language, and testing them on ChatGPT, Gemini and Google AI Mode, is how my AI Search and Conversion Audit starts. The free AI Visibility Check is a quick first look.
How this was done
- Questions: 24 of the 48 short buyer questions from Who ChatGPT recommends, 4 each from apparel, beauty and skincare, food and beverage, home, pet and supplements. Each was rewritten as one long question of 23 to 34 words describing a shopper’s situation and constraints, modeled on how customers describe themselves in product reviews.
- How they were asked: ChatGPT on a paid plan with the default model, each question in a new temporary chat. Short questions on November 4, 2026 and long questions on November 5, 2026. Google AI Mode in a new thread for each question, signed in with no apps connected, both versions on November 5, 2026.
- Brands: every brand named as an option to buy, matched against a list of known brands and aliases for each question and checked by hand. Brands named only in the question itself, like a big name the shopper wanted to compare against, were left out.
- Baseline: repeat runs of 12 short questions on ChatGPT, from how often AI recommendations change.
- Limits: one long version per question and one run of each version per AI. A different rewrite could shift the results in either direction.
No store or brand is named in this report.