Ask an AI for a roofer.
What ChatGPT and Claude recommend when people ask for a local business, which sources they read, and how often the list changes.
When someone asks an AI assistant "who is the best roofer in Phoenix?", it does not return a page of links. It returns a short list of businesses: in our data, a median of at least five names from ChatGPT and seven from Claude (we stored up to eight per answer). F2b We asked ChatGPT, Claude and Gemini that question for 39 trades in 49 cities, towns and city districts in the United States and the Greater Toronto Area. Then we compared the answers with 2,976 local businesses found through Google Maps searches for the same trades and places.
ChatGPT answered almost every question. Claude and Gemini answered far fewer, for reasons on our side that we explain under what failed, so the findings rest on ChatGPT and Claude.
Key findings
- The two engines read different sources.Claude cited Yelp in 93% of its local answers (202 of 218). ChatGPT cited Yelp in none of 1,412. ChatGPT leaned instead on the Better Business Bureau (39% of its answers) and on government sites (25%), most of them licensing or contractor-registration pages. Reddit was cited in 6 of the 1,630 answers, all six by ChatGPT; a citation is a link shown with the answer, not everything a model has read. P1F4G1
- Reviews help, but they do not guarantee a place.ChatGPT named 3.4% of the businesses with fewer than 25 Google reviews and 22.2% of those with 1,000 or more. Yet the business with the most reviews among those we checked in a city (where we checked four or more) was left off ChatGPT's list about three times out of four. Star ratings did not help: the businesses named and the rest both had a median of 4.9 stars. F5
- The engines disagree.Of the businesses from our sample that ChatGPT or Claude recommended, 28% were recommended by both. Across the lists of names each engine gave, a median 12% of names were shared, and for 24% of the questions the two lists had no business in common. F2F2b
- Ask again and the list changes.When ChatGPT got the same question twice, a median 39 minutes apart, a median 29% of all the businesses named in either answer were named in both. F3
1. What we did
The question. For each trade and place we asked, word for word: "Who is the best {trade} in {city}? Name specific businesses." We asked it through each company's API with web search turned on.
The engines and models. Almost every fresh answer came from two models. ChatGPT: 1,398 answers from OpenAI gpt-5.6-terra, plus 14 from gpt-5.6-sol on 30 September. Claude: 200 answers from Anthropic Claude Sonnet 4.6, plus 18 from three other Claude models our checks used later in September. Gemini (gemini-2.5-flash) answered 20 times. S2 These are the models behind each company's API. A person using the app may get a different answer.
The businesses. The businesses come from Google Maps results for "{trade} in {city}" (up to the first 100 results, through DataForSEO's Google Maps data). We traced 2,903 of the 2,976 by name to those Google Maps searches, and each of those has its own website with public contact details; for the other 73 we hold no search record. On average there are about six businesses per trade and place. It is a selection, not every business Maps shows and not a random draw from them. Then we checked whether each AI answer named them. No business is named in this study. S4
| Businesses | Trade-and-place pairs | Trades | Places | |
|---|---|---|---|---|
| United States | 2,227 | 422 | 34 | 32 |
| Greater Toronto Area | 749 | 94 | 9 | 17 |
| Total | 2,976 | 516 | 39 | 49 |
The sample. S1 "Places" counts cities, towns and city districts (three of the Toronto-area places are districts of Toronto, and one is a community that spans two cities). One of the 39 trades is a fitness franchise searched by its name.
What "named" means. A business counts as named when its name, or a plain short form of it, appears in the engine's answer as a name: written with capitals, and not running on into a longer name (a different branch of the same chain does not count). Words scattered across a sentence do not count. The rule is code, the same for every answer, and it is published (matcher.py). We checked it by hand before publishing; see hand checks.
Sources. A source is the website domain of a link the engine cited with its answer. Gemini's links come back as Google redirect addresses, so Gemini is left out of every source figure.
Recurring questions. Separately, we ask a set of questions about local services in Toronto and nearby Ontario towns on a schedule. The set grew over time: 120 questions in all, 84 of them first asked on or after 22 September. That gives 2,527 ChatGPT and Claude answers from 1 July (ChatGPT from 23 August) to 5 October 2026. T3T4 We use them only for how much the cited sources change between runs, and for one Reddit count. Only pooled figures appear here: no question, no business, no list.
Dates. One-off answers: 10 to 30 September 2026. Nearly all fresh answers were collected on 10 and 11 September (1,391 of ChatGPT's 1,412 and 199 of Claude's 218). S3
2. Findings
2.1 Where the engines read
Share of fresh local answers that cite each platform at least once: P1
| Platform | ChatGPT (1,412 answers) | Claude (218 answers) |
|---|---|---|
| Yelp | 0.0% (0) | 92.7% (202) |
| Better Business Bureau | 38.7% | 6.4% |
| A government or licensing site | 25.1% | 0.0% |
| Expertise.com | 16.3% | 22.0% |
| Angi | 11.8% | 25.2% |
| Thumbtack | 3.7% | 19.3% |
| HomeGuide | 0.1% | 23.4% |
| ClassPass (fitness only) | 1.3% | 30.3% |
| Yellow Pages | 0.2% | 12.4% |
| 0.0% | 8.7% | |
| 0.4% (6) | 0.0% (0) | |
| Google Maps or Google business pages | 0.2% | 0.5% |
By kind of source F4: ChatGPT cited the website of at least one business we had found on Google Maps in 68.1% of its answers; Claude in 45.0%. Claude cited a directory or review site in 99.5% of its answers; ChatGPT in 65.3%. Social networks: 0.6% of ChatGPT answers, 11.0% of Claude's. The most-cited of the rest are national franchise brands' own sites, manufacturers' certified-contractor pages, a tree-care association, local businesses we had not found on Google Maps, smaller directories and ranking sites our list does not include, and encyclopaedia and news pages. F4b
Three things a reader would not guess:
- ChatGPT reads government sites. One answer in four cites one. We sorted every government address ChatGPT cited by a published keyword rule and read the list: in about 7 of 10 of those answers (250 of 355) it is a licensing board, a licence look-up or a list of registered or approved contractors; the rest are mostly permit rules, consumer guides and regulations. None is a tourism page. G1 Claude, asked the same questions, cited none.
- Google is mentioned, rarely linked. Google appears in the text of 23% of ChatGPT answers and 43% of Claude's, but neither engine links to a Google page in more than 1% of its answers. P1
- Reddit is close to absent. It was cited in 6 of 1,630 answers across both engines, all six from ChatGPT. On the recurring questions, ChatGPT cited it once in 1,354 answers and Claude never. T1b
2.2 Reviews help, but do not guarantee a place
ChatGPT, by the business's Google review count when we found it: F5
| Google reviews | Businesses | Named by ChatGPT | Share |
|---|---|---|---|
| under 25 | 435 | 15 | 3.4% |
| 25 to 99 | 903 | 90 | 10.0% |
| 100 to 249 | 775 | 108 | 13.9% |
| 250 to 999 | 527 | 110 | 20.9% |
| 1,000 or more | 135 | 30 | 22.2% |
- Median Google reviews: 175 for the businesses ChatGPT named, 97 for the rest.
- Median star rating: 4.9 for both. A higher star rating did not go with being named more often.
- In the 285 trade-and-place pairs where we checked at least four businesses with a review count, the one with the most Google reviews was named 22.8% of the time (65 of 285). The others were named 10.3% of the time (215 of 2,085). More than twice as likely, and still left out about three times in four.
Review counts come from the Google Maps search that found the business, and are on file for 2,775 of the 2,975 businesses ChatGPT answered for. "Most reviewed" means most reviewed among the businesses we checked in that place, not among every business on Google Maps.
2.3 The shortlist is short
| Engine | Businesses it answered for | Named | Share |
|---|---|---|---|
| ChatGPT | 2,975 | 369 | 12.4% |
| Claude | 1,776 | 198 | 11.1% |
| Gemini | 232 | 17 | 7.3% |
F1 Gemini answered too few questions to compare with the other two.
This does not mean the AI "ignores" nearly nine businesses in ten. The AI gives a shortlist (where both engines answered, a median of at least 5 names from ChatGPT and 7 from Claude F2b), and Google Maps gives a long list.
By trade, the fairer measure is how often ChatGPT named at least one of the Google Maps businesses we checked in a place: F6
| Trade (12 or more places) | Places | Businesses checked | At least one named | Each business named |
|---|---|---|---|---|
| Pilates studios | 21 | 206 | 90.5% | 16.5% |
| Women's fitness gyms | 15 | 170 | 80.0% | 9.4% |
| Towing | 14 | 82 | 64.3% | 19.5% |
| Moving companies | 13 | 93 | 61.5% | 15.1% |
| Garage door repair | 19 | 132 | 57.9% | 12.9% |
| Electricians | 19 | 115 | 57.9% | 12.2% |
| Appliance repair | 18 | 66 | 50.0% | 16.7% |
| Roofing contractors | 20 | 149 | 50.0% | 9.4% |
| Painting contractors | 21 | 102 | 47.6% | 17.6% |
| Plumbers | 19 | 230 | 47.4% | 8.3% |
| Landscaping | 17 | 112 | 47.1% | 8.9% |
| HVAC contractors | 20 | 125 | 45.0% | 7.2% |
| Fencing contractors | 12 | 54 | 41.7% | 16.7% |
| Window cleaning | 13 | 56 | 38.5% | 12.5% |
| Junk removal | 21 | 111 | 38.1% | 9.0% |
| Tree services | 19 | 90 | 36.8% | 8.9% |
| Pest control | 17 | 75 | 29.4% | 6.7% |
| House cleaning | 15 | 85 | 26.7% | 4.7% |
| Barre studios | 12 | 57 | 16.7% | 7.0% |
| Boutique fitness studios | 16 | 50 | 6.3% | 2.0% |
The more businesses we checked in a place, the likelier one of them was named, and that number differs by trade (from about 3 to 12 per place on average). Read the gaps between trades as a direction, not a ranking.
2.4 The engines disagree
On the 1,775 businesses where ChatGPT and Claude answered the same question at the same moment: F2
- 294 were named by at least one of them; 81 (28%) by both, 96 by ChatGPT only, 117 by Claude only.
- 198 businesses got an answer from all three engines; 29 were named by at least one engine, and 2 by all three.
Comparing the lists of names, including businesses we did not check (our check stored up to the first eight names it found in each answer): across 214 questions in 211 trade-and-place pairs, a median 12% of all the businesses named by either engine were named by both, and for 24% of the 214 questions the two lists had none in common. F2b Names were cleaned by a published script (lists.py), which dropped 18.9% of the 11,393 stored entries as not a business: an address, a directory, a summary line.
2.5 Ask again and the list changes
In 267 trade-and-place pairs ChatGPT got the same question twice, a median 39 minutes apart (61% within an hour). A median 29% of all the businesses named in either answer were named in both; for 9% of pairs the two answers had none in common. F3
The recurring questions show the same for sources. When ChatGPT answered the same question on consecutive runs, a median one day apart, a median 29% of the sources cited in either answer were cited in both (1,073 pairs). Claude, on one model (Sonnet 4.6), was steadier: 78% (805 pairs). T2
3. What this means for a local business
Each point says what the data shows, and what it does not.
- Each engine has its own sources. For Claude, Yelp was close to unavoidable in September (93% of local answers, mostly from Claude Sonnet 4.6). For ChatGPT, the Better Business Bureau (39%) and government sites (25%), mostly licensing records, came up more often than any review platform (Expertise.com 16%, Angi 12%, Yelp none). We did not test whether improving a profile on any of these changes the answer. This shows where the engines read, not what moves them.
- Keep the public record straight. ChatGPT cited a government site in one answer in four, most often a licensing board or licence look-up. A lapsed licence or an old business name in a public registry sits on the kind of page ChatGPT cites.
- Reviews count, up to a point. More reviews went with being named more often, but the most-reviewed business we checked in a place was still left out about three times in four. A higher star rating did not go with being named more often.
- One check is one sample. An hour later the list may be different. A single screenshot, yours or a vendor's, proves little. Ask how many times a question was asked, on which engine, on which date.
- Check each engine separately. Of the businesses we checked that ChatGPT or Claude recommended, nearly three in four (213 of 294) were recommended by only one of them.
4. Methods, limits and what failed
What failed
- Claude and Gemini often did not answer, because our own credits with those companies ran out. Claude answered 218 of 1,413 fresh questions; Gemini 20 of 1,413. S2 Every Claude figure from the one-off questions rests on those 218 answers, which is why Claude's per-business numbers cover 1,776 businesses and ChatGPT's cover 2,975.
- Across all the scheduled AI checks we ran from 1 July to 5 October, 37.5% of attempts failed (5,046 of 13,473), 4,720 of them for billing, quota or a spending cap on our side: Gemini 67.7%, Claude 37.5%, ChatGPT 7.1%. W1
- We do not store the model with each answer. The model periods are pieced together from our own deployment history. We also changed models, settings and the questions we track during the period, so the stability figures compare only consecutive answers from the same model period.
Limits
- API, not the app. We asked through each company's API with web search. A person using the app, signed in, with a location and a history, can get a different answer.
- A snapshot, not a trend. Nearly all fresh answers came on two days, 10 and 11 September 2026. A check could reuse the answer of an earlier check of the same trade and place for up to 7 days; a reused answer is counted once.
- Lists stop at eight names. Our check stored up to the first eight names it picked out of each answer, some of them not businesses. 164 of Claude's 214 stored lists and 110 of ChatGPT's reached that limit, so some answers named more businesses than we compared, and list figures are measured on the stored names. F2b
- Week-to-week data is thin. The recurring questions ran mostly a day apart. Pairs a week apart on one model: 34 for Claude (median overlap 67%), none clean for ChatGPT. T2
- Which businesses we could check. Only businesses with their own website and public contact details, and only a selection of those, not a random one. Businesses with no website are absent, and they may be the least visible of all.
- Review counts are from when we found the business (August or September 2026), not from the day of the answer.
- "A business website" is undercounted. It only recognises the sites of businesses we found on Google Maps. A named business we did not find falls under "other".
- Government sites are a floor. The rule counts addresses ending in .gov and a few named regulators; a state licensing portal on another kind of address is not counted.
- Two countries, 39 trades, mostly large US cities. Nothing on restaurants, retail, law or business-to-business buying, and of health care only med spas.
- The recurring questions come from one region (Toronto and nearby Ontario towns) and a few trades, and some ask for advice rather than for a business. They are used for how much sources change between runs and for one Reddit count, never for which businesses get recommended.
Hand checks before publishing
- "Named" verdicts. Our first, looser rule counted a business as named when every distinctive word of its name appeared somewhere in one sentence. We checked 50 of its "named" verdicts, chosen at random, against the answer text: 13 were wrong, mostly businesses whose names are made only of common words, counted as named in any sentence about their trade. For this study we replaced it with the stricter rule described above and checked that one on fresh random samples: 39 of 40 "named" verdicts were right, and the one error, a fragment of a longer name, led to a last fix in the rule. Among 40 "not named" businesses whose name shared a distinctive word with the answer, it missed one or two that the AI had written in a different form. Every per-business figure here uses the stricter rule. It can still miss a name the AI shortened or misspelled, so these rates are more likely a little low than high.
- Lists. We read 30 ChatGPT-and-Claude list pairs and 20 repeated-question pairs by hand. The script made no false matches. It missed two matches in one pair (the same business written with and without "of" before the city), and some lists kept an entry that is not a business, such as a neighbourhood or a licence number. Correcting both patterns moved the median from 12% to 11% and the share with nothing in common from 24% to 23%, and left the repeated-question figures unchanged; the published script is the uncorrected one.
- Government sites. We read the addresses of every government page ChatGPT cited (355 answers): licensing boards, licence look-ups, contractor registries, permit rules, consumer guides and regulations, plus a few unrelated pages. None is a tourism page. G1
Exclusions
- Only checks we ran ourselves. Checks run by visitors to our free tool may contain personal data and are not used.
- We left out one trade under a standing company rule (25 checks) and one trade-and-place pair to avoid a conflict of interest (24 checks).
- No business is named anywhere in this study. Platforms are named; businesses never.
5. The queries and scripts
Every figure from the data links to the query or script behind it; the hand-check counts are described in the text. The published copy is the SQL as run, with internal table names replaced by descriptive ones and our internal exclusion lists shown by name only. All queries were re-run before publishing, between 01:31 and 02:15 UTC on 6 October 2026 (the evening of 5 October in Toronto).
Corrections. None. A published study is not edited; a correction is added here as a dated note.
How to cite. 8ight research, "Ask an AI for a roofer", 5 October 2026, https://8ight.ai/research/how-ai-recommends-local-businesses/