Key takeaways
- The Washington Post asked six popular chatbots to build voter guides for addresses in every US county, then checked more than 3,000 of them. Secondary coverage of the analysis reports the congressional district was missing 48% of the time and wrong 22% of the time when given.
- The errors were not random. Several traced back to small information sites that appear to be AI-built, which chatbots then repeated.
- A chatbot that answers "correctly" by skipping the district is not accurate. It is avoiding the hard part.
- For anything tied to an exact address, use the official election office or state lookup tool, and treat the chatbot as a starting point.
- Brands with local audiences face the same problem in a lower-stakes form, and can measure it by testing prompts from specific places.
What the Post actually tested
On October 24, 2026, Kevin Schaul and Ian Duncan of The Washington Post published We asked six chatbots to make voter guides. They made serious errors. The setup is simple. Take an address in every county in the country, ask each chatbot for a voter guide, and check what comes back.
The motivation is practical. The Post cites a National Conference on Citizenship survey in which 2 in 5 Americans said they had used AI to look up election information. People are asking exactly this kind of question.
A caveat on sourcing: the Post page is paywalled, so I could only read the intro directly. The numbers below come from secondary write-ups of the Post's analysis (mainly explainx.ai and shiftdelete.net), and I have not checked them against the full Post text or its methodology. Treat them as reported figures, and read the original if you plan to quote them.
The numbers as reported
| Question | Reported result |
|---|---|
| Chatbots tested | Six popular chatbots |
| Addresses | One or more in every US county |
| Guides checked | More than 3,000 |
| Congressional district missing | 48% of the time |
| District wrong, when named | 22% of the time |
| Worst on wrong districts | Claude and Alexa+, each wrong for more than 1 in 5 addresses |
| Looked better on paper | ChatGPT, Google AI Mode, Perplexity, reportedly because they usually gave no district at all |
| Copilot citing the AI-built Ballot Brief | 75% of midterm answers (ChatGPT: 22%) |
The examples in the Post's opening are blunt. A Missouri Senate race that does not exist. A House candidate in Washington who is not really on the ballot. Secondary coverage adds ChatGPT listing a Senate seat in Missouri that is not up this year, and Claude naming two Wyoming Senate candidates who, per the Post, are running for other offices.
Why location is the hard part
A voter guide is a stack of dependent answers. The address determines the congressional district. The district determines the House race. The county and state determine local contests and ballot measures. One wrong step at the top and every line below it is wrong, even if each line reads fine on its own.
That is why the district figures matter more than a general error rate. A chatbot has to do three things in order:
- Turn a street address into the right geographic units.
- Find current information for each unit.
- Present it without mixing up neighboring places.
Language models are good at step three. Steps one and two need a precise lookup against current data, and a chat model with a search tool is an awkward substitute for a geocoder.
The "best on paper" trap
The reported detail I find most interesting is about ChatGPT, Google AI Mode and Perplexity. They had fewer wrong districts, but according to the coverage that was largely because they often did not name a district at all. That is an honest failure mode compared with a confident wrong answer, but it still leaves the voter without a guide. A low error rate and a useful answer are different things, and a headline number hides that.
How search behavior can lose the address
There is a plausible mechanical reason location gets lost, and I want to be clear this part is my inference, not a finding from the Post.
When a chatbot browses the web, it usually does not send your whole prompt to a search engine. It splits it into short searches. Promptwatch Data's ChatGPT query fanout report shows an average of about 2.15 searches per response in early December 2025 and about 1.84 by early March 2026, with average query length falling from roughly 117 characters to the high 80s, and about 53 characters once fanouts returned in April. Short, keyword-like queries are a poor fit for a request that depends on a full street address. Something is plausibly lost between the prompt and the search.
And once searches run, the answer gets stitched together from many places. Promptwatch's data on unique domains per response found that 47.6% of ChatGPT responses cite 10 or more domains (Sep 28 to Oct 4, 2026), and organic prompts with no brand named were the most crowded at 52.5%. That report is built on commercial and brand-style prompts, not election queries, so it does not describe voter guides directly. But it illustrates the shape of the problem: many sources blended into one smooth answer, with no guarantee that all of them are about your district.
When the sources are machine-made too
The second thread in the Post's reporting is sourcing. A companion piece, How an obscure website became a force in America's election information machine, describes Ben Torres Ezrick's Ballot Brief. He built it because hunting for candidate information was frustrating, and he used teams of digital agents to build dossiers on thousands of candidates.
According to the secondary coverage, Copilot cited that site in 75% of midterm answers and ChatGPT in 22%, and some mistakes passed straight into voter guides. The pattern is a loop. An AI-built site publishes an error. A chatbot cites it with confidence. The reader has no reason to suspect the original page was machine-generated and unchecked.
I don't think the site's creator is the villain here. Building a one-stop shop for voters is a reasonable idea. The problem is that retrieval systems treat a tidy, well-structured page as authoritative, and nothing in the pipeline asks who verified it.
Is this a one-off, or does it generalize?
It generalizes, with a twist. Election answers are an extreme case because the correct response is a single, checkable fact tied to a place. Most local questions share the structure:
- Which clinic, store or office is closest to me?
- What are the rules where I live?
- Who is the best provider in my city?
The same failure modes apply: wrong geography, stale pages, thin third-party sources repeated as fact. The Post's earlier political bias test is a useful companion on method. It asked each model each question five times to check consistency, because the same chatbot can answer the same question differently from one run to the next. Any serious test of location behavior needs that repetition too.
What to do as a voter
The practical advice is boring, and it is right.
- Use your state or county election office to confirm your district, polling place and ballot. Those sites exist for exactly this lookup. The YouTube summary of the Post findings also says to use the official site, though I have not verified which sites it names.
- If you use a chatbot, give it the address and then check the district it names against the official lookup before reading anything else.
- Ask for sources and open them. If the guide leans on a site you have never heard of, treat it as unverified.
- Be suspicious of races that sound off. A Senate contest in a state with no Senate election this year is a quick tell.
- Use chatbots for explaining terms and ballot measures in plain language, where one wrong district does not poison the whole answer.
The point of the explainx.ai write-up is similar: where the right answer needs an exact address lookup, hand the user to the authoritative tool.
What to do if you run a local brand
You are not publishing voter guides, but your customers ask location-dependent questions too. The useful move is to test your own answers the way the Post tested chatbots: same question, different places, repeated runs.
That means checking whether the right branch, price or service area shows up when the query comes from different cities, and which sources each answer cites. If a thin third-party directory is the main source of facts about your business, that is the Ballot Brief problem in miniature.
Tools that can help with this kind of monitoring include Promptwatch, which tracks ChatGPT, Google AI Overviews and others in specific countries and can pin a monitor to exact coordinates for local queries.
Other options in the category include:
| Need | What to look for |
|---|---|
| Test answers from a specific city or spot | Location-level prompt tracking, not only country-level |
| See which sources AI cites about you | Citation reports by domain and page |
| Catch repeated mistakes | Daily runs, so one bad answer does not mislead you |
| Fix weak sources | Content and listing tools that update the pages AI reads |
If you are comparing options, the GEO software directory at bestgeosoftware.com lists more of them.
What this test does not tell us
Some honest limits. I have not seen the Post's methodology, so I cannot say exactly which prompts it used, on which dates, or the per-chatbot rates beyond what secondary outlets report. Chatbots change often, and a result from October may not hold in December. And the "differ by location" angle is supported indirectly: the Post prompted with addresses everywhere, and the district errors show location resolution failing, but the article is not a controlled study of how the same question changes between two cities.
That last point is worth testing yourself. Ask the same local question from two places and compare.
Bottom line
The Post's experiment shows chatbots are weakest where the answer depends on an exact place and a current fact, and where their sources are themselves unchecked. For voting, verify with your election office. For anything else that depends on where you are, assume the answer may be about the wrong place until you have checked.