Key takeaways
- AI visibility tools collect more than most buyers assume: tracked prompts, full AI responses, citations, fan-out queries, and sometimes server logs and analytics events. Each one needs its own retention answer.
- Ask two separate questions about training: does the vendor train on your data, and do its model providers? A "no" with no contract clause behind it is not a no.
- Retention is per data type and per feature. A single number for the whole stack usually means the vendor has not read its own contracts.
- Anything a vendor tells you in a sales call should end up in the DPA or the main contract. Paper beats promises.
- I found no published retention periods or training policies for most AI visibility vendors, so you will have to ask for them directly.
Why this vendor category needs its own questions
Most AI vendor questionnaires were written for copilots, support bots and transcription tools. AI visibility platforms are different. They mostly don't ingest your customer records. What they hold is your brand strategy: which prompts you track, which competitors you watch, which pages get cited, and in some cases your server logs, search console data and CMS credentials.
That is still sensitive. Your prompt list is a map of your commercial priorities. Your crawler logs show your site's technical weak spots. If the vendor also publishes aggregate research, you want to know whether your data feeds it.
The generic questionnaires are a useful base. Reco's 40-question AI vendor questionnaire makes the point that a vendor can hold a clean SOC 2 and still retain embeddings indefinitely or route data to a third-party model. SOC 2 does not tell you either thing.
Below, I've adapted those frameworks to the specifics of GEO and AI visibility tools.
Step one: map what the vendor actually collects
Before you ask about retention, work out what there is to retain. In this category, vendors typically handle some mix of the following.
| Data type | What it is | Why it matters |
|---|---|---|
| Tracked prompts | The questions you ask AI engines on your behalf | Reveals your strategy and competitors |
| Responses and citations | Full AI answers, source lists, click data | Usually not personal, but may be reused in vendor datasets |
| Fan-out queries | The web searches AI runs behind each response | Vendor-derived, often stored for analysis |
| Crawler and server logs | CDN or origin logs showing AI bot visits | Can contain IP addresses and URLs |
| Visitor analytics | Traffic and conversions from AI platforms | May include personal data under GDPR |
| Search Console and Bing data | Per-page search performance | Connected via OAuth, so scope matters |
| CMS credentials | Tokens for publishing to WordPress, Webflow or Framer | Write access to your site |
| Brand documents | Knowledge bases, brand books, uploaded source files | Often confidential |
Ask the vendor to fill in this table for their own product. If they can't tell you which of these they store, that tells you something.
Real UI monitoring versus API calls
There is a collection question that affects privacy and accuracy at once. Some platforms query model APIs. Others monitor the actual user interfaces of ChatGPT, Gemini, Perplexity and the rest. User-facing answers and citations can differ from API output, which is why this matters for data quality. Ask which method the vendor uses per engine, and what they store from each. Tools like Promptwatch say they monitor real interfaces across countries and languages and track API models alongside; whichever vendor you pick, get the same statement in writing.
Questions about retention and deletion
This is where vague answers hide. The Censinet checklist puts it well: get deletion dates for prompts, logs, backups and third-party copies, not a single reassurance.
Send these:
- How long do you retain tracked prompts, raw responses, citations and fan-out queries? Is there a separate period for each?
- How long do you retain crawler logs and visitor analytics events, and are IP addresses stored or truncated?
- Can we shorten any of these periods ourselves, or only the vendor can?
- When we delete a project, what is removed, and how fast? Are backups, search indexes and analytics warehouses included?
- At contract end, how do we export our data, and do you provide written confirmation of deletion?
- Do legal holds or abuse-monitoring requirements ever extend retention beyond what you've stated?
Red flags: "as long as necessary", one figure for everything, deletion that only removes data from the UI, or a refusal to put deletion timelines in the contract.
The model-provider layer
Many AI visibility tools use an LLM underneath for content generation, summarisation, sentiment scoring or chat-style analysis. That adds a second retention clock. The DeepInspect questionnaire points out that retention is now set per feature rather than per provider. It cites OpenAI keeping API inputs and outputs for up to 30 days by default for abuse monitoring, with zero data retention available only to approved customers on eligible endpoints. For Anthropic's Claude API it notes that conversation content isn't retained by default, but exceptions exist: batch jobs, stored files, code-execution containers, and trust-and-safety flags that can extend retention up to two years.
The lesson: ask which model features the vendor uses, then ask for retention on each. A vendor claiming "zero data retention" should be able to say whether that is a signed ZDR arrangement with each provider, or just default behaviour. Enterprise search vendors like GoSearch describe zero data retention agreements with LLM providers on their own site; treat any such claim as something to evidence, not accept.
Questions about model training
This is the section where a single wrong answer should stop the deal. Reco's rubric recommends automatic decline or escalation for any zero score on its training questions, because once data has trained a model, removing its influence is hard or impossible to verify.
Ask:
- Do you use our prompts, responses, uploaded documents, feedback or metadata to train, fine-tune or evaluate any model? Yours or a third party's?
- Is the answer the same on every plan? Defaults often differ between free, individual and business tiers, as the Osano guide warns.
- Is the no-training commitment in the contract and DPA, or only in a privacy policy that can change? Osano gives Slack's 2024 default-on analysis and Zoom's 2023 terms rewrite as examples of terms moving after signup.
- Do your LLM providers have contractual no-training terms, and can you point to the clause?
- Does the commitment cover AI features you add later?
- Do your staff or contractors read our content, and under what conditions?
Good answer: no, stated in the contract, covering prompts, outputs, files and metadata, with reading limited to logged support cases you opened. Red flags: opt-out only, a buried toggle any admin can flip, undefined "service improvement" language, or a sales answer that doesn't match the privacy policy.
Do your prompts feed their public datasets?
AI visibility vendors have an extra wrinkle: many publish research. Promptwatch, for example, publishes aggregated, non-identifiable trend reports through Promptwatch Data, built on its own large dataset of prompts and citations. Publishing aggregates is not a problem in itself, and the research is useful. But ask any vendor the direct question: are our tracked prompts or responses included in your aggregate or public datasets, how is identifying information removed, and can we opt out in the contract? Make the answer explicit rather than assumed.
Questions about crawler logs and analytics
If the platform offers AI crawler analytics, you are handing over server or CDN logs. Promptwatch's AI crawler traffic report is a good illustration of what that data looks like: it counts only IP-verified requests, so spoofed user agents are excluded, and it separates training crawlers from answer-time fetchers. That is useful analysis, and it is also exactly the sort of data you should scope carefully.
Ask:
- Which log fields do you ingest? Can we exclude IP addresses, query strings or cookies?
- Do you filter to AI bot traffic at the edge, or ingest everything and filter later?
- How long are raw logs kept versus aggregated results?
- Do you verify bot IPs against the operator's published ranges, or rely on user-agent strings?
- Which integration method do you use (CDN push, worker, plugin), and what permissions does it need?
- For visitor analytics, are you a processor under GDPR, and how do you handle consent and IP data?
If you are in Europe, the Kiteworks GDPR guide gives a useful list: a DPA, a complete sub-processor list, valid transfer mechanisms (SCCs or adequacy), and retention terms specific enough for GDPR Article 5(1)(e). It also notes that a DPA doesn't govern what an agent does with data once it has access, which matters for the next section.
Questions about sub-processors and data location
The Bitsight guide and the Osano piece both start from the same observation: most AI tools don't run their own models. Your data goes to a provider underneath.
Ask:
- Give us a current list of every sub-processor, including model providers, with purpose and location for each.
- How do you notify us of changes, and can we object?
- Can you pin storage, processing and backups to a named region? Does failover ever move data elsewhere?
- Do your sub-processors carry SOC 2 or ISO 27001 themselves?
- Can we bring our own model account for any AI feature?
Red flags: the model provider missing from the list, changes posted on a web page with no notice, and certificates whose scope excludes the AI product or belongs only to a parent company.
Questions about write access and automation
Some platforms stop at monitoring. Others write and publish content for you, which changes the risk profile completely. A tool that holds CMS credentials can change your live site.
- Which CMS permissions do you request, and can they be limited to drafts?
- Is there a human review step before anything publishes, and can we require it?
- What is logged: who approved what, which model version generated it, which sources it used?
- Can we revoke access instantly, and what happens to scheduled jobs when we do?
- Are your Knowledge Base and brand documents isolated per tenant?
For a content agent, ask whether the review inbox is mandatory or optional. A fully hands-off mode is fine if you chose it deliberately. It is not fine as a default.
Questions about security and incidents
Keep this short, since generic frameworks cover it well, but don't skip it:
- Which report covers this product (SOC 2 Type II, ISO 27001), with what scope and dates?
- Do you support SSO, role-based access and audit logs?
- What is your breach notification deadline? Censinet suggests terms like 24 to 72 hours.
- How do you test against prompt injection, especially for anything that reads third-party web pages?
- How are API keys and MCP or API access scoped?
A quick screening table
| Topic | Evidence to request | Red flag |
|---|---|---|
| Retention | Schedule per data type, deletion SOP, backup policy | One figure for everything |
| Training | DPA clause, provider terms, plan-by-plan defaults | Opt-out only, undefined improvement use |
| Sub-processors | Named list with purpose and region | Model provider missing, no change notice |
| Crawler logs | Field list, retention, IP verification method | Ingests everything, keeps indefinitely |
| Aggregate data | Written statement on dataset inclusion | No opt-out |
| Publishing | Permission scope, review workflow, audit log | Mandatory hands-off mode |
| Exit | Export format, deletion confirmation | Chat transcripts only |
Putting it in the contract
The single most useful rule across all the sources is the same: if the vendor won't write it down, discount the answer. Reco says its Question 40, getting the answers into the contract, matters most. Before go-live, make sure the paper covers:
- A no-training clause covering prompts, outputs, files, metadata and logs, including at model providers
- Retention periods per data type, with backups included
- Sub-processor notice and a right to object
- Region commitments for storage and inference
- Breach notification deadlines
- Export and deletion at exit, with written confirmation
- Notice before the vendor swaps the underlying model or adds a new provider
Then re-ask on every trigger: a new model, a new sub-processor, a new agent capability, an incident, and each renewal. Questionnaire answers are claims made on one day about a product that changes weekly.
How to run the process without it taking months
- Tier the vendor. A monitoring-only tool that sees prompts and public responses needs the training, retention and sub-processor sections. A tool with log access and CMS publishing needs everything above.
- Ask for existing artifacts first: DPA, sub-processor list, SOC 2 report, security page, model documentation.
- Send only the gaps, with a deadline and an evidence requirement per question.
- Score by weighting the domains that matter to your use case. Treat any training or contract refusal as a stop.
- Record refusals. A refusal is an answer.
A realistic timeline for a mature vendor is a couple of weeks, including documents. Much longer usually means they can't map your questions to their architecture.
Shortlisting vendors before you send the questionnaire
Security answers only matter once the product fits your needs. If you're still building a shortlist, the directories at bestgeosoftware.com and ai-rank-tools.com cover GEO platforms and AI rank trackers respectively. The catalog here includes monitoring-focused tools such as Otterly.AI, Peec AI, Profound and Scrunch, plus broader suites.
I did not verify retention or training policies for any of these, and I couldn't find them published in a form I'd rely on. That is the point of the questionnaire. Send the same questions to each vendor on your shortlist, including the one you like best, and compare the written answers side by side.
Final thought
The vendors worth buying from will answer quickly, name their sub-processors, and agree to the clause. The ones who respond with "we take privacy very seriously" and a link to a trust page have answered a different question. Hold the deal until the paper says what the sales call said.