Our crawler

DealerCitedAudit

DealerCitedAudit is the crawler DealerCited uses to check whether a dealership’s website can be read by AI search engines. It reads a handful of public pages, the way a plain crawler would, and reports what it found to the dealership.

How it identifies itself

Every request carries this user agent:

DealerCitedAudit/1.0 (+https://dealercited.ai/bot)

The robots.txt token is DealerCitedAudit.

It sends every request from one of two fixed addresses, used for nothing else:

100.61.47.62    our live service
54.221.234.191  our test environment

Every request is also signed with Web Bot Auth (HTTP Message Signatures), so bot protection that checks signatures can confirm it really comes from us. Our public key is at /.well-known/http-message-signatures-directory.

What it fetches

For each audit it reads your robots.txt, your sitemap, your llms.txt, your homepage and one inventory page. It checks the dealership’s own site and the sites of the competitors that dealership compares itself against.

About 10pages per site, per audit
1 secondbetween requests, or longer if your crawl-delay asks
1 retryat most, then it records what it got
Monthlyper site by default

It reads raw pages without running JavaScript. When a dealership’s own homepage builds key details with JavaScript, it may load that page again with its JavaScript running, along with the scripts and images the page itself asks for. Requests to your site carry the same name, addresses and signature. It never fills in forms, logs in, or follows links beyond those pages.

It obeys robots.txt

It reads your robots.txt first and skips anything your rules close to it. To keep it off your site entirely, add:

User-agent: DealerCitedAudit
Disallow: /

If you do, it stops there. It never switches to another user agent to get around your rules, and the dealership’s report says the audit couldn’t run.

Letting it past bot protection

Bot protection from your CDN or website provider can answer our auditor with a challenge page instead of your site. When that happens, the dealership’s report shows the checks as “couldn’t run” rather than guessing.

To let it through, allow requests whose user agent contains DealerCitedAudit and that come from 100.61.47.62 or 54.221.234.191. Matching the name and the address keeps out anyone else who borrows the name. If a website provider runs your site, send them this page.

Questions or problems

If the auditor is causing you trouble, write to [email protected] with your site’s address and we’ll look into it.