8 keyless GET endpoints: crawler-IP verification, robots.txt lint, AI-access audit and user-agent classification. No account, no key, no handshake, CORS open, cacheable, CC0. Each is the same implementation as the MCP tool of the same name.
ENDPOINTS
STRUCTURED FROM OPENAPI 3.1
GET
/data/agents.json
Every crawler record, plus categories and an endpoint map
GET
/crawler/{slug}.json
One crawler record by slug
GET
/data/agents.csv
The same table as CSV
GET
/data/ua-regex.json
Pre-escaped user-agent regexes, whole-list and per category
GET
/data/ip-sources.json
Which operators publish verifiable IP ranges, and where
GET
/ip-ranges/all.json
Union of every operator-published prefix, grouped by source
GET
/ip-ranges/all.txt
The same prefixes, one CIDR per line
GET
/ip-ranges/{source}.json
One operator's published prefix list, normalised
GET
/robots/{policy}.txt
A ready-made robots.txt policy file
GET
/policy/{policy}.json
A policy with its rationale and the crawlers it names
GET
/documents.json
The document ledger: every URL here, its ETag and its last-modified
GET
/documents.txt
The same ledger as TSV: path, kind, bytes, etag, last-modified