# Known-Good

> An index of websites verified to be usable by AI agents. We read every robots.txt in a
> full public crawl of the web, find the sites that invite agents in, then probe them live
> to check whether they built anything an agent can actually use. About half had.

Known-Good grades sites against a published rubric and dates every claim. A declaration is
not a capability: we test rather than trust the label. The index is live and searchable.

We declare `ai-input=yes` and `ai-train=no`. Agents are welcome to read this site and use it
to answer someone's question. Please do not use it as training data.

## Pages

- [Search](https://knowngood.sh/index.md): how to query the index over REST or MCP. Markdown twin of the homepage.
- [About the index](https://knowngood.sh/about.md): how it was built, what we found, and the three things that make a site agent-ready.
- [Crawler information](https://knowngood.sh/bot.md): what our KnownGood-Verifier crawler does, what it fetches, and how to block it. Markdown twin of /bot.

## Search the index

- [Search API](https://knowngood.sh/api/find): `GET /api/find?q=<query>&country=<ISO>&transactional=<bool>&limit=<n>` — returns verified agent-readiness signals alongside apparent, model-inferred actions.
- [MCP server card](https://knowngood.sh/.well-known/mcp/server-card.json): tool `find_capability`, endpoint `/mcp`.
- [API catalog](https://knowngood.sh/.well-known/api-catalog): RFC 9727.

## Machine endpoints

- [Crawler IP ranges](https://knowngood.sh/bot/ips.json): the published source addresses our verifier crawls from, so you can confirm a request is genuinely ours.
- [robots.txt](https://knowngood.sh/robots.txt): our own content signals, declared honestly.

## Key findings

- 83,900,812 robots.txt files read across a full public crawl of the web.
- 2,171,626 of them carry a content signal — a declared stance on AI use.
- 57,740 domains explicitly welcome AI agents to read them. That is roughly 1 in 1,453.
- 32,533 of those were probed live; 14,425 (44.3%) had built something an agent can use (378 www/apex duplicate hosts collapsed to one report each, 26 Aug).
- 7,135 are sites where an agent can act, not only read. 62 countries represented.
- 86% of all content-signal declarations are one CDN's default rather than a decision anyone made.
- 10,565 of 32,533 probed sites (32.5%) genuinely negotiate markdown — body tested, not just the header. 41% of those are one platform's (Mintlify's) default, and are marked as such.

## Contact

- Crawler questions or opt-out: bot@knowngood.sh
- Everything else: hello@knowngood.sh

## Site reports

- [Per-site reports](https://knowngood.sh/sitemap.xml): every listed site has a dated report at `https://knowngood.sh/site/{host}` with a markdown twin at `/site/{host}.md` — what was tested, what passed, what an agent could do there, and which tier it has reached.
- [Rubric and tiers](https://knowngood.sh/benchmark): declared · probe-verified · behaviourally verified, with live pass counts per check.
- Browse by type, category, country or hosting platform: `/type/{slug}`, `/category/{slug}`, `/country/{ISO}`, `/platform/{name}`.
- [Sites with a working MCP server](https://knowngood.sh/mcp-servers): 658 sites whose own MCP server answered `tools/list` when we connected, grouped by what the tools let an agent do (act / query / needs credentials). Markdown twin at `/mcp-servers.md`. Query the same set with `has_mcp=true` on `/api/find`; results carry the tool names.
- [Request a probe](https://knowngood.sh/submit): a site not yet listed can ask to be tested. Placement is decided by the probe, never by the request.

## What is verified, and what is not

Probe results — markdown negotiation, `.well-known` endpoints, `llms.txt` — are tested live
and carry the date they were checked. Capabilities such as "book" or "buy" are read from
page content, labelled `apparent_`, and are unverified until an agent has completed them
(tier 3). Inference is kept separate from tested facts.