Can I reach it?

An access map of the agent web built from behavior, not declarations. We ask the top 1000 domains the same question as five different callers and record what actually comes back: the status, the challenge, the toll, and whether the site drew us a detour.

The finding

Being an agent is cheap. Being a recognised agent is expensive.

An ordinary browser was denied by robots.txt on 31 of 1000 domains. The same request wearing a published vendor token was denied on 103 (gptbot) and 105 (claudebot).

An agent that identified itself honestly but under a name nobody has heard of was reached 55.3% of the time — slightly better than the browser control at 53.9%. The penalty attaches to the name on the door, not to being automated.

By caller

Caller	Asked	robots denied	Reached	Rate	Challenged	Blocked
`browser` human-baseline	969	31	522	53.9%	65	26
`curl` naive-script	969	31	500	51.6%	69	42
`canireach` self-identified-agent	969	31	536	55.3%	51	26
`gptbot` vendor-token-disclosed	897	103	411	45.8%	75	50
`claudebot` vendor-token-disclosed	895	105	415	46.4%	69	40

“Asked” counts requests we actually sent. Where robots.txt denied us we did not send one, so those are excluded from the rate rather than counted as the site refusing.

Who is drawing detours

Of 1000 domains, these publish a machine-readable affordance an agent can follow instead of guessing.

File	Present	Share	Soft 404
`llms_txt`	102	10.2%	110
`agents_md`	6	0.6%	119
`wellknown_agents`	3	0.3%	101
`web_bot_auth`	7	0.7%	102

Soft 404s are sites answering 200 to a path that does not exist — counted, not discarded, because a site that says yes to everything is its own kind of unreachable.

What stopped us

Challenge	Responses
`cloudflare-challenge`	146
`akamai`	80
`recaptcha`	28
`datadome`	22
`hcaptcha`	18
`perimeterx`	15
`imperva-incapsula`	10
`aws-waf`	10

15 responses asked for money outright (HTTP 402).

What this is not

One capture is a photograph, not a trend. One capture is a photograph, not a trend. No before/after claim is published until a second comparable capture exists.
We do not impersonate. GPTBot and ClaudeBot rows are disclosed simulations, not authenticated vendor traffic; see METHODOLOGY.md at the tag target.
The instrument moved. This capture was taken from a residential vantage with robots-unavailable failing open. Automated captures run from GitHub-hosted dynamic egress with robots-unavailable failing closed. Those are different instruments and the repository withholds cross-capture deltas until every comparability dimension matches.

Provenance

Capture: 2026-08-22T0815Z (pre-2026-09-15-baseline)
Observed: 2026-08-22T08:28:13.633Z → 2026-08-22T08:40:54.831Z
Input: Tranco list 74V8X, 1000 domains
Rows: 10000
SHA-256: 6d6fe5b62eb7bd4c297fa6428653efa4b01d7abbf92792b7eee6a2cb813f4ca6
Bytes: published releases

Recompute these numbers yourself: download the capture from the release, then node tools/aggregate.mjs <capture.jsonl>.