Cloudflare’s latest AI rankings put a spotlight on Anthropic’s web behavior with a number that is hard to hand-wave away: during the first seven days of May, its bots scraped sites 24,700 times for every one referral. That crawl-to-refer ratio is a proxy for whether AI companies take more from the web than they send back, and in Cloudflare’s data Anthropic remains the biggest outlier.

To understand why this matters, look at how the metric works. Cloudflare tracks AI web crawl requests and referrals from these bots. In plain English, it compares how often an AI company’s crawlers ask to pull pages versus how often the company then sends users back to those pages. For the week of July 1 through 7, Cloudflare’s “crawl-to-refer” data shows Anthropic bots crawled webpages about 2,800 times for every one referral sent back. OpenAI was far behind, followed by Perplexity in third, then Microsoft, and then Google, according to the same dataset.

If you are a founder, investor, or board member watching AI infrastructure turn into AI distribution, this is the uncomfortable part: the web historically ran on a bargain. Sites permitted free crawling in exchange for traffic, and that traffic created the incentive to publish quality content. In the generative AI era, the bargain is weakening because AI answer engines and chatbots increasingly provide direct answers. When users get what they need without visiting the original site, referrals drop, and the financial incentive to invest in content can decline.

Cloudflare’s data suggests the “free rider” dynamic is not evenly distributed. DuckDuckGo is one of the few players that comes close to offering a fair balance, with three scrapes for every one referral, according to Cloudflare data. Meanwhile, the report notes that Anthropic did improve from roughly 8,800-to-1 in early April, but it still scraped “like crazy” in subsequent seven-day windows. The contrast is important because it frames this as more than just one noisy week. Over time, the ratio staying extreme means the imbalance is structural, not a rounding error.

There is also a legitimacy and methodology dispute simmering underneath these rankings. The source notes that Anthropic previously disputed Cloudflare’s methodology, saying it could not verify the company’s calculations and arguing that new search features are increasing referrals. Translation: Anthropic is not accepting the underlying measurement at face value, and it is pointing to product changes that it says affect the referral side of the equation. That makes the story less like a simple scorecard and more like a live debate over attribution. Are crawls translating into traffic in ways Cloudflare can measure? Or are those referrals happening elsewhere, or through different user journeys?

Even with that caveat, the rankings land because they connect to a wider pattern of conflict in the AI supply chain. In recent months, Anthropic has criticized rivals for using its AI model outputs to develop and improve their own models. Anthropic reportedly calls these techniques “distillation attacks” and says this is against its terms of service. Step back, though, and the source draws a parallel: this resembles what Anthropic is doing to websites, collecting content, often against content owners’ wishes, and using it to develop or improve its own products. The point is not that ethics are one-size-fits-all; it is that the same arguments can flip depending on which side of the scrape you are on.

For executives and boards, the strategic stakes are bigger than a single company’s ratio. If AI systems become the primary interface to information, web publishers and traffic-dependent businesses face a sustainability problem. That can push regulators, courts, and negotiating tables toward new rules about scraping, licensing, and attribution. And even outside regulation, the market may force changes in how AI models train, how companies present sources, and how they measure “fair use” in practice. The Cloudflare rankings are essentially an operational warning label: if the crawl-to-refer balance keeps skewing, expect more friction across partnerships, content monetization, and compliance frameworks.

So what should peers take from this? Cloudflare’s tracking suggests an imbalance that persists even after some improvement from early April. Anthropic’s July 1 to July 7 ratio is still 2,800-to-1, and the early May ratio of 24,700-to-1 is the sharpest edge in the dataset. In a world where user attention is the scarcest resource, that edge likely shows up somewhere else too: in trust, in negotiating leverage, and in how quickly the “economic bargain” gets rewritten.