Claude Code with LSP: from searching text to understanding code
•
Artificial Intelligence
Tag: Cloudflare
2 entries found
Cloudflare Innovates Again: Cryptographic Authentication for Bots (The End of Fake User-Agents)
7 min read
The Problem We All Know (But Pretend Doesn’t Exist)
As someone who has managed web infrastructure for decades, I’ve seen too many bots pretending to be what they’re not. User-Agent: “Mozilla/5.0 (Windows NT 10.0; Win64; x64)” when in reality it’s a Chinese scraper sucking up your content at 3 in the morning.
Traditional methods for identifying legitimate bots have been, let’s be honest, pathetic:
- IP ranges: That change, are shared, or are spoofed
- User-Agent headers: That anyone can forge in one line of code
- Reverse DNS: That works until it doesn’t
Today Cloudflare announced that it’s integrating HTTP Message Signatures directly into its Verified Bots program. And after reading the technical details, I have to admit something: this could definitively change the rules of the game.
Cloudflare Just Changed the Game with 'Pay per Crawl' (and it was about time)
5 min read
“Content Independence Day”: the day the web said “enough is enough”
Today, July 1, 2025, Cloudflare officially declared “Content Independence Day” with the launch of “Pay per Crawl”, its new tool that allows website owners to charge AI crawlers for accessing their content. As a DevOps Manager who manages web infrastructures daily, I can say it was about time.
And the numbers don’t lie: while Google maintains a ratio of 18 crawls for every referral it sends (which is already brutal compared to 6:1 from six months ago), OpenAI has a ratio of 1,500:1 and Anthropic reaches 73,000:1. Basically, they’re sucking up all our content without returning even a crumb of traffic.