AI search biz Perplexity claims that Cloudflare has mischaracterized its site crawlers as malicious bots and that the content delivery network made technical errors in its analysis of Perplexity’s operations.
“This controversy reveals that Cloudflare’s systems are fundamentally inadequate for distinguishing between legitimate AI assistants and actual threats,” Perplexity said in a social media post published on Monday afternoon. “If you can’t tell a helpful digital assistant from a malicious scraper, then you probably shouldn’t be making decisions about what constitutes legitimate web traffic.”
The dispute started earlier on Monday, when Cloudflare published a post claiming that Perplexity has been disguising its web crawling bots by altering the user-agent identifier and by using unexpected IP address ranges to evade web application firewall blocking. It did not suggest the bots were malicious, merely that they were obfuscating their identity to avoid being blocked.
Cloudflare CEO Matthew Prince didn’t use the term “malicious” either but came closer.
“Some supposedly ‘reputable’ AI companies act more like North Korean hackers,” Prince said on social media in reference to his company’s post. “Time to name, shame, and hard block them.”
Cloudflare, which provides network infrastructure services like hosting and security, argues that web publishers should be able to control the mechanisms used to access their content.
The issue is that automated page visits, whether from Perplexity or another AI service, don’t generate ad impressions and revenue for publishers (assuming ad fraud systems function properly) .
The rise of AI crawlers that answer search queries by summarizing content culled from websites without compensation has led to a decline in search engine traffic referrals to websites and has thrown the web’s dominant business model into question.
Perplexity’s position is that because a human user entered a search query, the Perplexity bot fetching that information from a publisher’s website should be treated as a human visitor rather than an automaton.
To support that claim, Perplexity argues that, when Google’s search engine crawls a page to include in its web index, that’s different from when Google Search fetches a webpage and presents a preview of the content.
“When companies like Cloudflare mischaracterize user-driven AI assistants as malicious bots, they’re arguing that any automated tool serving users should be suspect – a position that would criminalize email clients and web browsers, or any other service a would-be gatekeeper decided they don’t like,” Perplexity said.
That’s a disingenuous comparison, however. A search engine doesn’t intend for a thumbnail image or text snippet to be a complete substitute for visiting the web page. If Perplexity’s response answers a search query and obviates the need to visit the source webpage to obtain that answer, that’s clearly a different scenario.
This is not the first time Perplexity has come under fire for bot behavior. In early June last year, Forbes accused the company of ripping off news content, an allegation subsequently investigated by AWS for terms of service compliance. Shortly thereafter, blogger and podcaster Robb Knight accused Perplexity of lying about its user agent, an allegation supported by a report from Wired.
Kingsley Uyi Idehen, founder and CEO of OpenLink Software, an AI-oriented middleware business, pushed back against Perplexity’s claims.
Citing the company’s contention that its bot is acting to fulfill a human user’s query, he said in response, “This lack of clarity around who is acting and on whose behalf isn’t a technical footnote – it’s a foundational gap at the center of growing concerns about LLM-based tools and AI agents.”
Identity, authenticity, and accountability continue to be important for online interaction, Idehen argues. So if Perplexity is obfuscating its bots, as Cloudflare has claimed, that’s a problem.
Craig DeWitt, founder of SkyFire, a payment network for AI-based services, told The Register in an interview that both Perplexity and Cloudflare are right in their own way, though he disagreed with Prince comparing Perplexity to North Korea.
“What Perplexity is right about is that the nature of the internet has changed with these AI interfaces,” DeWitt said, citing similar observations from Vercel CEO Guillermo Rauch that the internet must adapt to AI.
“The problem now is like it’s no longer human directly to website, it’s human through an intermediary to website,” DeWitt said. “And the problem for the websites – this is talking on Cloudflare’s side, now – is that the monetization model for a lot of these, in terms of servicing ads, goes away.”
Websites, he said, do not want to deliver their content without attribution through AI service interfaces. But, he said, collaboration is the key because if websites start blocking everything, that hurts everyone.
Cloudflare and Perplexity did not respond to requests for comment. ®