Cloudflare enables content owners to charge AI crawlers for access
MAITHEM AL ANBARI (ABU DHABI)Internet company Cloudflare has launched a new 'pay-per-crawl' system, which allows website owners to charge AI crawlers whenever they seek access to their content.Pay-per-crawl will give domain owners complete control over their monetisation strategies, Cloudflare said. They can establish a consistent fee per request for their entire site. By default, this new system will prevent artificial intelligence crawlers from accessing content without the consent or financial remuneration of website owners, the company explained.Cloudflare mentioned that the request headers will indicate whether access is granted or if a payment is required, along with the associated pricing. The company will act as the Merchant of Record for the pay-per-crawl system and will also provide the necessary technical infrastructure.On average, Cloudflare handles 78 million HTTP requests per second and provides data services for 330 cities across more than 125 countries.
Approximately 16% of worldwide internet traffic is processed directly by Cloudflare's content delivery network, according to the company's estimates.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
TECHx
5 days ago
- TECHx
Cloudflare Reports Record-Breaking DDoS Attacks in Q2 2025
Home » Emerging technologies » Cyber Security » Cloudflare Reports Record-Breaking DDoS Attacks in Q2 2025 Cloudflare, Inc., the security, performance, and reliability company, has announced its Q2 2025 DDoS report. The report revealed key insights and emerging trends across the global DDoS threat landscape. According to Cloudflare, Q2 saw a surge in hyper-volumetric DDoS attacks. The company automatically blocked its largest attacks ever recorded peaking at 7.3 terabits per second (Tbps) and 4.8 billion packets per second (Bpps). In total, Cloudflare mitigated 7.3 million DDoS attacks in Q2. This marked a drop from 20.5 million in Q1, which included a major 18-day attack campaign. Still, Q2 figures were 44% higher compared to the same period in 2024. Cloudflare reported over 6,500 hyper-volumetric DDoS attacks this quarter, averaging 71 per day. These included: L3/4 DDoS attacks exceeding 1 Tbps or 1 Bpps HTTP DDoS attacks exceeding 1 million requests per second (Mrps) While overall attack numbers declined, HTTP DDoS attacks increased 9% to 4.1 million. Layer 3/4 attacks, however, dropped 81% to 3.2 million. The Telecommunications sector was the most targeted industry. Other affected sectors included Internet, IT & Services, Gaming, and Banking. Top targeted countries by billing location were China, Brazil, Germany, India, and South Korea. Vietnam, Russia, and Azerbaijan also saw significant increases. Cloudflare noted that 71% of respondents were unsure of the threat actor. Of the rest: 63% blamed competitors, especially in Gaming, Gambling, and Crypto 21% pointed to state-level actors 5% each cited self-DDoS, extortionists, or disgruntled users Ransom DDoS attacks also rose. The number of affected Cloudflare customers jumped 68% from Q1 and 6% from Q2 2024. Cloudflare revealed that botnets launched 71% of HTTP DDoS attacks. For L3/4, DNS floods remained the top vector, followed by SYN and UDP floods. The report also highlighted emerging threats: Teeworlds flood attacks rose 385% RIPv1 floods increased 296% RDP floods climbed 173% Demon Bot and VxWorks floods also surged Most DDoS attacks remained small. About 94% of L3/4 attacks were below 500 Mbps, and 85% stayed under 50,000 packets per second. However, hyper-volumetric attacks are rising. Six out of every 100 HTTP DDoS attacks exceeded 1 million requests per second. Bashar Bashaireh, AVP Middle East, Türkiye & North Africa at Cloudflare, commented, 'The Q2 data highlights how quickly the DDoS threat landscape is evolving. Attackers are launching faster, shorter, and more aggressive campaigns.' Cloudflare emphasized the importance of always-on protection. The company remains focused on delivering unmetered, automated DDoS protection that scales with modern threats.
TECHx
28-07-2025
- TECHx
Cloudflare Reveals Q2 2025 Global Internet and Cyber Threat
Home » Emerging technologies » Cyber Security » Cloudflare Reveals Q2 2025 Global Internet and Cyber Threat Trends Cloudflare has announced its Q2 2025 Global Internet Trends and Insights report, revealing important shifts in web traffic and cybersecurity patterns worldwide. The company reported blocking an average of 190 billion cyber threats daily, marking a 21% increase year-over-year (YoY). Despite this, threat volume dropped 23% quarter-over-quarter (QoQ) following a surge earlier in the year. Additionally, Cloudflare handled 5.8 trillion daily internet requests globally, showing a 9% increase QoQ and a 40% rise YoY. Regionally, North America blocked 41.5 billion daily threats, up 18% YoY but down 12% QoQ. Europe, Middle East & Africa (EMEA) saw 62.7 billion threats blocked daily, up 22% YoY and down 16% QoQ. Asia-Pacific (APAC) recorded 57.4 billion daily blocks, up 24% YoY and down 29% QoQ. Latin America (LATAM) blocked 19 billion threats daily, with a 15% YoY rise but a 36% drop QoQ. In the Kingdom of Saudi Arabia (KSA), internet traffic remained steady from Q1 to Q2 2025. However, mitigated cyber threats decreased by 6% QoQ. The region faced 294 million cyber threats blocked daily, showing a significant 76% decrease QoQ, following a major attack spike in Q1. Key targeted sectors included Internet, Consumer Goods, IT Services, and Airlines. Meanwhile, the United Arab Emirates (UAE) experienced a 15% increase in internet traffic and a 17% rise in mitigated threats QoQ. The UAE blocked 1 billion cyber threats daily, a 63% increase QoQ. Top targeted industries were Retail, Consumer Services, Banking, and Information Services. Cloudflare also detailed defenses against application layer attacks. In KSA, Web Application Firewall (WAF) rules accounted for 92% of defenses, with HTTP Anomaly, Cross-site Scripting (XSS), and Directory Traversal as the most active rules. In the UAE, DDoS mitigations and WAF rules were nearly balanced at 50% and 49%, respectively, with HTTP Anomaly, SQL Injection (SQLi), and XSS as leading categories. Bashar Bashaireh, AVP Middle East, Türkiye & North Africa at Cloudflare, stated that the data highlights the dual reality of rapid internet growth alongside increasingly sophisticated cyber threats. He emphasized Cloudflare's commitment to providing a secure, fast, and reliable internet experience worldwide. Key points: Cloudflare blocked 190 billion daily cyber threats in Q2 2025, up 21% YoY. Internet traffic and cyber threats varied regionally, with notable increases in UAE. Application layer defenses focused on WAF rules and DDoS mitigations. This report underscores the evolving landscape of internet usage and cybersecurity challenges globally and regionally.
The National
07-07-2025
- The National
AI audits and 'pay per crawl': How Cloudflare is trying to fix a 'broken' web model
An announcement from network giant Cloudflare is deepening a divide between the worlds of tech and content publishing, which are at odds over the data used to train AI platforms. Cloudflare said publishers using the cloud company's tools for hosting websites will now block AI crawlers by default from accessing and poaching content without permission. 'Upon sign-up with Cloudflare, every new domain will now be asked if they want to allow AI crawlers, giving customers the choice upfront to explicitly allow or deny AI crawlers access,' the company said. 'This significant shift means that every new domain starts with the default of control, and eliminates the need for webpage owners to manually configure their settings to opt out.' Cloudflare first addressed concerns about AI data-scraping last year when it gave websites the option to block AI companies from poaching content. 'Now by default you have control over who crawls your site and what that information is used for,' said Stephanie Cohen, a chief strategy officer for Cloudflare. 'The benefit of that is that it creates the conditions for a new business model of the internet to develop,' she told The National after Cloudflare introduced the new settings and options. Some of the strongest proponents of AI tools, and many of the tools' creators, have justified data scraping, saying it is akin to the early days of search engines when controversy briefly surfaced over whether or not search companies should be able to index sites. Others say that comparison isn't appropriate, because search engines didn't poach the contents of entire websites. Additionally, during the early days of web browsers, search engines and the crawlers they implemented provided a framework that built much of the internet as we know it. It was a win-win situation for the likes of Google and media companies which provided information and sought to attract audiences by delivering web traffic through internet searches. The debut of OpenAI's ChatGPT in 2022 and other AI platforms turned that economic model on its head. Instead of directing traffic to websites, AI summaries have quickly become a destination unto themselves, siphoning traffic from the same websites from which they scrape data. Ms Cohen said publishers and content creators using Cloudflare's services soon noticed a major dip in web traffic. 'Not only was it getting more difficult to get web traffic – to the tune of it being 10 times harder – but it was also getting more difficult at a faster and faster rate,' she said. The web's economics based on search that built up over the last decade, she said, started to erode over a period of six months. In 2024, Ms Cohen said Cloudflare allowed users to see which AI companies were scraping their sites and turn off that ability. This year they are taking things further by introducing 'pay per crawl'. The tool gives publishers and website operators the option of allowing AI scraping for free, charging for it 'at the configured domain-wide price,' or blocking scraping entirely. As AI developments quickens, so too does the bad blood between media organisations and the tech firms driving the AI boom. Several lawsuits have been filed. The New York Times has sued OpenAI and Microsoft for allegedly using its articles to power increasingly popular chatbots.
