Cloudflare launches tool to help website owners monetize AI bot crawler access
NEW YORK (Reuters) -Cloudflare has launched a tool that blocks bot crawlers from accessing content without permission or compensation to help websites make money from AI firms trying to access and train on their content, the software company said on Tuesday.
The tool allows website owners to choose whether artificial intelligence crawlers can access their material and set a price for access through a "pay per crawl" model, which will help them control how their work is used and compensated, Cloudflare said.
With AI crawlers increasingly collecting content without sending visitors to the original source, website owners are looking to develop additional revenue sources as search traffic referrals that once generated advertising revenue decline.
The initiative is supported by major publishers including Condé Nast and Associated Press, as well as social media companies such as Reddit and Pinterest.
Cloudflare's Chief Strategy Officer Stephanie Cohen said the goal of such tools was to give publishers control over their content, and ensure a sustainable ecosystem for online content creators and AI companies.
"The change in traffic patterns has been rapid, and something needed to change," Cohen said in an interview. "This is just the beginning of a new model for the internet."
Google, for example, has seen its ratio of crawls to visitors referred back to sites drop to 18:1 from 6:1 just six months ago, according to Cloudflare data, suggesting the search giant is maintaining its crawling but decreasing referrals.
The decline could be a result of users finding answers directly within Google's search results, such as AI Overviews. Still, Google's ratio is much higher than other AI companies, such as OpenAI's 1,500:1.
For decades, search engines have indexed content on the internet directing users back to websites, an approach that rewards creators for producing quality content. However, AI companies' crawlers have disrupted this model because they harvest material without sending visitors to the original source and aggregate information through chatbots such as ChatGPT, depriving creators of revenue and recognition.
Many AI companies are circumventing a common web standard used by publishers to block the scraping of their content for use in AI systems, and argue they have broken no laws in accessing content for free.
In response, some publishers, including the New York Times, have sued AI companies for copyright infringement, while others have struck deals to license their content.
Reddit, for example, has sued AI startup Anthropic for allegedly scraping Reddit user comments to train its AI chatbot, while inking a content licensing deal with Google.
(Reporting by Krystal Hu in New York; Editing by Kate Mayberry)
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Sun
2 hours ago
- The Sun
CyCraft Launches XecGuard: LLM Firewall for Trustworthy AI
TAIPEI, TAIWAN - Media OutReach Newswire - 1 July 2025 - CyCraft, a leading AI cybersecurity firm, today announced the global launch of XecGuard, the industry's first plug-and-play LoRA security module purpose-built to defend Large Language Models (LLMs). XecGuard's introduction marks a pivotal moment for secure, trustworthy AI, addressing the critical security challenges posed by the rapid adoption of LLMs. Trustworthy AI Matters The transformative power of Large Language Models (LLMs) brings significant security uncertainty, requiring enterprises to urgently safeguard their AI models from malicious attacks like prompt injection, prompt extraction, and jailbreak attempts. Historically, AI security has been an 'optional add-on' rather than a fundamental feature, leaving valuable AI and data exposed. This oversight can compromise sensitive data, undermine service stability, and erode customer trust. CyCraft emphasizes that 'AI security must be a standard feature—not an optional add-on,' believing it's paramount for delivering stable and trustworthy intelligent services. The Imminent Need for Proactive AI Defense The need for immediate and effective AI security is more critical than ever before. As AI becomes increasingly embedded in core business operations, the attack surface expands exponentially, making proactive defenses an absolute necessity. CyCraft has leveraged its extensive 'battle-tested expertise across critical domains—including government, finance, and high-tech manufacturing' to precisely address these emerging AI-specific threats. The development of XecGuard signifies a shift from 'using AI to tackle cybersecurity challenges' to now 'using AI to protect AI' , ensuring that security and resilience are embedded from day one. 'AI security must be a standard feature—not an optional add-on,' stated Benson Wu, CEO, highlighting XecGuard's resilience and integration of experience from defending critical sectors. Jeremy Chiu, CTO and Co-Founder, emphasized, 'In the past, we used AI to tackle cybersecurity challenges; now, we're using AI to protect AI,' adding that XecGuard enables enterprises to confidently adopt AI and deliver trustworthy services. PK Tsung, CISO, concluded, 'With XecGuard, we're empowering enterprises to embed security and resilience from day one' as part of their vision for the world's most advanced AI security platform. CyCraft's Solution: XecGuard Empowers Secure AI Deployment CyCraft leads with the global launch of XecGuard, the industry's first plug-and-play LoRA security module purpose-built to defend LLMs. XecGuard provides robust protection against prompt injection, prompt extraction, and jailbreak attacks, ensuring enterprise-grade resilience for AI models. Its seamless deployment allows instant integration with any LLM without architectural modification, delivering powerful autonomous defense out of the box. XecGuard is available as a SaaS, an OpenAI-compatible LLM firewall on your cloud (e.g., AWS or Cloudflare Workers AI), or an embedded firewall for on-premises, NVIDIA-powered custom LLM servers. Rigorously validated on major open-source models like Llama 3B, Qwen3 4B, Gemma3 4B, and DeepSeek 8B, it consistently improves security resilience while preserving core performance, enabling even small models to achieve protection comparable to large commercial-grade systems. Real-world validation through collaboration with APMIC, an NVIDIA partner, integrated XecGuard into the F1 open-source model, demonstrating an average 17.3% improvement in overall security defense scores and up to 30.1% in specific attack scenarios via LLM Red Teaming exercises. With XecGuard and the Safety LLM service, CyCraft delivers enterprise-grade AI security, accelerating the adoption of resilient and trustworthy AI across industries, empowering organizations to deploy AI securely, protect sensitive data, and drive innovation with confidence. Even small models gain enterprise-level defenses, approaching large commercial-grade performance.
Malay Mail
4 hours ago
- Malay Mail
Meta's AI talent war raises questions about strategy
NEW YORK, July 2 — Mark Zuckerberg and Meta are spending billions to recruit top artificial intelligence talent, triggering debates about whether the aggressive hiring spree will pay off in the competitive generative AI race. OpenAI CEO Sam Altman recently complained that Meta has offered US$100 million (RM419 million) bonuses to lure engineers away from his company, where they would join teams already earning substantial salaries. Several OpenAI employees have accepted Meta's offers, prompting executives at the ChatGPT maker to scramble to retain their best talent. 'I feel a visceral feeling right now, as if someone has broken into our home and stolen something,' Chief Research Officer Mark Chen wrote in a Saturday Slack memo obtained by Wired magazine. Chen said the company was working 'around the clock to talk to those with offers' and find ways to keep them at OpenAI. Meta's recruitment drive has also landed Scale AI founder and former CEO Alexandr Wang, a Silicon Valley rising star, who will lead a new group called Meta Superintelligence Labs, according to an internal memo, whose content was confirmed by the company. Meta paid more than US$14 billion for a 49 per cent stake in Scale AI in mid-June, bringing Wang aboard as part of the acquisition. Scale AI specialises in labelling data to train AI models for businesses, governments, and research labs. 'As the pace of AI progress accelerates, developing superintelligence is coming into sight,' Zuckerberg wrote in the memo, which was first reported by Bloomberg. 'I believe this will be the beginning of a new era for humanity, and I am fully committed to doing what it takes for Meta to lead the way,' he added. US media outlets report that Meta's recruitment campaign has also targeted OpenAI co-founder Ilya Sutskever, Google rival Perplexity AI, and the buzzy AI video startup Runway. Seeking ways to expand his business empire beyond Facebook and Instagram, Zuckerberg is personally leading the charge, driven by concerns that Meta is falling behind competitors in generative AI. The latest version of Meta's AI model, Llama, ranked below heavyweight rivals in code-writing performance on the LM Arena platform, where users evaluate AI technologies. Meta is integrating new recruits into a dedicated team focused on developing 'superintelligence' — AI that surpasses human cognitive abilities. 'Mercenary' approach Tech blogger Zvi Moshowitz believes Zuckerberg had little choice but to act aggressively, though he expects mixed results from the talent grab. 'There are some extreme downsides to going pure mercenary... and being a company with products no one wants to work on,' Moshowitz told AFP. 'I don't expect it to work, but I suppose Llama will suck less.' While Meta's stock price approaches record highs and the company's valuation nears US$2 trillion, some investors are growing concerned. Institutional investors worry about Meta's cash management and reserves, according to Baird strategist Ted Mortonson. 'Right now, there are no checks and balances' on Zuckerberg's spending decisions, Mortonson noted. Though the potential for AI to enhance Meta's profitable advertising business is appealing, 'people have a real big concern about spending.' Meta executives envision using AI to streamline advertising from creation to targeting, potentially bypassing creative agencies and offering brands a complete solution. The AI talent acquisitions represent long-term investments unlikely to boost Meta's profitability immediately, according to CFRA analyst Angelo Zino. 'But still, you need those people on board now and to invest aggressively to be ready for that phase' of generative AI development. The New York Times reports that Zuckerberg is considering moving away from Meta's Llama model, possibly adopting competing AI systems instead. — AFP
The Star
11 hours ago
- The Star
Premier League forms five-year AI partnership with Microsoft
FILE PHOTO: Microsoft logo and AI Artificial Intelligence words are seen in this illustration taken, May 4, 2023. REUTERS/Dado Ruvic/Illustration/File Photo (Reuters) -The English Premier League and Microsoft on Tuesday announced a five-year partnership where the cloud giant will infuse its artificial intelligence Copilot into the league's digital platforms to provide quick facts and statistics about matches. Audiences and fans will be able to learn about Premier League clubs, players, matches through an AI companion powered by Microsoft's Copilot which can pull information from over 30 seasons of stats, 300,000 articles and 9,000 videos, they said. AI has strongly resonated with sports leagues and sports entertainment companies as they look to streamline the vast troves of data to attract larger audiences and drive engagement. Spain's LaLiga soccer league, which features clubs such as Real Madrid and FC Barcelona, also uses AI in match analysis and media production while clubs roll out AI-driven experiences to engage more fans. The Premier League, England's top soccer league, is also migrating its core digital infrastructure to Microsoft Azure to allow for easier AI integration and create a unified platform for the league. (Reporting by Zaheer Kachwala in Bengaluru; Editing by Shailesh Kuber)
