Google Decided Against Offering Publishers Options In AI Search

NDTV

20-05-2025

While using web site data to build a Google Search topped with artificial intelligence-generated answers, an Alphabet Inc. executive acknowledged in an internal document that there was an alternative way to do things: They could ask web publishers for permission, or let them directly opt out of being included.
But giving publishers a choice would make training AI models in search too complicated, the company concludes in the document, which was unearthed in the company's search antitrust trial. It said Google had a "hard red line" and would require all publishers who wanted their content to show up in the search page to also be used to feed AI features. Instead of giving options, Google decided to "silently update," with "no public announcement" about how they were using publishers' data, according to the document, written by Chetna Bindra, a product management executive at Google Search. "Do what we say, say what we do, but carefully."
Google's dominance in search, which a federal court ruled last year is an illegal monopoly, has given it a decisive advantage in the ongoing AI wars. According to Google's rules - and previous trial testimony from a company vice-president of product - the tech giant may use the content that feeds into its search engine results to develop other search-related AI products. Publishers can only shield their data from search AI if they opt out of search altogether, Google has said.
Site owners that rely on traffic can't afford to skip listing on Google, which still holds more than 90% of the search market, making it a gateway to the modern web. Many have reluctantly let Google use their content to power search AI features, like AI Overviews, which provides AI-generated responses for some queries - despite the fact that the feature often eats into their traffic. By answering questions directly, AI Overviews obviates the need for users to click on links, depriving sites of opportunities to make money by showing ads and selling products.
The Google document displayed in court shows the company recognized from the beginning the possibility of giving publishers more control, said Paul Bannister, the chief strategy officer at Raptive, which represents online creators.
"It's a little bit damning," he said. "It pretty clearly shows that they knew there was a range of options and they pretty much chose the most conservative, most protective of them - the option that didn't give publishers any controls at all."
Google was recently on trial in Washington as a federal judge mulled what steps the tech giant must take to restore competition in online search. Judge Amit Mehta, who presided over the hearings, is now considering a set of remedies proposed by antitrust enforcers aimed at curbing Google's market dominance. The final day of testimony was May 9, with closing arguments set for later this month. A ruling on the proposed remedies is expected in August.
One part of the Justice Department's proposal is compelling Google to give online publishers and creators a way to opt out of having the content of their web pages be used to train Google's generative AI models "on a model-by-model basis," as well as having an opt-out for individual generative AI products "on a product-by-product basis," without penalty.
Among the options discussed in internal company slides, Google listed the possibility of "SGE-only opt-outs" - which would have let publishers opt out of having their content used in some generative AI features in Google Search, without disappearing from the search engine itself. One item under discussion would have allowed publishers to "choose to opt their content out of being displayed within" AI Overviews, though their data "would still be used for training purposes." Another, which Google presented as the most extreme, would have let publishers "opt out of their data being used for grounding" - a process in which Google and other AI companies anchor their models in real-world sources, with the aim of preventing AI from making up information and making its responses more accurate.
Google ultimately chose to give publishers no new options. The presentation advised introducing "no new controls BUT reposition publicly" to point publishers toward an existing opt-out called "no snippet," that allows publishers to be exempt from AI Overviews and other search features. Choosing this option also causes summaries of their website to disappear from the search page, making people unlikely to click on the link.
"Publishers have always controlled how their content is made available to Google as AI models have been built into Search for many years, helping surface relevant sites and driving traffic to them," a Google spokesperson said in a statement in response to questions about the trial exhibit from Bloomberg. "This document is an early-stage list of options in an evolving space and doesn't reflect feasibility or actual decisions." They added that Google continually updates its product documentation for search online.
The document shown in court included recommendations for how company representatives might consider communicating the information, as well as what not to say explicitly. "If aligned, as a next step, we will work on actual language and get this out," Bindra's document, which was written in April 2024, concluded. One month later, at its annual developers conference in Mountain View, California, Google broadly infused search with AI, in what it called a "fully revamped" experience.
In the year since AI Overviews launched, traffic to some publishers' sites has dropped precipitously. Even more significant to publishers in the long run is advancing the development of models that produce something good enough to replace their content, said Brooke Hartley Moy, chief executive officer of Infactory, an AI startup that works with publishers.
"If Google's models get to a point where the human element of content is diminished, then they've kind of signed their own death warrant," Hartley Moy said of publishers.
As publishers search for new revenue streams, allowing their content to be used for retrieval augmented generation, or RAG - a technique in which AI models refer back to specific sources to provide more accurate responses - has emerged as a promising contender, Hartley Moy said. That's what makes Google's move to take RAG off the negotiating table so significant, she said.
"RAG doesn't exist without publishers," Hartley Moy said. "To me, this is a strategy in ensuring that Google has full market power, and the publishers lose one of their key chips in the negotiation."
Under questioning by Google lawyer Kenneth Smurzynski, Liz Reid, the company's head of search, testified that creating multiple opt-outs for different products and models would be challenging.
"That would mean if Search has multiple GenAI features on the page, which it can easily do, each of those would be required to have a separate model powering it. But we don't build separate models for those," Reid said, according to court transcripts of the trial testimony on May 6.
"And so by saying a publisher could be like, 'I want to be in this feature but not that feature,' it doesn't work," she continued. "Because then we would essentially have to say, every single feature on the page needs a different model." This would be very costly not only because of the significant investment in hardware and chips that it would require, Reid said, but also because it would be a challenge to ensure the different AI models operated efficiently and delivered fast responses. "It adds enormous complexity," she testified.