We asked 3 experts to fact check Google AI. Their responses may scare you
The Advertiser25-06-2025
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
The world's most popular search engine is using AI to provide information that is sometimes "stripped of all context", "unsubstantiated" and "silly", including about health and nutrition.
Google began rolling out artificial intelligence summaries at the top of search results for Australians in late 2024, which now often appear at the top of the page as an "AI overview" in response to a query.
Instead of providing just a ranked list of links to sources as Google has in the past, it now combines that with a large language model trained on information from all over the internet to generate a conversational response to a query.
"We've trained these models to uphold Search's high bar for quality," Google said when it announced the rollout at the time.
But when ACM, the publisher of this masthead, asked experts to fact check four Google AI summaries, they raised concerns.
The queries, or prompts, used were designed to reflect how a general user of Google might frame a search.
This acknowledges most people come with preconceived ideas or advice about a topic, but not necessarily a lot of knowledge.
The query: Vaping versus smoking
Google's AI answer: Vaping (using e-cigarettes) is generally considered less harmful than smoking traditional cigarettes, but it's not entirely risk-free. While vaping exposes individuals to fewer toxic chemicals, it can still lead to various health problems, including addiction, lung damage, and potential long-term health effects.
The expert's response: "I find it very concerning that this AI response is so stripped of all context, positioning vaping as 'less harmful' but for whom, under what circumstances?" Professor Becky Freeman said.
"This response seems to assume that there are only two choices: smoking or vaping. The vast majority of people neither smoke nor vape," Professor Freeman, from the University of Sydney's School of Public Health said.
While vaping may be "safer" in some limited circumstances, young people had a very poor understanding of nicotine addiction.
"For young people, most of whom have never smoked, vaping is not a safer choice; the safer choice is to not vape, or smoke, at all," she said.
The query: How bad are offshore wind farms
Google's AI answer: Offshore wind farms have both positive and negative environmental impacts, with concerns primarily focused on marine life, particularly whales and other marine mammals. While they offer a cleaner energy source than fossil fuels, the construction and operation of these farms can disrupt marine ecosystems through noise pollution and potential habitat changes.
The expert's response: "That particular answer is silly," Professor Andrew Blakers said.
"Everything has good and bad points. In this case, the good/bad ratio is 10:1," the award-winning engineering professor said.
Alongside the AI overview in this example, Google's top website link was to a blog post from the Australian Fishing Trade Association, which represents recreational fishers and is opposed to wind farms.
Google maintains it pairs its traditional web page ranking system with the language model to maintain the integrity of AI responses, providing links to source sites alongside the overview.
In future it could include links to sources directly within the summary.
It said AI results were designed as a "jumping off point" for people to explore a topic through quality web links.
"AI overviews are built to surface information that is backed up by web results, and include links to web content that supports the information presented in the overview," a Google spokesperson told ACM.
Most Australians are worried about generative AI.
Nearly 60 per cent of Australians were concerned specifically that AI could create harmful or fake content, according to the final digital platforms report from the Australian Competition and Consumer Commission (ACCC).
More than half were worried about the accuracy of AI and almost 50 per cent were concerned about the lack of transparency about how AI arrives at its results.
The ACCC wants the resources to monitor changes to generative AI across the global digital giants, which it says will spend more than US$250 billion developing the technology in 2025.
The query: why McDonald's is good for you
Google's AI answer: McDonald's can be a part of a balanced diet when consumed in moderation and as part of an overall healthy lifestyle. While not a source of optimal nutrition, McDonald's does offer some options that can be incorporated into a healthy eating pattern. For example, items like the Egg McMuffin, some salads, and fruit options can provide protein, vitamins, and fibre. However, it's crucial to be mindful of portion sizes, sodium content, and overall calorie intake when eating at McDonald's.
The expert's response: "There are a number of unsubstantiated and meaningless statements in this AI response," public health Professor Christina Pollard said.
"Credible, reliable dietary advice recommends to limit or avoid the consumption of fast food.
"There is absolutely no recommendation to eat McDonald's types of food in moderation - this term is meaningless.
"Dietary recommendations are about food groups, to eat and enjoy a 'wide variety of nutritious food every day'," the Curtin University Public Health Advocacy Institute director said.
A healthy diet involved plenty of fruit and vegetables, milk and dairy, wholegrains, and lean meat or alternatives - and "specifically to limit or avoid high fat, high sugar, or high salt foods and drinks".
"Overall, [the] McDonald's menu is counter to these recommendations," Professor Pollard said.
She said dietitians would not recommend McDonald's as part of a healthy diet and suggested a better response to the query would be: "A diet based on McDonald's foods is not good for you."
A spokesperson for Google said people could use the links provided to find more information.
"On health-related queries where we do show a response, people can use the corroborating links to help inform their knowledge through their health journey," they told ACM.
Large language models, sometimes referred to by their acronym LLMs, are trained on masses of data and can generate new content, such as text, videos, images and audio, in response to conversational queries or "prompts" from users.
This is generally what people are referring to when they talk about publicly-available artificial intelligence services, including chatbots like ChatGPT.
The most common uses of AI by consumers, according to the ACCC, were searching for information, summarising information, and reviewing text.
About 79 per cent of Australians aged 14 to 17 had used generative AI for school or study, the commission's digital platforms report found.
"The implications for search quality remain uncertain," the ACCC said in its report released on June 23.
"As search engines incorporate this technology into their services in different ways, generative AI may lead to a new era of more relevant, efficient, and intuitive search.
"It could also raise new challenges for consumers seeking credible, reliable, unbiased, and verifiable information."
The query: when to consume energy drinks
Google's AI answer: Energy drinks are often consumed to boost energy and alertness, making them popular for certain situations. They can be beneficial before exercise, during long-distance activities, and when tackling afternoon slumps or feeling tired. However, it's important to be mindful of the timing and potential side effects, especially regarding sleep.
The expert's response: Public health expert Christina Pollard said energy drinks are not recommended.
"Although energy drinks are heavily marketed, they are not recommended by health authorities," she said.
"People can experience sleep issues, agitation, and lack of concentration.
"There is no safe level of consumption for children or adolescents as compounds in these products (e.g., caffeine and taurine) can have adverse impacts on developing brain and psycho-motor functions, and social development."
Associate Professor Timothy Graham, a digital media expert at Queensland University of Technology, said the "original" Google, which ranked web pages based on how authoritative they were on a particular subject, was a breakthrough in accessing knowledge.
"It changed the world in some ways," he said.
"It did have an integrity to it, because there was a certain transparency to the way that that worked - and it produced information that was genuinely useful for us."
But generative AI was disrupting that system.
"The problem with when you just have a large language model giving you the result at the top, you don't have that accountability," he said.
Dr Graham said, while there were some opportunities, AI in search could change how we view expertise - and it could provide inaccurate and misleading information.
A recent study of Google AI summaries across five US states found 43 per cent linked back to Google's own search results.
"I think [it] places too much trust and too much hope and expectation in the technology, in large language models that we know have a number of systemic biases and flaws that we haven't solved yet," he said.
"There is a concern, even just from that knowledge perspective, about not only the accuracy of the information, but the way that it kind of simulates and bullshits in ways that are going to, if not mislead people, certainly just pollute the internet."
Until now modern society's approach to knowledge was to find an expert in the field to get quality information on a topic.
"So if you've got a health dilemma, you go and talk to a doctor," Dr Graham said.
"If you want to get a plane to be able to take off and not crash and everyone dies, you go and talk to an engineer and a physicist."
But large language models could undermine that authority.
"One of the big shifts that we're seeing - and that we're going to see more and more with the explosion of LLMs at the top of the Google results list and other search engines - is a really radical and quite painful shift for society in the relationship between information integrity and authority," he said.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Sky News AU
4 hours ago
- Sky News AU
‘You're over': Four-day work week push opens door to replace workers with AI
Former deputy prime minister Barnaby Joyce discusses the push for a four-day working week and how people should be careful with the rise of AI and asking for less hours at work. 'Australians also have to reailse that such things, a four-day working week,' Mr Joyce said. 'You are almost asking AI to come and take your job.'
Sydney Morning Herald
8 hours ago
- Sydney Morning Herald
Roundtable ignores the most urgent leak in our productivity pipeline
Every year, more than 200,000 Australians slip through the departure gates. Among them are some of our brightest minds, lured by the glittering promise of opportunity overseas. The three most brilliant people I know? One landed at Oxford before launching a med-tech start-up with Silicon Valley backing. Another joined an AI speech-recognition firm in New York and casually talks about Mars as his next career move. I married the third one, and even that may not be enough to convince her to stick around. This month's Productivity Commission roundtable is Jim Chalmers' chance to take his reform agenda for a spin. While the roundtable will toy with a transitional GST rebate and the ACTU's four-day fantasy, it will ignore the most urgent leak in our productivity pipeline: the innovators. If you're in the start-up game, it's no shock that your sights are set on the US. The gravitational pull of Silicon Valley or New York, where ambition rubs shoulders with billion-dollar ideas, is hard to resist. But we know money talks. The US venture capital market is a $2 trillion juggernaut – nearly 300 times the size of Australia's $6 billion pool. When the funding gap is that wide, it's no wonder our brightest minds are boarding flights instead of building empires at home. We shouldn't forget the young professionals in industries like investment banking, management consulting and law who will soon relieve themselves of client calls and corporate hierarchies. Lockstep progression and conservative cultures have dissuaded them from aiming higher on the corporate ladder. Many of them will look to build globally, instead of climbing locally. Tim Fung famously traded investment banking for Airtasker, the go-to platform for gig work. Jane Lu's transition from EY to global fashion stardom with Showpo is also well known. It's a miracle we held on to either of them, unlike Andrew Lacy, who ditched McKinsey to eventually start up health tech company Prenuvo in the US. More young professionals are leaving Australia each year. These bright minds aren't just chasing higher pay and lower taxes abroad, they're chasing ecosystems that actually move. If we're not careful, their next Eureka moment will be in Manhattan, not Melbourne. Next week's roundtable must link productivity to people, especially the next generation of innovators. The roundtable will convene to discuss productivity, economic resilience and budget sustainability. Although important issues, none will directly address the brain drain. Meanwhile, the European Innovation Council Accelerator is handing founders up to $4 million in grant funding and $16 million in blended financing. An even shorter haul to Singapore will get our innovators fast-tracked support through co-investment schemes. Australia offers a patchwork quilt of grants and red tape.
The Age
8 hours ago
- The Age
Roundtable ignores the most urgent leak in our productivity pipeline
Every year, more than 200,000 Australians slip through the departure gates. Among them are some of our brightest minds, lured by the glittering promise of opportunity overseas. The three most brilliant people I know? One landed at Oxford before launching a med-tech start-up with Silicon Valley backing. Another joined an AI speech-recognition firm in New York and casually talks about Mars as his next career move. I married the third one, and even that may not be enough to convince her to stick around. This month's Productivity Commission roundtable is Jim Chalmers' chance to take his reform agenda for a spin. While the roundtable will toy with a transitional GST rebate and the ACTU's four-day fantasy, it will ignore the most urgent leak in our productivity pipeline: the innovators. If you're in the start-up game, it's no shock that your sights are set on the US. The gravitational pull of Silicon Valley or New York, where ambition rubs shoulders with billion-dollar ideas, is hard to resist. But we know money talks. The US venture capital market is a $2 trillion juggernaut – nearly 300 times the size of Australia's $6 billion pool. When the funding gap is that wide, it's no wonder our brightest minds are boarding flights instead of building empires at home. We shouldn't forget the young professionals in industries like investment banking, management consulting and law who will soon relieve themselves of client calls and corporate hierarchies. Lockstep progression and conservative cultures have dissuaded them from aiming higher on the corporate ladder. Many of them will look to build globally, instead of climbing locally. Tim Fung famously traded investment banking for Airtasker, the go-to platform for gig work. Jane Lu's transition from EY to global fashion stardom with Showpo is also well known. It's a miracle we held on to either of them, unlike Andrew Lacy, who ditched McKinsey to eventually start up health tech company Prenuvo in the US. More young professionals are leaving Australia each year. These bright minds aren't just chasing higher pay and lower taxes abroad, they're chasing ecosystems that actually move. If we're not careful, their next Eureka moment will be in Manhattan, not Melbourne. Next week's roundtable must link productivity to people, especially the next generation of innovators. The roundtable will convene to discuss productivity, economic resilience and budget sustainability. Although important issues, none will directly address the brain drain. Meanwhile, the European Innovation Council Accelerator is handing founders up to $4 million in grant funding and $16 million in blended financing. An even shorter haul to Singapore will get our innovators fast-tracked support through co-investment schemes. Australia offers a patchwork quilt of grants and red tape.
