Android's screen reader can now answer questions about images
Today is Global Accessibility Awareness Day (GAAD), and, as in years past, many tech companies are marking the occasion with the announcement of new assistive features for their ecosystems. Apple got things rolling on Tuesday, and now Google is joining in on the parade. To start, the company has made TalkBack, Android's built-in screen reader, more useful. With the help of one of Google's Gemini models, TalkBack can now answer questions about images displayed on your phone, even they don't have any alt text describing them.
"That means the next time a friend texts you a photo of their new guitar, you can get a description and ask follow-up questions about the make and color, or even what else is in the image," explains Google. The fact Gemini can see and understand the image is thanks to the multi-modal capabilities Google built into the model. Additionally, the Q&A functionality works across the entire screen. So, for example, say you're doing some online shopping, you can first ask your phone to describe the color of the piece of clothing you're interested in and then ask if it's on sale.
Separately, Google is rolling out a new version of its Expressive Captions. First announced at the end of last year, the feature generates subtitles that attempt to capture the emotion of what's being said. For instance, if you're video chatting with some friends and one of them groans after you make a lame joke, your phone will not only subtitle what they said but it will also include "[groaning]" in the transcription. With the new version of Expressive Captions, the resulting subtitles will reflect when someone drags out the sound of their words. That means the next time you're watching a live soccer match and the announcer yells "goallllllll," their excitement will be properly transcribed. Plus, there will be more labels now for sounds like when someone is clearing their throat.
The new version of Expressive Captions is rolling out to English-speaking users in the US, UK, Canada and Australia running Android 15 and above on their phones.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Business Insider
43 minutes ago
- Business Insider
AI leaders have a new term for the fact that their models are not always so intelligent
As academics, independent developers, and the biggest tech companies in the world drive us closer to artificial general intelligence — a still hypothetical form of intelligence that matches human capabilities — they've hit some roadblocks. Many emerging models are prone to hallucinating, misinformation, and simple errors. Google CEO Sundar Pichai referred to this phase of AI as AJI, or "artificial jagged intelligence," on a recent episode of Lex Fridman's podcast. "I don't know who used it first, maybe Karpathy did," Pichai said, referring to deep learning and computer vision specialist Andrej Karpathy, who cofounded OpenAI before leaving last year. AJI is a bit of a metaphor for the trajectory of AI development — jagged, marked at once by sparks of genius and basic mistakes. In a 2024 X post titled "Jagged Intelligence," Karpathy described the term as a "word I came up with to describe the (strange, unintuitive) fact that state of the art LLMs can both perform extremely impressive tasks (e.g. solve complex math problems) while simultaneously struggle with some very dumb problems." He then posted examples of state of the art large language models failing to understand that 9.9 is bigger than 9.11, making "non-sensical decisions" in a game of tic-tac-toe, and struggling to count. The issue is that unlike humans, "where a lot of knowledge and problem-solving capabilities are all highly correlated and improve linearly all together, from birth to adulthood," the jagged edges of AI are not always clear or predictable, Karpathy said. Pichai echoed the idea. "You see what they can do and then you can trivially find they make numerical errors or counting R's in strawberry or something, which seems to trip up most models," Pichai said. "I feel like we are in the AJI phase where dramatic progress, some things don't work well, but overall, you're seeing lots of progress." In 2010, when Google DeepMind launched, its team would talk about a 20-year timeline for AGI, Pichai said. Google subsequently acquired DeepMind in 2014. Pichai thinks it'll take a little longer than that, but by 2030, "I would stress it doesn't matter what that definition is because you will have mind-blowing progress on many dimensions." By then the world will also need a clear system for labeling AI-generated content to "distinguish reality," he said. "Progress" is a vague term, but Pichai has spoken at length about the benefits we'll see from AI development. At the UN's Summit of the Future in September 2024, he outlined four specific ways that AI would advance humanity — improving access to knowledge in native languages, accelerating scientific discovery, mitigating climate disaster, and contributing to economic progress.
Digital Trends
3 hours ago
- Digital Trends
3 big iOS 19 changes that I hope Apple reveals at WWDC 2025
We're less than two days away from Apple's big WWDC 2025 keynote, where the company will reveal new versions of each of its software platforms. One of the biggest changes this year is the expected shift from iOS 19 to iOS 26, with new versions of macOS, iPadOS, tvOS, watchOS, and visionOS also set to follow suit. We're also expecting to see the evolution of Apple Health, including a new AI doctor and Health subscription. iOS 26, if it is to be named that, is expected to introduce one of the biggest evolutions in design for Apple software since the first iPhone was launched. Inspired by visionOS and the Apple Vision Pro, it's expected to be a monumental redesign, but I hope that Apple also takes the time to make a few improvements. Recommended Videos I've used the iPhone for over a decade, which I carry every day along with one of the best Android phones. While iOS is an excellent platform, there are a few key areas where it could improve. Here are the five improvements I'm hoping to see on Monday. 1. True support for third parties It's somewhat antithetical for Apple to embrace third parties, evidenced by its ongoing lawsuit with Fortnite developer Epic Games, but this might be the best time for the company to do exactly this. It may also prevent regulatory challenges, like the Epic Games lawsuit, which resulted in support for third-party app stores on the iPhone, but only in Europe. The success of the iPhone can be largely attributed to its early adoption by third-party developers. These developers built the apps and experiences that enabled the iPhone to have the impact it has had. While this has been great for apps, Apple only enabled third-party solutions to replace certain system functions, and these features don't work anywhere near as well. Third-party keyboards have been supported for over a decade, yet it's still a bit-part installation, which results in the OS often freezing and resorting to the default keyboard. Apple's new Passwords app in iOS 18 also made third-party password managers less stable. This could also help address a significant issue for Apple. The company's troubles with Apple Intelligence and the new Siri could be somewhat solved by enabling third-party assistants to be set as the default, at least in the interim. This would also benefit its partnership with OpenAI and ChatGPT, or allow an assistant like Google Gemini to be set as the default. 2. A rebuilt OS that just works again For Apple to truly allow third-party support, it needs to rebuild iOS. Over the past 19 years, Apple's iPhone software has largely stayed the same under the hood. The result is that while many features have been added over the years, they have directly contributed to the OS's load and instability. Consider iOS 18 and the launch of the theming engine, as well as the more flexible approach to the homescreen layout, and the transformation of widgets. All are great additions to the customizability of iOS 18, but none launched with the same stability and aplomb that made Apple as successful as it has been. Apple products are infamously meant to 'just work', and it's time to return to that era, starting with a rebuild of iOS. The visionOS platform demonstrates that Apple can still design beautiful, first-generation software experiences, and the iPhone needs a next-generation experience. 3. Advanced features from Android There are many features available on Android that aren't available on the iPhone. While iOS 18 brought the iPhone closer to the customizability of an Android phone, there are still a few things that are needed for the iPhone to continue competing with the best phones. Consider the Galaxy S25 Ultra: it features multiple telephoto lenses, the S-Pen stylus, and true multitasking capabilities. Or the Oppo Find N5 and Galaxy Z Fold 7, which feature two screens and advanced stylus input. Then there are phones like the Oppo Find X8 Ultra, Galaxy S25 Ultra, and Vivo X200 Ultra, which all feature multiple telephoto lenses and advanced camera systems that are more capable than the iPhone, at least for still photography. Then there's Apple's approach to the homescreen. Last year, Apple introduced significant improvements to the homescreen, but there are still many ways for Apple to take this further. I'd love to see resizable icons, a choice of grid layouts, and more advanced management of icon positioning and grid. And yes, I hope they fix the current homescreen introduced in iOS 18, so we can put apps wherever we want, and they'll stay in place. The next-generation iPhone experience needs to do more than fix the bugs; it should set the iPhone up to compete with the very best phones. Apple is rumored to launch an iPhone Fold eventually, and iOS 19 should lay the groundwork for this to compete with the best folding phones. I can't wait to see what Apple launches on Monday during the WWDC 2025 livestream. I use an Apple product in each category, and I am genuinely excited to see what the company does next. I suspect we'll see a mix of bug fixes, design changes, and some new features to highlight, but less from Apple Intelligence than we did last year. We'll all find out soon!
New York Times
5 hours ago
- New York Times
Bill Atkinson, Who Made Computers Easier to Use, Is Dead at 74
Bill Atkinson, the Apple Computer designer who created the software that enabled the transformative visual approach pioneered by the company's Lisa and Macintosh computers, making the machines accessible to millions of users without specialized skills, died on Thursday night at his home in Portola Valley, Calif., in the San Francisco Bay Area. He was 74. In a Facebook post, his family said the cause was pancreatic cancer. It was Mr. Atkinson who programmed QuickDraw, a foundational software layer used for both the Lisa and Macintosh computers; composed of a library of small programs, it made it possible to display shapes, text and images on the screen efficiently. The QuickDraw programs were embedded in the computers' hardware, providing a distinctive graphical user interface that presented a simulated 'desktop,' displaying icons of folders, files and application programs. Mr. Atkinson is credited with inventing many of the key aspects of graphical computing, such as 'pull down' menus and the 'double-click' gesture, which allows users to open files, folders and applications by clicking a mouse button twice in succession. Before the Macintosh was introduced in January 1984, most personal computers were text-oriented; graphics were not yet an integrated function of the machines. And computer mice pointing devices were not widely available; software programs were instead controlled by typing arcane commands. The QuickDraw library had originally been designed for Apple's Lisa computer, which was introduced in January 1983. Intended for business users, the Lisa predated many of the Macintosh's easy-to-use features, but priced at $10,000 (almost $33,000 in today's money), it was a commercial failure. Want all of The Times? Subscribe.
