Apple plots expansion into AI robots, home security and smart displays

Business Times

3 days ago

[SAN FRANCISCO] Apple is plotting its artificial intelligence (AI) comeback with an ambitious slate of new devices, including robots, a lifelike version of Siri, a smart speaker with a display and home-security cameras.
A tabletop robot that serves as a virtual companion, targeted for 2027, is the centrepiece of the AI strategy, according to sources with knowledge of the matter. The smart speaker with a display, meanwhile, is slated to arrive next year, part of a push into entry-level smart-home products.
Home security is seen as another big growth opportunity. New cameras will anchor an Apple security system that can automate household functions. The approach should help make Apple's product ecosystem stickier with consumers, said the sources, who asked not to be identified because the initiatives have not been announced.
It's all part of an effort to restore Apple's mojo. Its most recent moon-shot project, the Vision Pro headset, remains a sales flop, and the design of its bestselling devices has remained largely unchanged for years.
At the same time, Apple has come under fire for missing the generative AI revolution. And OpenAI may even threaten the company's home turf by developing new AI-driven devices with the help of former Apple design chief Jony Ive.
Though Apple is still in the early stages of turning around its AI software, executives see the pipeline of hardware as a key piece of its resurgence – helping it challenge Samsung Electronics, Meta Platforms and others in new categories.
BT in your inbox
Start and end each day with the latest news stories and analyses delivered straight to your inbox.
Sign Up
Sign Up
A spokesperson for Cupertino, California-based Apple declined to comment. Because the products have not been announced, the company's plans could still change or be scrapped. Many of the initiatives and their timelines rely on Apple's continued progress in AI-powered software.
Chief executive officer Tim Cook told employees in an all-hands meeting this month that Apple must win in AI and hinted at the upcoming devices. 'The product pipeline, which I can't talk about, it's amazing, guys. It's amazing,' Cook said. 'Some of it you will see soon. Some of it will come later. But there's a lot to see.'
Beyond the home devices, Apple is preparing thinner and redesigned iPhones for release this year. And further out, it aims to introduce smart glasses, a foldable phone, a 20-year anniversary iPhone and a revamped headset dubbed N100. It's also planning a large foldable device that melds a MacBook and an iPad.
Apple is looking to boost sales after years of slowing growth for its flagship products. It also nixed some expansions into new areas, such as self-driving cars, adding pressure to find other sources of revenue. Moreover, the new initiatives will help rebut the idea that the company is no longer innovating like it used to.
Bloomberg News first reported last year that Apple was moving forward with a tabletop robotics project, code-named J595, and developing a new smart-home strategy. But now a clearer picture is forming of its push into that market, and what it means for its AI ambitions.
Robots
The tabletop robot resembles an iPad mounted on a movable limb that can swivel and reposition itself to follow users in a room. Like a human head, it can turn towards a person who is speaking or summoning it, and even seek to draw the attention of someone not facing it.
The hope is to bring AI to life in ways that other hardware makers have yet to do. Apple imagines customers placing it on a desk or kitchen counter and using it to get work done, consume media and manage their day.
FaceTime calls will also be a key function of the device. During videoconferencing, the display will be able to shift to lock onto people around a room. Apple is testing a feature that turns an iPhone screen into a joystick, letting users move around the robot to show different people or items in a room during video calls.
But the hallmark of the device is an entirely new version of the Siri voice assistant that can inject itself into conversations between multiple people. It will be able to engage with users throughout the day and more easily recall information.
The idea is for the device to act like a person in a room. It could interrupt a conversation between friends about dinner plans, say, and suggest nearby restaurants or relevant recipes. It's also being designed to engage in back-and-forth discussions for things such as planning a trip or getting tasks done, similar to OpenAI's voice mode.
Apple is planning to put Siri at the centre of the device operating system and give it a visual personality to make it feel lifelike. The approach, dubbed Bubbles, is vaguely reminiscent of Clippy, an animated paper clip from the 1990s that served as a virtual assistant in Microsoft Office.
Apple has tested making Siri look like an animated version of the Finder logo, the iconic smiley face representing the Mac's file management system. A final decision on its appearance has not been made, with designers considering ideas that veer closer to Memoji, the playful characters that represent Apple user accounts.
Device prototypes use a roughly 7-inch horizontal display, approaching the size of an iPad mini. The motorised arm can extend the display away from the base roughly half a foot in any direction.
Some sources familiar with the product call it the 'Pixar Lamp', referring to the animated film company's famous logo. Apple has previously disclosed some research in this area: It published a paper in January detailing a light fixture that uses robotics to move around.
Apple has multiple teams across its AI, hardware, software and interface design groups tackling on the project. The work is being led in part by Kevin Lynch, who previously oversaw a push into smartwatch software and cars.
The technology giant is developing several other robots. It has teams exploring a mobile bot with wheels, something akin to Amazon.com's Astro, and has loosely discussed humanoid models.
Apple has a group actively developing a large mechanical arm for use in manufacturing facilities or handling tasks in the back of retail stores, a move that could potentially replace some staff. Such a robot, code-named T1333, remains several years away.
Charismatic
The smart-home push includes a standalone display poised to launch by the middle of next year. That device, code-named J490, is a stripped-down variant of the robot, lacking the arm and conversational Siri, at least to start. It will still have home control, music playback, note taking, web browsing and videoconferencing. It may also include the new Siri visual interface.
Both the smart display and tabletop robot will run a new operating system dubbed Charismatic, which is designed to be used by multiple people. The interface largely centres on clock faces and widgets, small software features that are typically dedicated to specific tasks.
Charismatic, which was previously known as Pebble and Rock earlier in development, blends the approach of the Apple TV and Apple Watch operating systems. It offers features such as multiuser modes and clock-face themes, such as one based on Snoopy, the beagle from the Peanuts comic strip.
The devices are meant to be easily shared: They include a front-facing camera that can scan users' faces as they walk towards it and then automatically change the layout, features and content to the preferences of that person.
Some versions of the software use circular app icons and feature a hexagonal grid of apps. Apple is planning to include many of its core apps, including the calendar, camera, music, reminders and notes software. But the interface will be heavily reliant on voice interaction and widgets, rather than jumping in and out of apps.
Though the device will have a touch screen, the primary input method will be Siri and an upcoming upgrade to a feature dubbed App Intents. That software lets users precisely control the interface and applications via voice.
The hardware itself looks similar to a Google Nest Hub but is shaped like a square, with thin black or white bezels and rounded corners.
The non-robotic 7-inch display sits on a half-dome-shaped base, which includes some of the electronics and is perforated around the bottom edges for speakers and microphones. It can also be mounted on a wall.
The launch will mark the first time Apple is making a serious push into the smart home and comes nearly a decade after Amazon and Alphabet's Google started shipping smart speakers with screens.
The home is a critical space for Apple to target, especially as more users consume content from the living room and automate household functions.
Apple has long had a strong foothold in mobile devices and quickly became a player in the automotive industry via CarPlay, but that success has not followed into the smart home. Though the company launched HomeKit for controlling third-party devices in 2014, it has had limited success with its own HomePod speakers.
Linwood and Glenwood
Core to the new home devices, and current products such as iPhones and iPads, is an overhaul to the underpinnings of Siri.
Engineers are working on a version code-named Linwood with an entirely new brain built around large language models, the foundation of generative AI. The goal is to tap into personal data to fulfil queries, an ability that was delayed due to hiccups with the current version.
That new software, known internally as LLM Siri, is planned for release as early as next spring, Bloomberg News has reported. But work is going even further: Apple is preparing a visually redesigned assistant for iPhones and iPads that will also debut as early as next year.
Craig Federighi, senior vice-president of software engineering, hinted at a bigger-than-anticipated overhaul in an internal meeting with employees this month.
'The work we've done on this end-to-end revamp of Siri has given us the results we needed,' he said, adding that 'this has put us in a position to not just deliver what we announced, but to deliver a much bigger upgrade than we envisioned.' He said that 'there is no project people are taking more seriously'.
Linwood is based on technology developed by the Apple Foundation Models team, but the company has a competing project dubbed Glenwood as well that would power Siri with outside technology.
A final decision has not been made on which models will be used, but Apple has been testing Anthropic PBC's Claude for this purpose. Mike Rockwell, the former Vision Pro chief who was put in charge of Siri earlier this year, is overseeing both the Linwood and Glenwood efforts.
During the development of the tabletop robot, Apple engineers have made heavy use of ChatGPT and Google Gemini to build and test features. Within Apple's AI and Siri teams as a whole, software developers are increasingly using third-party systems as part of their development process.
Ring competitor
Apple is working on a camera, code-named J450, designed for home security, detecting people and automating tasks. The device will be battery-powered and could last from several months to a year on a single charge, on par with rival products.
The device has facial recognition and infrared sensors to determine who is in a room. Apple believes users will place cameras throughout their home to help with automation. That could mean turning lights off when someone leaves a room or automatically playing music liked by a particular family member.
Apple is planning to develop multiple types of cameras and home-security products as part of an entirely new hardware and software lineup. The goal is to compete with Amazon Ring, Google Nest and Roku, capitalising on its customer loyalty to sell more products.
It has also tested a doorbell that uses facial recognition technology to unlock a door. Apple already sells iCloud+ subscriptions with online storage for security footage, but they are aimed at third-party cameras. BLOOMBERG