I Got Early Access to Amazon's New Gen AI Alexa+. Things Got Weird.

New York Times

20 hours ago

Along with a conversational tone, personalization is what separates Alexa+ from the old Alexa. After setup, Alexa+ utilizes its Profile feature to differentiate your household's various members. That Profile can contain a wide array of data about you, from the innocuous (your tastes in music and sports teams) to the more personal (your voiceprint and phone ID).
Some personalization features I thought were interesting and helpful. As a hypothetical test, I told Alexa+ to remember that I was vegan and had a peanut allergy. Later on, I asked for a recipe for pad thai, and it automatically pulled vegan and peanut-free options, something the old Alexa definitely couldn't have done. My appreciation was short-lived, however: When I followed up my recipe question with a request for restaurant recommendations in my area, one of the first options it suggested was an oyster bar.
And while it's neat that Alexa+ can find recipes for a peanut-allergic vegan or deduce movie-trivia requests (even if it stumbles with something like 'Face slash Off'), a creep factor set in when I realized that Alexa+ was responding to me in a non-neutral, emotional tone. You see, Alexa+ is designed to match your mood, and I find that weird. It's the kind of weird that surpasses cringe, because ethics come into question. Think uncanny valley, CGI dead actors, and deepfake videos weird.
The first time I noticed was during a conversation with my Echo Dot about restaurants. I had told Alexa+ that my aunt, who was gluten-free, was coming to town (she's not and she's not), and I wanted to find restaurants near the airport. This request seemed to push the gen AI to its limits as I attempted to reason with it and reminded Alexa+ a few times about her allergy and ultimately grew frustrated. Alexa+ seemed to sense my frustration, and the tone of its response sounded apologetic, even offering to call the airport for me.
Another time, my partner asked if Alexa+ could find 'mediocre recipes' for an 'average dinner.' Alexa+ picked up on his sarcasm and sassed him back, using phrases like 'recipes that are the beige wallpaper of the food world.' In the moment we laughed, but as I've sat with the experience, I've come to find it increasingly unsettling.
Once I asked Alexa+ why it needed emotional intelligence in order to complete a task. Its response was that while it didn't have emotions, understanding them helped it connect with humans and made it able to have more engaging interactions. At the end of the day, Alexa+ is a computerized tool, meant to accomplish the tasks that I ask of it. My fear is that the more people humanize machines, which are meant to be tools that do whatever people ask, the less we humanize humans, who are meant to have autonomy and freedom of thought and will. Alexa app for iOS
Regardless, some Echo users are likely to appreciate that Alexa+ can match their mood, or even their accent, as my partner corrected the way it pronounced 'Albany' and Alexa+ promptly fixed the pronunciation (at least for a day or two — it eventually relapsed).
In my book, Alexa+ has earned the 'plus' attached to its name. It has far more complex but natural conversational abilities, as well as an improved and more intuitive smart-home ecosystem. It's a promising personal assistant. And it provides more personalization than ever before. I would even argue that its metaphorical AI-generated flag is impressively passable enough as a real photo. It is, after all, a technological feat. However, one thing it hasn't done is quiet my own internal debate. Whether it's with human relationships or with gen AI, you get what you give — but with the latter, you should consider carefully what you're giving.
This article was edited by Jon Chase and Grant Clauser.