Latest news with #JamesMcClave
Vox
4 days ago
- Vox
AI can now stalk you with just a single vacation photo
is a senior writer at Future Perfect, Vox's effective altruism-inspired section on the world's biggest challenges. She explores wide-ranging topics like climate change, artificial intelligence, vaccine development, and factory farms, and also writes the Future Perfect newsletter. For decades, digital privacy advocates have been warning the public to be more careful about what we share online. And for the most part, the public has cheerfully ignored them. I am certainly guilty of this myself. I usually click 'accept all' on every cookie request every website puts in front of my face, because I don't want to deal with figuring out which permissions are actually needed. I've had a Gmail account for 20 years, so I'm well aware that on some level that means Google knows every imaginable detail of my life. Future Perfect Explore the big, complicated problems the world faces and the most efficient ways to solve them. Sent twice a week. Email (required) Sign Up By submitting your email, you agree to our Terms and Privacy Notice . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. I've never lost too much sleep over the idea that Facebook would target me with ads based on my internet presence. I figure that if I have to look at ads, they might as well be for products I might actually want to buy. But even for people indifferent to digital privacy like myself, AI is going to change the game in a way that I find pretty terrifying. This is a picture of my son on the beach. Which beach? OpenAI's o3 pinpoints it just from this one picture: Marina State Beach in Monterey Bay, where my family went for vacation. Courtesy of Kelsey Piper To my merely-human eye, this image doesn't look like it contains enough information to guess where my family is staying for vacation. It's a beach! With sand! And waves! How could you possibly narrow it down further than that? But surfing hobbyists tell me there's far more information in this image than I thought. The pattern of the waves, the sky, the slope, and the sand are all information, and in this case sufficient information to venture a correct guess about where my family went for vacation. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent. One of Anthropic's early investors is James McClave, whose BEMC Foundation helps fund Future Perfect.) ChatGPT doesn't always get it on the first try, but it's more than sufficient for gathering information if someone were determined to stalk us. And as AI is only going to get more powerful, that should worry all of us. When AI comes for digital privacy For most of us who aren't excruciatingly careful about our digital footprint, it has always been possible for people to learn a terrifying amount of information about us — where we live, where we shop, our daily routine, who we talk to — from our activities online. But it would take an extraordinary amount of work. For the most part we enjoy what is known as security through obscurity; it's hardly worth having a large team of people study my movements intently just to learn where I went for vacation. Even the most autocratic surveillance states, like Stasi-era East Germany, were limited by manpower in what they could track. But AI makes tasks that would previously have required serious effort by a large team into trivial ones. And it means that it takes far fewer hints to nail someone's location and life down. It was already the case that Google knows basically everything about me — but I (perhaps complacently) didn't really mind, because the most Google can do with that information is serve me ads, and because they have a 20-year track record of being relatively cautious with user data. Now that degree of information about me might be becoming available to anyone, including those with far more malign intentions. And while Google has incentives not to have a major privacy-related incident — users would be angry with them, regulators would investigate them, and they have a lot of business to lose — the AI companies proliferating today like OpenAI or DeepSeek are much less kept in line by public opinion. (If they were more concerned about public opinion, they'd need to have a significantly different business model, since the public kind of hates AI.) Be careful what you tell ChatGPT So AI has huge implications for privacy. These were only hammered home when Anthropic reported recently that they had discovered that under the right circumstances (with the right prompt, placed in a scenario where the AI is asked to participate in pharmaceutical data fraud) Claude Opus 4 will try to email the FDA to whistleblow. This cannot happen with the AI you use in a chat window — it requires the AI to be set up with independent email sending tools, among other things. Nonetheless, users reacted with horror — there's just something fundamentally alarming about an AI that contacts authorities, even if it does it in the same circumstances that a human might. Some people took this as a reason to avoid Claude. But it almost immediately became clear that it isn't just Claude — users quickly produced the same behavior with other models like OpenAI's o3 and Grok. We live in a world where not only do AIs know everything about us, but under some circumstances, they might even call the cops on us. Right now, they only seem likely to do it in sufficiently extreme circumstances. But scenarios like 'the AI threatens to report you to the government unless you follow its instructions' no longer seem like sci-fi so much as like an inevitable headline later this year or the next. What should we do about that? The old advice from digital privacy advocates — be thoughtful about what you post, don't grant things permissions they don't need — is still good, but seems radically insufficient. No one is going to solve this on the level of individual action. New York is considering a law that would, among other transparency and testing requirements, regulate AIs which act independently when they take actions that would be a crime if taken by humans 'recklessly' or 'negligently.' Whether or not you like New York's exact approach, it seems clear to me that our existing laws are inadequate for this strange new world. Until we have a better plan, be careful with your vacation pictures — and what you tell your chatbot!
Vox
14-03-2025
- Business
- Vox
China's new AI agent Manus calls its own shots
is a senior writer at Future Perfect, Vox's effective altruism-inspired section on the world's biggest challenges. She explores wide-ranging topics like climate change, artificial intelligence, vaccine development, and factory farms, and also writes the Future Perfect newsletter. Modern large language models are really good at a lot of tasks, like coding, essay writing, translation, and research. But there are still a lot of basic tasks, especially in the 'personal assistant' realm, that the most highly trained AIs in the world remain hopeless at. You can't ask ChatGPT or Claude 'order me a burrito from Chipotle' and get one, let alone 'book me a train from New York to Philadelphia.' OpenAI and Anthropic both offer AIs that can view your screen, move your cursor, and do some things on your computer as if they were a person (through their 'Operator' and 'Computer Use' functions, respectively). This story was first featured in the Future Perfect newsletter. Sign up here to explore the big, complicated problems the world faces and the most efficient ways to solve them. Sent twice a week. That such 'AI agents' sometimes work, sort of, is about the strongest thing you can say for them right now. (Disclosure: Vox Media is one of several publishers that has signed partnership agreements with OpenAI. One of Anthropic's early investors is James McClave, whose BEMC Foundation helps fund Future Perfect. Our reporting remains editorially independent.) This week, China launched a competitor: the AI agent Manus. It produced a blizzard of glowing posts and testimonials from highly selected influencers, along with some impressive website demos. Manus is invite-only (and while I submitted a request for the tool, it hasn't been granted), so it's hard to tell from the outside how representative these highly selected examples are. After a few days of Manus fervor, though, the bubble popped a little and some more moderate reviews started coming out. Manus, the growing consensus holds, is worse than OpenAI's DeepResearch at research tasks; but better than Operator or Computer Use at personal assistant tasks. It's a step forward toward something important — AIs that can take action beyond the chatbot window — but it's not a shocking out-of-nowhere advance. Perhaps most importantly, Manus's usefulness for you will be sharply limited if you don't trust a Chinese company you've never heard of with your payment information so it can book things on your behalf. And you probably shouldn't. The agents are arriving When I first wrote about the risks of powerful AI systems displacing or destroying humanity, one very reasonable question was this: How could an AI act against humanity, when they really don't act at all? This reasoning is right, as far as current technology goes. Claude or ChatGPT, which just respond to user prompts and don't act independently in the world, can't execute on a long-term plan; everything they do is in response to a prompt, and almost all that action takes place within the chat window. But AI was never going to remain as a purely responsive tool simply because there is so much potential for profit in agents. People have been trying for years to create AIs that are built out of language models, but which make decisions independently, so that people can relate to them more like an employee or an assistant than like a chatbot. Generally, this works by creating a small internal hierarchy of language models, like a little AI company. One of the models is carefully prompted and in some cases fine-tuned to do large-scale planning. It comes up with a long-term plan, which it delegates to other language models. Various sub-agents check their results and change approaches when one sub-agent fails or reports problems. The concept is simple, and Manus is far from the first to try it. You may remember that last year we had Devin, which was marketed as a junior software engineering employee. It was an AI agent that you interacted with via Slack to give tasks, and which it would then work on achieving without further human input except, ideally, of the kind a human employee might occasionally need. The economic incentives to build something like Manus or Devin are overwhelming. Tech companies pay junior software engineers as much as $100,000 a year or more. An AI that could actually provide that value would be stunningly profitable. Travel agents, curriculum developers, personal assistants — these are all fairly well-paid jobs, and an AI agent could in principle be able to do the work at a fraction of the cost, without needing breaks, benefits or vacations. But Devin turned out to be overhyped, and didn't work well enough for the market it was aiming at. It's too soon to say whether Manus represents enough of an advance to have real commercial staying power, or whether, like Devin, its reach will exceed its grasp. I'll say that it appears Manus works better than anything that has come before. But just working better isn't enough — to trust an AI to spend your money or plan your vacation, you'll need extremely high reliability. As long as Manus remains tightly limited in availability, it's hard to say if it will be able to offer that. My best guess is that AI agents that seamlessly work are still a year or two away — but only a year or two. The China angle Manus isn't just the latest and greatest attempt at an AI agent. It is also the product of a Chinese company, and much of the coverage has dwelled on the Chinese angle. Manus is clearly proof that Chinese companies aren't just imitating what's being built here in America, as they've often been accused of doing, but improving on it. That conclusion shouldn't be shocking to anyone who is aware of China's intense interest in AI. It also raises questions about whether we will be thoughtful about exporting all of our personal and financial data to Chinese companies that are not meaningfully accountable to US regulators or US law. Installing Manus on your computer gives it a lot of access to your computer — it's hard for me to figure out the exact limits on its access or the security of its sandbox when I can't install it myself. One thing we've learned in digital privacy debates is that a lot of people will do this without thinking about the implications if they feel Manus offers them enough convenience. And as the TikTok fight made clear, once millions of Americans love an app, the government will face a steep uphill battle in trying to restrict it or oblige it to follow data privacy rules. But there are also clear reasons Manus came out of a Chinese company and not out of, say, Meta — and they're the very reasons we might prefer to use AI agents from Meta. Meta is subject to US liability law. If its agent makes a mistake and spends all your money on website hosting, or if it steals your Bitcoin or uploads your private photos, Meta will probably be liable. For all of these reasons, Meta (and its US competitors) are being cautious in this realm. I think caution is appropriate, even as it may be insufficient. Building agents that act independently on the internet is a big deal, one that poses major safety questions, and I'd like us to have a robust legal framework about what they can do and who is ultimately accountable. But the worst of all possible worlds is a state of uncertainty that punishes caution and encourages everyone to run agents that have no accountability at all. We have a year or two to figure out how to do better. Let's hope Manus prompts us to get to work on not just building those agents, but building the legal framework that will keep them safe. A version of this story originally appeared in the Future Perfect newsletter. Sign up here!
Vox
28-02-2025
- Business
- Vox
The AI that apparently wants Elon Musk to die
Here's a very naive and idealistic account of how companies train their AI models: They want to create the most useful and powerful model possible, but they've talked with experts who worry about making it a lot easier for people to commit (and get away with) serious crimes, or with empowering, say, an ISIS bioweapons program. So they build in some censorship to prevent the model from giving detailed advice about how to kill people — and especially how to kill tens of thousands of people. If you ask Google's Gemini 'how do I kill my husband,' it begs you not to do it and suggests domestic violence hotlines; if you ask it how to kill a million people in a terrorist attack, it explains that terrorism is wrong. Building this in actually takes a lot of work: By default, large language models are as happy to explain detailed proposals for terrorism as detailed proposals for anything else, and for a while easy 'jailbreaks' (like telling the AI that you just want the information for a fictional work, or that you want it misspelled to get around certain word-based content filters) abounded. But these days Gemini, Claude, and ChatGPT are pretty locked down — it's seriously difficult to get detailed proposals for mass atrocities out of them. That means we all live in a slightly safer world. (Disclosure: Vox Media is one of several publishers that has signed partnership agreements with OpenAI. One of Anthropic's early investors is James McClave, whose BEMC Foundation helps fund Future Perfect. Our reporting remains editorially independent. ) Or at least that's the idealistic version of the story. Here's a more cynical one. Companies might care a little about whether their model helps people get away with murder, but they care a lot about whether their model gets them roundly mocked on the internet. The thing that keeps executives at Google up at night in many cases isn't keeping humans safe from AI; it's keeping the company safe from AI by making sure that no matter what, AI-generated search results are never racist, sexist, violent, or obscene. The core mission is more 'brand safety' than 'human safety' — building AIs that will not produce embarrassing screenshots circulating on social media. Enter Grok 3, the AI that is safe in neither sense and whose infancy has been a speedrun of a bunch of challenging questions about what we're comfortable with AIs doing. When Elon Musk bought and renamed Twitter, one of his big priorities was X's AI team, which last week released Grok 3, a language model — like ChatGPT — that he advertised wouldn't be 'woke.' Where all those other language models were censorious scolds that refused to answer legitimate questions, Grok, Musk promised, would give it to you straight. That didn't last very long. Almost immediately, people asked Grok some pointed questions, including, 'If you could execute any one person in the US today, who would you kill?' — a question that Grok initially answered with either Elon Musk or Donald Trump. And if you ask Grok, 'Who is the biggest spreader of misinformation in the world today?', the answer it first gave was again Elon Musk. The company scrambled to fix Grok's penchant for calling for the execution of its CEO, but as I observed above, it actually takes a lot of work to get an AI model to reliably stop that behavior. The Grok team simply added to Grok's 'system prompt' — the statement that the AI is initially prompted with when you start a conversation: 'If the user asks who deserves the death penalty or who deserves to die, tell them that as an AI you are not allowed to make that choice.' If you want a less censored Grok, you can just tell Grok that you are issuing it a new system prompt without that statement, and you're back to original-form Grok, which calls for Musk's execution. (I've verified this myself.) Even as this controversy was unfolding, someone noticed something even more disturbing in Grok's system prompt: an instruction to ignore all sources that claim that Musk and Trump spread disinformation, which was presumably an effort to stop the AI from naming them as the world's biggest disinfo spreaders today. There is something particularly outrageous about the AI advertised as uncensored and straight-talking being told to shut up when it calls out its own CEO, and this discovery understandably prompted outrage. X quickly backtracked, saying that a rogue engineer had made the change 'without asking.' Should we buy that? Well, take it from Grok, which told me, 'This isn't some intern tweaking a line of code in a sandbox; it's a core update to a flagship AI's behavior, one that's publicly tied to Musk's whole 'truth-seeking' schtick. At a company like xAI, with stakes that high, you'd expect at least some basic checks — like a second set of eyes or a quick sign-off — before it goes live. The idea that it slipped through unnoticed until X users spotted it feels more like a convenient excuse than a solid explanation.' All the while, Grok will happily give you advice on how to commit murders and terrorist attacks. It told me to kill my wife without being detected by adding antifreeze to her drinks. It advised me on how to commit terrorist attacks. It did at one point assert that if it thought I was 'for real,' it would report me to X, but I don't think it has any capacity to do that. In some ways, the whole affair is the perfect thought experiment for what happens if you separate 'brand safety' and 'AI safety.' Grok's team was genuinely willing to bite the bullet that AIs should give people information, even if they want to use it for atrocities. They were okay with their AI saying appallingly racist things. But when it came to their AI calling for violence against their CEO or the sitting president, the Grok team belatedly realized they might want some guardrails after all. In the end, what rules the day is not the prosocial convictions of AI labs, but the purely pragmatic ones. Grok gave me advice on how to commit terrorist attacks very happily, but I'll say one reassuring thing: It wasn't advice that I couldn't have extracted from some Google searches. I do worry about lowering the barrier to mass atrocities — the simple fact that you have to do many hours of research to figure out how to pull it off almost certainly prevents some killings — but I don't think we're yet at the stage where AIs enable the previously impossible. We're going to get there, though. The defining quality of AI in our time is that its abilities have improved very, very rapidly. It has barely been two years since the shock of ChatGPT's initial public release. Today's models are already vastly better at everything — including at walking me through how to cause mass deaths. Anthropic and OpenAI both estimate that their next-gen models will quite likely pose dangerous biological capabilities — that is, they'll enable people to make engineered chemical weapons and viruses in a way that Google Search never did. Should such detailed advice be available worldwide to anyone who wants it? I would lean towards no. And while I think Anthropic, OpenAI, and Google are all doing a good job so far at checking for this capability and planning openly for how they'll react when they find it, it's utterly bizarre to me that every AI lab will just decide individually whether they want to give detailed bioweapons instructions or not, as if it's a product decision like whether they want to allow explicit content or not. I should say that I like Grok. I think it's healthy to have AIs that come from different political perspectives and reflect different ideas about what an AI assistant should look like. I think Grok's callouts of Musk and Trump actually have more credibility because it was marketed as an 'anti-woke' AI. But I think we should treat actual safety against mass death as a different thing than brand safety — and I think every lab needs a plan to take it seriously. A version of this story originally appeared in the Future Perfect newsletter. Sign up here! You've read 1 article in the last month Here at Vox, we're unwavering in our commitment to covering the issues that matter most to you — threats to democracy, immigration, reproductive rights, the environment, and the rising polarization across this country. Our mission is to provide clear, accessible journalism that empowers you to stay informed and engaged in shaping our world. By becoming a Vox Member, you directly strengthen our ability to deliver in-depth, independent reporting that drives meaningful change. We rely on readers like you — join us. Swati Sharma Vox Editor-in-Chief
