Copilot Vision just launched on Windows — here's what it actually does

Tom's Guide

21 hours ago

Microsoft just flipped the switch on one of its most ambitious Copilot features yet. Copilot Vision with Highlights is now rolling out to Windows 11 users in the U.S.
The new tool allows Copilot to 'see' what's on your screen and provide contextual help — a move that puts it in direct competition with Google's Gemini Live and Apple's upcoming Apple Intelligence.
Essentially, it's Microsoft's answer to the next generation of AI assistants: ones that are proactive, ambient and deeply integrated into your device.
At its core, Copilot Vision gives the AI access to "see" whatever you're currently doing on your PC. Whether you're browsing, editing a document, watching a video or working in Excel, and allows it to offer help based on that screen content.
For example, you can ask questions like:
Get instant access to breaking news, the hottest reviews, great deals and helpful tips.
Copilot can now view the apps and windows on your screen (with permission), making the AI smarter and more responsive in real time.
Highlights is a companion feature that automatically surfaces useful content from your apps, browser and documents.
Think of it like an AI assistant that notices what you've been working on and suggests relevant files, reminders or actions; but no prompt is necessary.
Highlights appear in a refreshed Copilot interface, which now docks to the side of your screen for quick access.
These features are now available to U.S. users running Windows 11 version 23H2 with Copilot+ PCs, or select devices that meet the hardware requirements. You'll need to have screen reading enabled in the Copilot settings. Note, Vision only activates when you give it permission.
You can try it today by opening Copilot from the taskbar and clicking the new Vision icon in the corner.
A pop-up will confirm screen access and let you toggle Highlights on or off. While you're at it, try the prompts in the video to help get you started.
Microsoft's move signals a major shift towards staying competitive, giving their AI assistant more capabilities, similar to those of rivals like Google's Gemini and OpenAI's ChatGPT. With OpenAI powering Copilot and Meta and Apple launching their own ambient AI tools, we're entering the age of 'AI that sees.' Whether that's helpful or a little creepy may depend on how well it works — and how much you choose to share.