Microsoft tests Copilot Vision desktop share in Windows 11 Insider Build: Here's what's new
Copilot can answer questions, suggest improvements, and guide you through tasks based on what's on your screen. Microsoft says it may assist with things like editing your resume, offering feedback on creative projects, or helping you navigate a new game. The feature is similar to what Google's Gemini Live and Apple Intelligence are beginning to offer.
Copilot Vision allows the assistance to access and interpret the visible content on your screen in real time. It helps you with what you are working on, reads that content and offers relevant suggestions, summaries or whatever you ask.
For example, suppose you have a PDF document open on your computer screen, you can ask Copilot to summarise it or tell main points of the content. If you are working in an Excel sheet, Copilot might help you spot errors, generate new formulas, help you in solving your issues with quick formulas, and compare values in a table without needing to describe the sheet. When browsing the web, it could highlight product differences or summarise a long review page. All of this happens through screen context, reducing the need to switch apps or copy and paste information.
Importantly, this feature uses screen capture permissions, and Microsoft says Copilot only accesses your screen after you give permission. You also have full control over what it sees and when it stops.
Now these will help you understand better how Copilot Vision helps you in real life. Document review: Copilot can pick out key terms or show what's changed in a contract, so you don't have to scroll through the whole thing.
Spreadsheets: It can look at your sales data and give quick insights, fix errors, or turn numbers into charts right on your screen.
E-commerce: Compare specs of products side by side directly from your screen without searching again.
Email writing: If you have a report or slide deck open, Copilot could help draft a quick summary to send to a colleague.
Troubleshooting: If you are stuck on a system setting or error screen, Copilot could explain the issue or offer a fix based on what's visible.
The new Desktop Share feature makes it possible for Copilot to view your screen in real time and respond to you based on what is on the screen. To enable this feature open the Copilot app and click the glasses icon. From there, choose which part of your desktop to share. Once active, you can ask questions like 'What does this error mean?' or 'How do I create a table in Excel?' and Copilot will respond based on what it sees.
You can even stop sharing any time by selecting the "Stop" option in the same panel. You can also activate the Vision using your voice, thanks to its feature called Voice Conversation. You can verbally say your request and Copilot will respond using visual content on the screen. Copilot listens and responds based on what's on your screen, without needing to type or click.
Microsoft has added a Highlights feature that shows helpful tips based on the apps, files or websites you are using without you having to ask.
Copilot Vision stands out by removing the need to type long prompts. It sees what's on your screen and helps right away, making it feel more like a built-in assistant than a chatbot. Unlike screen sharing or remote tools, it works locally and is just for you.
The feature is still in testing with no set release date, but Microsoft is clearly moving toward assistants that respond to both what you type and what you see on screen.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Economic Times
27 minutes ago
- Economic Times
Meta names former OpenAI researcher Shengjia Zhao as Chief scientist for superintelligence lab
Meta taps former OpenAI researcher Shengjia Zhao as Chief Scientist of its new Superintelligence AI Lab, signaling a bold move in the AGI race. Meta Platforms has appointed Shengjia Zhao, a former OpenAI researcher and co-author of the original ChatGPT paper, as the Chief Scientist of its new Superintelligence AI group, the company confirmed Friday (July 25). Zhao, who joined Meta in June, was a key contributor to OpenAI's early breakthroughs, including the development of ChatGPT and the company's first reasoning model, known as o1. The model helped set off a new wave of 'chain-of-thought' AI systems adopted by companies such as Google and DeepSeek. Meta CEO Mark Zuckerberg announced Zhao's formal appointment in a post on Threads, calling him 'our lead scientist from day one.' Zuckerberg added: 'Now that our recruiting is going well and our team is coming together, we have decided to formalize his leadership role.' Zhao will report to Alexandr Wang, the former CEO of Scale AI, who joined Meta in June as Chief AI Officer. Wang is leading Meta's efforts in building artificial general intelligence (AGI), AI that can think and reason at or beyond human capability. The move comes amid Meta's aggressive recruitment campaign in the AI sector. Over the past two months, the company has hired more than a dozen researchers from OpenAI, Apple, Google, and Anthropic. That includes two top Apple AI scientists, Tom Gunter and Mark Lee, according to Bloomberg. Meta launched the Superintelligence Lab in June 2025 as part of its renewed focus on developing advanced AI models. The lab is separate from Meta's long-standing AI research group FAIR, which will continue to be led by Yann LeCun, who now reports to Wang. Meta's recent hires have drawn attention for the size of the compensation packages involved. Some reports suggested offers exceeding $100 million, although the company has denied rumors of higher figures, including claims of $300 million current open-source model, LLaMA 4, has not yet matched the capabilities of OpenAI's GPT-4 or Google's Gemini models. The company is expected to release a more advanced model, internally codenamed 'Behemoth,' later this expressed optimism about the lab's future, saying, 'Together we are building an elite, talent-dense team that has the resources and long-term focus to push the frontiers of superintelligence research.'
Mint
27 minutes ago
- Mint
Meta Clashes With Apple, Google Over Age Check Legislation
(Bloomberg) -- The biggest tech companies are warring over who's responsible for children's safety online, with billions of dollars in fines on the line as states rapidly pass conflicting laws requiring companies to verify users' ages. The struggle has pitted Meta Platforms Inc. and other app developers against Apple Inc. and Alphabet Inc.'s Google, the world's largest app stores. Lobbyists for both sides are moving from state to state, working to water down or redirect the legislation to minimize their clients' risks. This year alone, at least three states — Utah, Texas and Louisiana — passed legislation requiring tech companies to authenticate users' ages, secure parental consent for anyone under 18 and ensure minors are protected from potentially harmful digital experiences. Now, lobbyists for all three companies are flooding into South Carolina and Ohio, the next possible states to consider such legislation. The debate has taken on new importance after the Supreme Court this summer ruled age verification laws are constitutional in some instances. A tech group on Wednesday petitioned the Supreme Court to block a social media age verification law in Mississippi, teeing up a highly consequential decision in the next few weeks. Child advocates say holding tech companies responsible for verifying the ages of their users is key to creating a safer online experience for minors. Parents and advocates have alleged the social media platforms funnel children into unsafe and toxic online spaces, exposing young people to harmful content about self harm, eating disorders, drug abuse and more. Meta supporters argue the app stores should be responsible for figuring out whether minors are accessing inappropriate content, comparing the app store to a liquor store that checks patrons' IDs. Apple and Google, meanwhile, argue age verification laws violate children's privacy and argue the individual apps are better-positioned to do age checks. Apple said it's more accurate to describe the app store as a mall and Meta as the liquor store. The three new state laws put the responsibility on app stores, signaling Meta's arguments are gaining traction. The company lobbied in support of the Utah and Louisiana laws putting the onus on Apple and Google for tracking their users' ages. Similar Meta-backed proposals have been introduced in 20 states. Federal legislation proposed by Republican Senator Mike Lee of Utah would hold the app stores accountable for verifying users' ages. Still, Meta's track record in its state campaigns is mixed. At least eight states have passed laws since 2024 forcing social media platforms to verify users' ages and protect minors online. Apple and Google have mobilized dozens of lobbyists across those states to argue that Meta is shirking responsibility for protecting children. 'We see the legislation being pushed by Meta as an effort to offload their own responsibilities to keep kids safe,' said Google spokesperson Danielle Cohen. 'These proposals introduce new risks to the privacy of minors, without actually addressing the harms that are inspiring lawmakers to act.' Meta spokesperson Rachel Holland countered that the company is supporting the approach favored by parents who want to keep their children safe online. 'Parents want a one-stop-shop to oversee their teen's online lives and 80% of American parents and bipartisan lawmakers across 20 states and the federal government agree that app stores are best positioned to provide this,' Holland said. As the regulation patchwork continues to take shape, the companies have each taken voluntary steps to protect children online. Meta has implemented new protections to restrict teens from accessing 'sensitive' content, like posts related to suicide, self-harm and eating disorders. Apple created 'Child Accounts,' which give parents more control over their children's' online activity. At Apple, spokesperson Peter Ajemian said it 'soon will release our new age assurance feature that empowers parents to share their child's age range with apps without disclosing sensitive information.' As the lobbying battle over age verification heats up, influential big tech groups are splintering and new ones emerging. Meta last year left Chamber of Progress, a liberal-leaning tech group that counts Apple and Google as members. Since then, the chamber, which is led by a former Google lobbyist and brands itself as the Democratic-aligned voice for the tech industry, has grown more aggressive in its advocacy against all age verification bills. 'I understand the temptation within a company to try to redirect policymakers towards the company's rivals, but ultimately most legislators don't want to intervene in a squabble between big tech giants,' said Chamber of Progress CEO Adam Kovacevich. Meta tried unsuccessfully to convince another major tech trade group, the Computer & Communications Industry Association, to stop working against bills Meta supports, two people familiar with the dynamics said. Meta, a CCIA member, acknowledged it doesn't always agree with the association. Meta is also still a member of NetChoice, which opposes all age verification laws no matter who's responsible. The group currently has 10 active lawsuits on the matter, including battling some of Meta's preferred laws. The disagreements have prompted some of the companies to form entirely new lobbying outfits. Meta in April teamed up with Spotify Technology SA and Match Group Inc. to launch a coalition aimed at taking on Apple and Google, including over the issue of age verification. Meta is also helping to fund the Digital Childhood Alliance, a coalition of conservative groups leading efforts to pass app-store age verification, according to three people familiar with the funding. Neither the Digital Childhood Alliance nor Meta responded directly to questions about whether Meta is funding the group. But Meta said it has collaborated with Digital Childhood Alliance. The group's executive director, Casey Stefanski, said it includes more than 100 organizations and child safety advocates who are pushing for more legislation that puts responsibility on the app stores. Stefanski said the Digital Childhood Alliance has met with Google 'several times' to share their concerns about the app store in recent months. The App Association, a group backed by Apple, has been running ads in Texas, Alabama, Louisiana and Ohio arguing that the app store age verification bills are backed by porn websites and companies. The adult entertainment industry's main lobby said it is not pushing for the bills; pornography is mostly banned from app stores. 'This one-size fits all approach is built to solve problems social media platforms have with their systems while making our members, small tech companies and app developers, collateral damage,' said App Association spokesperson Jack Fleming. In South Carolina and Ohio, there are competing proposals placing different levels of responsibility on the app stores and developers. That could end with more stringent legislation that makes neither side happy. 'When big tech acts as a monolith, that's when things die,' said Joel Thayer, a supporter of the app store age verification bills. 'But when they start breaking up that concentration of influence, all the sudden good things start happening because the reality is, these guys are just a hair's breath away from eating each other alive.' (Updates with App Association statement in 24th paragraph.) More stories like this are available on
Time of India
an hour ago
- Time of India
Meta names former OpenAI researcher Shengjia Zhao as Chief scientist for superintelligence lab
Meta Platforms has appointed Shengjia Zhao , a former OpenAI researcher and co-author of the original ChatGPT paper, as the Chief Scientist of its new Superintelligence AI group, the company confirmed Friday (July 25). Zhao, who joined Meta in June, was a key contributor to OpenAI's early breakthroughs, including the development of ChatGPT and the company's first reasoning model, known as o1. The model helped set off a new wave of 'chain-of-thought' AI systems adopted by companies such as Google and DeepSeek. Explore courses from Top Institutes in Please select course: Select a Course Category Digital Marketing MCA Finance PGDM Data Science Project Management Design Thinking Public Policy Healthcare Data Science Operations Management Leadership Management CXO Degree MBA Cybersecurity Others Artificial Intelligence Product Management Technology others Data Analytics healthcare Skills you'll gain: Digital Marketing Strategy Search Engine Optimization (SEO) & Content Marketing Social Media Marketing & Advertising Data Analytics & Measurement Duration: 24 Weeks Indian School of Business Professional Certificate Programme in Digital Marketing Starts on Jun 26, 2024 Get Details Skills you'll gain: Digital Marketing Strategies Customer Journey Mapping Paid Advertising Campaign Management Emerging Technologies in Digital Marketing Duration: 12 Weeks Indian School of Business Digital Marketing and Analytics Starts on May 14, 2024 Get Details Meta CEO Mark Zuckerberg announced Zhao's formal appointment in a post on Threads, calling him 'our lead scientist from day one.' Zuckerberg added: 'Now that our recruiting is going well and our team is coming together, we have decided to formalize his leadership role.' by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like They Lost Their Money - Learn From Their Lesson Best Solutions | Search ads Undo Zhao will report to Alexandr Wang, the former CEO of Scale AI, who joined Meta in June as Chief AI Officer. Wang is leading Meta's efforts in building artificial general intelligence (AGI), AI that can think and reason at or beyond human capability. Aggressive AI push Live Events The move comes amid Meta's aggressive recruitment campaign in the AI sector. Over the past two months, the company has hired more than a dozen researchers from OpenAI, Apple, Google, and Anthropic. That includes two top Apple AI scientists, Tom Gunter and Mark Lee, according to Bloomberg. Meta launched the Superintelligence Lab in June 2025 as part of its renewed focus on developing advanced AI models. The lab is separate from Meta's long-standing AI research group FAIR, which will continue to be led by Yann LeCun, who now reports to Wang. Talent and tensions Meta's recent hires have drawn attention for the size of the compensation packages involved. Some reports suggested offers exceeding $100 million, although the company has denied rumors of higher figures, including claims of $300 million deals. Meta's current open-source model, LLaMA 4, has not yet matched the capabilities of OpenAI's GPT-4 or Google's Gemini models. The company is expected to release a more advanced model, internally codenamed 'Behemoth,' later this year. Zuckerberg expressed optimism about the lab's future, saying, 'Together we are building an elite, talent-dense team that has the resources and long-term focus to push the frontiers of superintelligence research.'
