Microsoft tests Copilot Vision desktop share in Windows 11 Insider Build: Here's what's new
Copilot can answer questions, suggest improvements, and guide you through tasks based on what's on your screen. Microsoft says it may assist with things like editing your resume, offering feedback on creative projects, or helping you navigate a new game. The feature is similar to what Google's Gemini Live and Apple Intelligence are beginning to offer.
Copilot Vision allows the assistance to access and interpret the visible content on your screen in real time. It helps you with what you are working on, reads that content and offers relevant suggestions, summaries or whatever you ask.
For example, suppose you have a PDF document open on your computer screen, you can ask Copilot to summarise it or tell main points of the content. If you are working in an Excel sheet, Copilot might help you spot errors, generate new formulas, help you in solving your issues with quick formulas, and compare values in a table without needing to describe the sheet. When browsing the web, it could highlight product differences or summarise a long review page. All of this happens through screen context, reducing the need to switch apps or copy and paste information.
Importantly, this feature uses screen capture permissions, and Microsoft says Copilot only accesses your screen after you give permission. You also have full control over what it sees and when it stops.
Now these will help you understand better how Copilot Vision helps you in real life. Document review: Copilot can pick out key terms or show what's changed in a contract, so you don't have to scroll through the whole thing.
Spreadsheets: It can look at your sales data and give quick insights, fix errors, or turn numbers into charts right on your screen.
E-commerce: Compare specs of products side by side directly from your screen without searching again.
Email writing: If you have a report or slide deck open, Copilot could help draft a quick summary to send to a colleague.
Troubleshooting: If you are stuck on a system setting or error screen, Copilot could explain the issue or offer a fix based on what's visible.
The new Desktop Share feature makes it possible for Copilot to view your screen in real time and respond to you based on what is on the screen. To enable this feature open the Copilot app and click the glasses icon. From there, choose which part of your desktop to share. Once active, you can ask questions like 'What does this error mean?' or 'How do I create a table in Excel?' and Copilot will respond based on what it sees.
You can even stop sharing any time by selecting the "Stop" option in the same panel. You can also activate the Vision using your voice, thanks to its feature called Voice Conversation. You can verbally say your request and Copilot will respond using visual content on the screen. Copilot listens and responds based on what's on your screen, without needing to type or click.
Microsoft has added a Highlights feature that shows helpful tips based on the apps, files or websites you are using without you having to ask.
Copilot Vision stands out by removing the need to type long prompts. It sees what's on your screen and helps right away, making it feel more like a built-in assistant than a chatbot. Unlike screen sharing or remote tools, it works locally and is just for you.
The feature is still in testing with no set release date, but Microsoft is clearly moving toward assistants that respond to both what you type and what you see on screen.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Mint
28 minutes ago
- Mint
Move over, passwords: Android's next big update lets your smartwatch unlock your phone anytime, anyplace
Google is working on a new update that could quietly change how you unlock your phone in public spaces. Hidden details in recent Play Services beta code reveal that your smartwatch might become your new ally in digital security. This shift points toward making everyday tech both safer and less of a hassle when you are out and about. Today, Android's Identity Check depends on trusted locations like your home or office. Once you leave these comfort zones, your phone gets stricter about demands for PINs or fingerprints. This serves as a barrier to thieves and is meant to protect you, but it can get annoying when you are on the move and want quick access. Hints from the latest Play Services suggest something new. Soon, if your Android smartwatch is connected and near, your phone may recognise that as a sign you are still in control. Instead of always asking you to verify your identity, the phone could let you in more easily as long as it senses your watch. As a result, people who keep their watch on and use a PIN to secure it might face fewer login interruptions, even outside their regular safe places. This is not a total removal of security. It is a way to make authentication smarter. If the watch is there and locked to your wrist, the risk that someone else is holding both devices is low. But if you get separated from your phone or your watch, the extra checks and verification would still kick in. Google's update is about making things less intrusive for regular users. Your daily routine should get easier, but not at the expense of safety. You will still need your PIN, fingerprint, or face scan at times. The difference is that you might only need them less often if your smartwatch is acting as a backup. People are putting more of their lives on their phones and wearables. Payments, travel alerts, private chats - all of it sits in your pocket or on your wrist. With more important data in one place, user-friendly security now matters as much as tough security. Some concerns remain. If someone steals both your phone and your watch at once, this system alone won't protect you. But for most people, the convenience far outweighs this rare risk. Having a trusted device like a smartwatch as an added barrier can mean less hassle and more protection in busy daily life. It's important to note that this information was found in an APK teardown, which means that this feature may or may not ever see the light of day.
Mint
an hour ago
- Mint
Microsoft Layoffs: In 15,000 job cut memo, CEO Satya Nadella says 'by every objective measure...'
Microsoft CEO Satya Nadella has addressed the recent wave of job cuts at the tech giant, even as the company reports soaring profits, record stock prices, and unprecedented investment in artificial intelligence (AI). In a memo sent to employees on Thursday, Nadella acknowledged the concerns around the layoffs, saying, 'Before anything else, I want to speak to what's been weighing heavily on me and what I know many of you are thinking about: the recent job eliminations.' So far in 2025, Microsoft has laid off over 15,000 employees. In addition, around 2,000 staff who were deemed underperformers have also been let go. This comes at a time when Microsoft is enjoying strong financial performance. The company has reported approximately $75 billion in net income over the past three fiscal quarters, according to Business Insider. Meanwhile, it is investing $80 billion in AI infrastructure, and its shares recently closed above $500 for the first time on 9 July — a record high. Nadella attempted to explain the reasoning behind the job cuts, despite the company's apparent success. 'By every objective measure, Microsoft is thriving—our market performance, strategic positioning, and growth all point up and to the right,' he wrote. 'We're investing more in CapEx than ever before. Our overall headcount is relatively unchanged, and some of the talent and expertise in our industry and at Microsoft is being recognised and rewarded at levels never seen before. And yet, at the same time, we've undergone layoffs,' Nadella added. He described the situation as 'the enigma of success in an industry that has no franchise value.' He went on to say: 'Progress isn't linear. It's dynamic, sometimes dissonant, and always demanding. But it's also a new opportunity for us to shape, lead through, and have a greater impact than ever before.' Nadella also stressed the importance of 'unlearning' and 'learning' as Microsoft adapts to new challenges and technologies. He reaffirmed the company's commitment to its mission, priorities, and culture — referring to the importance of understanding the "why, what, and how" behind Microsoft's direction. Despite the optimistic tone of the memo, many former employees have shared feelings of disappointment and sadness over their layoffs, according to a report by CNBC. Some said they had thoroughly enjoyed working at Microsoft and felt disheartened by the decision. Microsoft is currently the second most valuable publicly listed company in the world after Nvidia. Its Windows and Office products remain market leaders, and its Azure cloud division continues to experience rapid growth.
India Today
an hour ago
- India Today
Donald Trump doesn't want FAANG to hire in India, but supply gap means companies may not listen to him
US President Donald Trump, a day ago, urged tech giants like Google and Microsoft to stop hiring in India and 'focus on Americans'. His comments immediately went viral, with people posting the clip of him slamming tech companies for having a 'globalist mindset.' His comments were also rather clear and stark: he said that tech companies like Google, Microsoft, Amazon, Meta and others should put Americans first and should have their offices in America rather than setting them in India. advertisement'Many of our largest tech companies have reaped the blessings of American freedom while building their factories in China, hiring workers in India and stashing profits in Ireland, you know that,' said Trump. 'Under President Trump, those days are over.'The question then is this: Will Google, Microsoft and others follow and heed the call put forth by Trump? Will they stop hiring tech engineers and software developers in India? More significantly, will they look to set up new offices in America instead of in India? The answer to most of these questions is — unlikely. That is because the reality of tech talent supply, demand, and cost tells a different story. The US simply doesn't produce enough engineers to meet the growing needs in its tech sector, and hiring in India offers huge financial advantages. So, despite political pressure, tech hiring in India by FAANG and others is likely to enough tech engineers in the USLet's look at the numbers. Each year, according to the Integrated Postsecondary Education Data System, the US produces around 150,000 engineering graduates at the bachelor's level. In addition, around 50,000 students complete their master's in engineering, and roughly 12,000 receive doctorates. These sound like large numbers, but they're not enough when you consider how many people the tech industry needs. Just to give you a hint of the scale: Google employs around 1,80,000 employees. According to forecasts by the US Bureau of Labour Statistics, STEM jobs — especially in areas like software development — are expected to grow by about 11 per cent every year through 2032. In comparison, non-STEM jobs are expected to grow at just 2 per cent a year. That means demand for engineers is rising far faster than America can train and supply this creates is a talent gap. There are simply not enough American engineers to meet the demand of growing tech companies, particularly at this time, as they get into an AI race with each other as well as the rest of the world. And this gap forces companies to look outside the country, with India being one of the top choices. advertisementCompared to the US, the number of Indian engineering graduates is extremely high. According to estimates by India's Ministry of Education and AICTE, India produces over 1.5 million engineering graduates every year. While not all of them are in tech and software, and not all of them have the skillset that top tech companies look for in their employees, the sheer number of graduates itself is an attractive proposition for hiring managers working with global giants. At the same time, there is also a perception that fresh Indian IT engineers, particularly from top institutes like IIT, are eager and ready to grind in a way that US graduates are not. Then there is the cost factor Along with the mismatch in demand and supply, another reason why tech giants love to have their offices in India is the cost. Hiring Indians in India is simply more cost-effective for tech giants. Take salaries, for example. The median annual salary of a software developer in the US is typically around $120,000 - $140,000 (roughly Rs 1 - Rs 1.2 crores). In cities like San Francisco or Seattle, the pay is even higher, often ranging between $150,000 - $200,000 (roughly Rs 1.3 - Rs 1.73 crores) annually. advertisementMeanwhile, an Indian software developer earns significantly less. The average salaries of Indian software developers working in the US are around $75,000 (about Rs 65 lakh) per year, and in many cases, they are even India, the salaries for the same role and same work are even lower. In some cases, they can be as low as Rs 20 lakh (around $25,000), or even lower for positions that are open to only trainees and interns. The lower cost is one of the biggest reasons why companies continue to hire from India. For the same skills and productivity, firms can hire more people, manage larger projects, or simply reduce expenses. Tech companies likely to ignore TrumpThe US President is the world's most powerful person. But even he cannot compel any private tech company to hire or not hire people in some other country. The US government can surely force the companies with some specific laws, but the Trump administration is unlikely to do that because it would hit the US tech companies adversely. That means Trump can only suggest, and that is precisely what he did on July 24. The logic and ground realities mean that tech companies such as Google and Meta will simply ignore his comments. At best, they may, as a measure of reverence, hire a few more people in the US. But in general, Trump's comments are unlikely to change anything in connection with the FAANG hiring policies in India. The US currently just doesn't have enough engineers, and is definitely not cheap to hire, to fill all the vacancies that the tech companies have.- Ends
