Microsoft came out swinging with its Windows Copilot 365 and Copilot for Windows announcements last year, pitching Copilot as a general virtual assistant to help you with your entire digital life. So far, we’ve yet to see Copilot reach its potential, but it looks like we’re one step closer: Microsoft is reportedly preparing to add a powerful new feature that will allow users to take a screenshot, send it to Copilot , and ask Copilot to explain what’s in the screenshot.
As far as we know, an “add a screenshot” button is being rolled out to the general public, which means you might already be able to try it out. If not, you should be able to do it very soon. This button should appear on the Copilot dashboard, prompting you to select a portion of the screen, confirming that you’ve captured what you want to discuss with Copilot, and then upload it to the right-side panel of Copilot or Bing. When I tested it, I also had to send some accompanying text, such as a question or additional context.
Once the screenshot is uploaded, you can talk about it and ask about anything it contains or is related to Bing Chat or Copilot in Windows 11.
As you can see in the screenshot, the new button is located next to the existing upload image button. You can try the new feature at Microsoft Copilot websiteor on Bing Chat.
Windows Latest has its own demo of this new feature, and I tried it myself. First, I took a screenshot selection showing a full description of video creation platform HeyGen’s YouTube channel and asked it to tell me two things: who the channel is and what it’s about.
The co-pilot returned this:
This is HeyGen’s YouTube channel. It’s a next-generation video creation platform that turns text into professional speaking videos in minutes. They offer premium avatars that speak multiple languages and professional video templates for various use cases, including marketing, e-learning, and corporate communication. You can find more information about HeyGen on their website.犀利士
p>
This is pretty accurate and reminded me of one particular feature I really like about Bing Chat and Copilot: they easily and conspicuously provide sources and websites you can visit to verify information.
Once you have selected your screen, you can make marks on it and draw on it. You can also add specific instructional images to help Copilot understand your query and you can move the selection window to a completely different part of the screen.
According to Windows Latest, Bing Chat recently received a ChatGPT-4 boost giving it a new level of functionality and this is likely coming to Copilot as well. Apparently, access is currently only granted to select users, and this development allows Copilot to engage in conversations about emotions. Currently, there is a limited group of users who can try this for themselves and access is apparently granted at random, and will be available to everyone who accesses Windows Copilot and Bing Chat very soon.
Microsoft charts a course to follow with Copilot
Microsoft has been pretty definitive in its message that Copilot is a big deal for the company and will be a core feature in several products like Microsoft 365 and Windows, but not just them.
In a pretty major (though not too surprising) development, Microsoft plans to add an actual physical Copilot button to the hardware of newly manufactured products starting in 2024. Microsoft is doing this in its continued effort to make computing, especially computer-driven by AI. computing, simpler and more fluid for users. This was detailed and confirmed in a recent Windows Experience blog post written by Yusuf Mehdi, executive vice president and head of consumer marketing at Microsoft.
For the rest of us who aren’t quite ready to ditch our older Windows devices for this new button, you can open Windows Copilot with the Win+C shortcut (if you’ve upgraded your version of Windows 11 to one that has Windows Copilot included) .
According to Microsoft itself, the introduction of the Copilot key will be the most notable update to the Windows keyboard in almost thirty years. Compare this future introduction to the addition of the Windows Start key, which is putting a lot of faith in Copilot, so I imagine we’ll continue to see major developments in Copilot throughout this year. I think, especially with the development of Copilot, Microsoft is one of the most interesting companies to watch this year.
