This isn’t the first time we’ve heard of this Copilot VisionActually Microsoft told us about it last Septemberthough it’s true that at the time it was just a preview of what was about to be added to the feature set grouped into the Copilot family. And it’s true that even with the caveat that we’ve recently experienced “certain precedents” that made us worry about security and privacy, there’s no doubt that with good implementation we found it an interesting proposition.

Copilot Vision extends the capabilities of Microsoft’s AI assistant by analyze in real time the text and images of the web pages that the user visits. This allows you to answer specific questions about the displayed content and suggest actions or other relevant information. For example, when you read a recipe online, Copilot Vision they could suggest alternative ingredients or related cooking techniques to enrich the user experience.

The implementation of this tool was designed with a focus on privacy and user control. Copilot Vision It is an optional feature that must be activated manually. Once used, it processes the information locally on the device and deletes the data at the end of the session, ensuring that it is not stored or used to train AI models. Initially, its operation is limited to a selected set of websites, with gradual expansion planned in the future.

It is important to emphasize this Microsoft seems to have learned from past experiencessuch as feature-related privacy issues Appeal in Windows 11. Appeal threatened to take regular, indiscriminate screenshots of the user, raising significant privacy concerns and prompting the company to make modifications to its implementation. WITH Copilot VisionMicrosoft says it has taken proactive measures to prevent similar issues and ensure that this feature is optional and that data is processed and deleted locally.

At the moment, Copilot Vision andIt is available in a testing phase for a limited number of subscribers. Copilot Pro in the United Statesthrough an experimental program Copilot Labs. This phase will allow Microsoft to get feedback and make adjustments before a wider rollout. The company emphasized that while this feature offers deeper interaction with web content, it does not take actions on behalf of the user or interact directly with pages without their express consent.

The technology behind it Copilot Vision It is based on a key area of ​​artificial intelligence known as computer vision. This field focuses on providing machines with the ability to interpret images and videos in a similar way that a human being would. By combining advanced image processing and deep learning techniques, it enables systems like this to understand visual elements such as graphics or photographs and the wider context within a web page.

More information