Microsoft Tests New Copilot Vision Feature to Enhance Edge Browser's Intelligent Browsing

Microsoft has recently initiated trials for its newly launched Copilot Vision feature. Initially announced in October last year, this feature enables Microsoft's AI assistant to access and analyze the web pages users visit on the Edge browser. Users can then ask questions related to the displayed text, images, and content, or leverage this functionality for assistance.

The Microsoft Copilot team explained in a blog post that activating Copilot Vision allows the feature to examine the web page a user is currently viewing, read in tandem with the user, and enable discussions about any issues encountered. Browsing the web will no longer be a solitary experience for users toggling between multiple tabs.

The feature is entirely optional; users must explicitly grant permission for Copilot Vision to access web pages within Microsoft's Edge browser. In October of last year, Microsoft first showcased this functionality, demonstrating how its AI assistant can extract images from OneDrive collections on web pages, recognize handwritten recipes, and offer cooking suggestions. Additionally, users can utilize this feature to discover product recommendations while shopping online.

Currently, Copilot Vision is in a limited testing phase, available exclusively to Copilot Pro subscribers through Microsoft's Copilot Labs program. Given that this feature allows AI models to read the content displayed in users' web browsers, it naturally raises privacy concerns. Microsoft has evidently adopted a cautious approach in response. The security issues related to the Recall feature, which began testing only last month, likely also contribute to Microsoft's careful handling of Copilot Vision.

The Copilot team stated that upon the feature's launch, Vision will interact only with a select group of websites. Over time, this list will be cautiously expanded. It is crucial to highlight that Vision does not capture, store, or utilize any publisher data to train its models. In summary, Microsoft prioritizes copyright, creator rights, and the privacy and security of users.

A limited number of Copilot Pro subscribers in the United States will have access to Copilot Vision. Microsoft will gather user feedback and iteratively enhance Copilot Vision based on this input. In the future, Microsoft plans to extend access to a broader range of Pro subscribers and websites.