Microsoft's AI CEO Mustafa Suleyman has announced on the Bluesky platform that the Vision feature of Microsoft Copilot is now available for free use in the Edge browser. This AI assistant functionality can identify on-screen content and help users interact with applications.
The Vision feature operates in "voice interaction" mode, allowing users to communicate with Copilot through voice commands while waiting for its response. Suleyman explained that once activated, Copilot Vision can observe screen content in real time. For instance, when cooking, users can rely on this feature to follow recipe steps progressively or let it analyze job descriptions to generate tailored interview preparation plans or resume ideas. However, Microsoft’s support page notes that while Copilot Vision may highlight specific areas of the screen to assist users in locating information, it won't automatically click links or perform other actions.
Currently, the broader system-level Copilot Vision remains exclusive to Copilot Pro subscribers. These users can utilize the feature outside the Edge browser, such as aiding in Photoshop tasks, video editing software, or even guiding them through gameplay in titles like Minecraft.
To experience Copilot Vision, users need to visit a designated link on Microsoft’s website via the Edge browser, where they will be prompted to enable the feature. After authorization, users can access the Copilot sidebar on any website, initiate a Vision session by clicking the microphone icon, at which point the browser will emit a prompt tone and adjust its color scheme.
In practice, some users report needing multiple attempts before the Edge browser prompts them to enable the feature, and after activation, control interface loading issues may occur. However, actual experiences might vary due to differences in device performance.
Microsoft has stated that it only records Copilot's responses and does not collect user inputs, images, or page content during Vision sessions. Users can end the screen sharing by terminating the session or closing the browser window directly.
