Microsoft Launches New Copilot Experience and Multiple Updates

Microsoft introduced the all-new Copilot experience during a recent launch event in New York City. This upgrade extends Copilot's capabilities across mobile devices, web applications, and the Windows operating system, while introducing an innovative card-based user interface. New features include Copilot Vision, enabling AI to recognize the content users are engaging with; in addition, a natural voice conversational mode similar to OpenAI’s has been added, along with a virtual news anchor feature capable of reading out news headlines.

Windows 11 has been enhanced with new features, such as the integration of Phone Link status into the Start menu, enabling the display of notifications and phone battery levels. Moreover, the Paint and Photos applications have received exciting AI enhancements, including generative fill and erase capabilities. For PCs equipped with Copilot Plus, Windows Search has been upgraded with AI technology, introducing a one-click operation feature similar to Google’s Circle to Search, as well as the ability to search for photos using only text descriptions.

Earlier this year, Microsoft underwent an executive restructuring, with Pavan Davuluri taking charge of the Windows and Surface divisions, while Mustafa Suleyman was appointed as the CEO of the newly established AI department. This event also marks Microsoft’s continued advancement in the 'AI PC' sector following the abrupt departure of Panos Panay a year ago.

In the realm of software applications, Microsoft's Paint app will receive generative AI fill and erase capabilities akin to those in Adobe Photoshop, enabling users to more precisely add or remove objects within images. These features are implemented through adjustable brush sizes, facilitating more convenient image editing.

In terms of improvements to Windows Search, Microsoft has significantly enhanced the search experience by leveraging AI models, particularly on Copilot Plus PCs. The new search functionalities make it easier to locate and interact with content such as images, emails, documents, and videos. These enhancements will first debut in the File Explorer on Copilot Plus PCs next month, allowing users to search for images using text descriptions even if the search terms do not appear in the photo or file names.

Additionally, Microsoft has begun rolling out the Windows 11 2024 Update (version 24H2), which includes a series of minor yet practical enhancements for Windows 11, covering aspects such as the Start menu, File Explorer, and Settings. Specifically, for users utilizing the Phone Link software, the Start menu will feature a new sidebar that provides phone battery status, notifications, and quick access to messages, calls, and photos.

This major overhaul of Copilot not only introduced a comprehensive redesign but also incorporated voice and visual features, transforming it into a more personalized AI assistant. This shift is closely related to the talent Microsoft recruited from Inflection AI earlier this year, including Mustafa Suleyman, who now serves as the CEO of Microsoft's AI department. This update also marks his first significant transformation since taking over the consumer-oriented AI assistant.