Google Launches New AI and Accessibility Features for Android and Chrome

Google has unveiled new AI and accessibility features for Android and Chrome. Notably, TalkBack, the screen reader for Android, now enables users to inquire about images and on-screen content with Gemini.

In the previous year, Google incorporated Gemini's capabilities into TalkBack, providing AI-generated image descriptions for those who are blind or visually impaired, even if the image lacks Alt text. Now, users can pose questions about images and receive responses.

For instance, if a friend sends you a photo of their new guitar, you can obtain its description and ask about the brand and color. Moreover, you can now get a description of the entire phone screen and ask questions. Therefore, if you're shopping in an app, you can inquire Gemini about the material of the item you like or whether there's any discount.

Google also announced today that it is enhancing Expressive Captions, Android's real-time captioning feature, using AI to capture both what people say and how they express it.

Google states that it recognizes one way people convey themselves is by elongating the sound of words, which led to the development of a new duration feature in Expressive Captions. Now, you'll know if a sports commentator is shouting "What a fantastic goal" or if someone isn’t just saying "no," but "don't." You'll also start seeing new audio tags, such as whistling or throat-clearing.

This update is rolling out to Android 15 devices and above in the United States, United Kingdom, Canada, and Australia.

Google has also streamlined the process of accessing PDFs on Chrome. Previously, you couldn't interact with scanned PDFs using a screen reader in the desktop Chrome browser. Now, Chrome automatically identifies these types of PDFs, allowing you to highlight, copy, search for text, and read them using a screen reader, thanks to the introduction of Optical Character Recognition (OCR), according to Google.

Additionally, page zoom in Chrome on Android now lets you increase text size without affecting the webpage layout. You can customize the zoom level and choose to apply your preferences to all pages you visit or just specific ones. This feature can be accessed by clicking the three-dot menu in the upper right corner of Chrome.