More Gemini Features Coming to Google TV
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreThe subtitles of audio content are not only crucial for deaf and hearing-impaired individuals, but also beneficial for everyone. Watching videos on mute is a common practice, whether it's on a train, in a meeting, at night, or when children are sleeping. Research has shown that subtitles can increase the viewing time of users by nearly 40%. However, subtitle support plugins are scattered across various applications and even within applications, making it difficult to access a large amount of audio content.
Recently, the Google AI team introduced Live Caption, a new feature that automatically adds subtitles to the media you play on your phone. The subtitles are added in real-time and completely on the device, without the need for network resources, thus protecting privacy and reducing latency. This feature is currently only available on Google Pixel 4 and Pixel 4 XL, and will be launched on Pixel 3 models later this year, and will soon be widely available on other Android devices.
Real-time captions work by combining three deep learning models on the following devices: a recursive neural network (RNN) sequence transduction model for speech recognition (RNN-T), a text-based recursive neural network model for speech recognition without punctuation, and a convolutional neural network (CNN) model for sound event classification. Real-time captions integrate signals from the three models to create a subtitle track where the process of speech recognition results is not interrupted when sound event labels appear. It automatically predicts punctuation while updating the text in parallel.
For speech recognition, the team utilizes models built on the AudioSet dataset to prepare for sound event detection. The speech recognition model is not only used to generate popular sound effect labels, but also to detect speech segments. The automatic speech recognition (ASR) RNN-T is only active during speech segments to minimize memory and battery usage. For example, when music is detected and there is no speech in the audio stream, the [MUSIC] label appears on the screen and the ASR model is unloaded. The ASR model is only loaded back into memory when speech reappears in the audio stream. To enable Live Caption to run continuously for a long time, the ASR model of Live Caption is optimized for edge devices using various techniques such as neural connection pruning, reducing power consumption by 50% compared to full-size speech models. However, despite being much more energy-efficient, the model still performs well in various scenarios and is robust to background noise.
The text-based punctuation model is optimized to run continuously on devices using a smaller architecture than cloud computing. As subtitles are formed, the speech recognition results are updated several times per second. To save computational resources and provide a smooth user experience, punctuation prediction is performed at the end of the text in the most recently recognized sentence, and if the next updated ASR result does not change the text, the previous punctuation result is retained and reused.
Live Caption is now available in English on Pixel 4 and will soon be available on Pixel 3 and other Android devices. We hope to bring this feature to more users by expanding support for other languages and further improving the formatting to enhance the perceived accuracy and coherence of the subtitles.
Google TV integrates Gemini AI for image/video generation, enhances Google Photos, and adds a dedicated YouTube Shorts feed to the homepage.
Read moreAlphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Read moreAnthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Read moreApple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Read moreGoogle launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
Read moreYouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Read moreGoogle grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Read moreMistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.
Read moreCustom subliminal audio maker

AI Social Media Manager

AI Search Visibility

Workout and Calorie Tracker
Stock research platform

AI interview assistant

Scientific figure generator

AI courses with real code