Gemini's New Testing Feature: Personalized Responses Based on Your Photos, Emails, and More

Google has unveiled a new experimental feature within its Gemini app that enables the AI assistant to tailor its responses by connecting with your Google ecosystem, initially drawing from Gmail, Photos, Search, and YouTube history.

While Gemini could already fetch information from these applications, it can now reason across your data to offer proactive results, such as linking an email thread with a video you watched. Google states this allows Gemini to grasp context without explicit instructions.

The tech giant clarified that this test experience, termed "Personal Intelligence," is turned off by default. Users have the choice of whether and when to link their Google apps to Gemini. Naturally, not everyone may want an AI reviewing their photos or YouTube history. If you opt to connect your apps, Google notes that Gemini will only utilize personal intelligence when it determines it would be beneficial.

"Personal intelligence offers two core advantages: reasoning across complex sources and retrieving specific details from emails or photos to answer your queries," wrote Josh Woodward, Vice President for Gemini apps at Google Labs and AI Studio, in a blog post. "It often combines these, delivering uniquely tailored answers across text, photos, and videos."

Woodward shared an example where, while queuing at a tire shop, he couldn't recall his car's tire size. While most AI chatbots could identify a car's tire specifications, Woodward explained that Gemini goes a step further by providing personalized responses. In his case, after recognizing a family road trip photo in Google Photos, Gemini suggested all-weather tires. Woodward also mentioned forgetting his license plate number, but Gemini was able to extract it from a photo.

"I've also received excellent recommendations on books, shows, clothing, and travel," Woodward wrote. "Just this week, it excelled at planning our upcoming spring break. By analyzing our family's interests and past trips in Gmail and Photos, it skipped tourist traps. Instead, it suggested an overnight train journey and specific board games we could play along the way."

Google stated it has safeguards for sensitive topics, as Gemini will avoid making proactive assumptions about sensitive data like health. However, the company also noted that Gemini will discuss such data if explicitly asked.

Furthermore, Gemini is not trained directly on the contents of your Gmail inbox or Google Photos library. Instead, training occurs on specific prompts within Gemini and the model's corresponding responses. In the examples above, the road trip photo, the license plate image within it, and the email in Gmail are not directly used to train the model. Google clarified they are only referenced to generate a response.

Personal Intelligence is rolling out to Google AI Pro and AI Ultra subscribers in the United States. Google plans to extend the feature to more countries and to Gemini's free tier.