Google made it abundantly clear at its annual I/O developer conference this week – that artificial intelligence is taking center stage across its products and services.
The tech giant mentioned “AI” a staggering 120+ times during its keynote address, signaling an all-in push on advanced AI capabilities.
However, not all of Google’s announced AI innovations were groundbreaking. Some were more incremental updates or revisits of previous initiatives.
To help parse the truly significant releases, here are the top new AI products and features unveiled at Google I/O 2024.
Generative AI in Search Results
In a potentially transformative move, Google plans to utilize generative AI models to entirely reconstruct how Google Search result pages are organized and presented.
The revamped AI-curated pages will adapt their format based on the nature of the query.
For searches around travel planning inspiration, for example, the results may showcase AI-generated summaries of reviews, excerpts from Reddit discussions, and lists of personalized suggestions.
Google stated it would soon expand this AI-enhanced experience to searches for dining, recipes, movies, books, hotels, e-commerce, and more categories.
Project Astra and Gemini Live
Google aims to elevate its AI chatbot, Gemini, to better understand and interact with the real world through camera and voice integration.
The company previewed “Gemini Live,” an upcoming experience allowing users to engage in voice-based conversations with Gemini on their smartphones.
Gemini Live leverages Project Astra, a DeepMind initiative focused on creating AI agents with real-time multimodal skills.
With Live, users can interrupt Gemini mid-conversation to ask clarifying questions as it adapts its speech patterns dynamically. Remarkably, Gemini can also analyze a user’s surroundings through smartphone cameras to identify objects, describe scenes, and answer context-aware queries.
Planned for later this year, Gemini Live represents a major step forward in AI’s ability to perceive and converse about the physical environment.
Google Veo
Taking aim at OpenAI’s Sora, Google introduced its own AI video generator called Veo. This powerful model can create polished 1080p videos up to a minute long from text prompts alone.
Veo exhibits an impressive understanding of cinematography concepts like camera movements, visual effects, editing techniques, and even simulating realistic physics.
It can mimic diverse visual styles, generate videos from still images, and even piece together longer video narratives from iterative text descriptions.
While OpenAI’s Sora was first to market, Google’s Veo already appears to match or exceed it in many capabilities related to video generation.
Ask Photos
Google Photos users will soon gain the ability to search their photo libraries using natural language queries powered by the Gemini generative AI.
The new “Ask Photos” feature, rolling out this summer, allows lookups based on Gemini’s multimodal understanding of images and their metadata.
Instead of basic object detection, users can perform open-ended searches like “Find my best photos from each national park I visited.”
Gemini will analyze cues like lighting, sharpness, backgrounds, and location data to determine and retrieve the most compelling matches to the descriptive prompt.
Gemini in Gmail
Google’s versatile Gemini AI will soon lend its capabilities directly to Gmail for smarter email management.
Users will be able to ask Gemini to summarize messages and attachments, extract key details, automate organizational workflows like processing receipts and expense reports, and even draft new emails from prompts.
Detecting Scams During Calls
In a fusion of AI and privacy, Google previewed upcoming Android functionality to detect potential scams on audio calls using its on-device Gemini Nano model.
By analyzing conversation patterns, Nano can alert users to suspicious language in real time without uploading conversations to the cloud.
AI Accessibility Enhancements
For those with visual impairments, the familiar TalkBack screen reader for Android will soon tap into Gemini Nano to generate rich descriptions of unlabeled images and objects users encounter on their devices each day.
While ambitious, Google’s array of I/O AI releases demonstrates its determination to infuse generative AI across its entire product ecosystem – from Search to Photos, productivity apps like Gmail, Android’s core OS, and more.
By leveraging advances in multimodal and real-world perception, Google aims to make interacting with AI a more seamless, ubiquitous experience.
