California, May 16, 2024: Google recently showcased its advanced AI capabilities at the Google I/O conference, demonstrating how its new AI systems can interpret information from images, videos, sounds, and spoken language via phone cameras.

One impressive demo featured a prototype AI assistant that could answer the age-old question, “Where did I put my glasses?”

Multimodal AI Demonstrations

In the demo, a prototype AI-powered assistant running on a phone used its camera to locate misplaced glasses. This unveiling came a day after rival OpenAI introduced its latest AI system, GPT-4, which demonstrated its ability to read human expressions through a phone camera and engage in fluent conversations, including playful banter.

Google emphasized that its tools are equally adept at “multimodal” understanding—integrating and processing multiple forms of data, such as images, video, and sound. The firm had teased its system’s capabilities just before OpenAI’s announcement, highlighting the competitive nature of AI advancements.

Gemini Nano and Scam Spotting

Google showcased several new features in its Gemini Nano AI assistant, designed to run on its Pixel phones and the Gemini App. Among these features was a prototype scam alert system that can listen to phone calls and detect potential scams without sending any information off the device.

Google I/O Highlights

At the Google I/O conference, Sir Demis Hassabis, head of Google DeepMind, repeatedly stressed the company’s long-term focus on multimodal AI. He showcased Project Astra, which explores the future of AI assistants. In a demo, the virtual assistant answered spoken questions about what it saw through a phone camera and successfully identified the location of a pair of glasses on a nearby desk.

Live Demos and New Features

Google also presented a live demo of using video in Google Search. For example, when shown a malfunctioning record player, Google Search suggested ways to fix it. Other notable announcements included:

• AI-Generated Overviews: Text that answers search questions before the listed results. Currently tested in the UK, these will soon be rolled out across the US and other countries.
• Enhanced Google Photos Search: AI-powered search features to make finding photos easier.
• New Creative Tools: AI-generated images, videos, and music will be previewed to selected musicians, artists, and filmmakers.
• Gmail Enhancements: AI features such as summarizing emails on specific topics will be added to Gmail.
• Future AI Assistants: A prototype system demonstrated creating a virtual “teammate” that could perform tasks like attending multiple online meetings simultaneously.

The Future of AI at Google

Google’s advancements in AI showcase the company’s commitment to integrating AI into everyday tasks, making life easier and more efficient. The ability to find lost items, detect scams, and enhance creativity with AI tools highlights a future where AI plays a significant role in simplifying and enriching human experiences. As these technologies continue to evolve, users can expect even more innovative solutions from Google.