Google has slowly but surely been releasing exciting new features relating to its Google Gemini platform, from a revamped app for mobile to the neat Circle to Search feature available on the newest Google Pixel and Samsung Galaxy devices. At Google I/O earlier this week, Google announced several enhancements. Here are some of the key upgrades you’ll want to know about and might actually use.
Generative AI Search
With the revamped generative AI for Google Search, you can now ask Google for assistance with more nuanced tasks. This might be helping you complete things, researching a trip, planning a meeting, or even brainstorming ideas for a project. It now uses Gemini’s multi-step reasoning, planning, and multimodality.
AI Overviews
An AI Overview feature, which has been in play through Search Labs for some time, provides a quick overview of a topic then links to learn more should you so desire. It’s now rolling out to users in the U.S., with additional countries coming soon. According to Google, AI Overviews result in people using search more and visiting a greater diversity of websites for help with more complex questions. A sample question might be, for example, not just asking for a good yoga studio, but asking for Google to “find the best yoga or pilates studios in Boston and show me details on their intro offers and walking time from Beacon Hill.”
Google also says that links included in AI Overviews get more clicks than if the page appeared as a traditional web listing for the same query. The result, says Google, will be sending traffic to more publishers and creators, while organic and sponsored results will continue to be distinguishable from one another.
You’ll also have the ability to adjust your AI Overview to simplify the language or break it down in more detail.
Planning
Search is also being enhanced to help you better plan ahead, starting with everything from vacations for individuals, families, or even large groups to meal planning for the week, right down to the starting point and recipes. You can make adjustments on the fly as well, like changing a recommended dish to a vegetarian alternative. You can then export the details to Docs or Gmail. Meal and trip planning are available now in Search Labs in English in the U.S. Later this year, Google will add customization capabilities and more categories like parties, date night, and workouts.
Brainstorming and AI-Generated Results Pages
While seeking out ideas and inspiration, Search will use generative AI to help you brainstorm and create an AI-organized results page that’s easier to navigate. Results will be categorized under unique AI-generated headlines, featuring a range of perspectives and content types. For English searches in the U.S., you’ll start to see this new AI-organized search results page when you look for inspiration. It will begin with dining and recipes, followed by movies, music, books, hotels, shopping, and more.
Ask Questions With Video
Visual search will be kicked up a notch, allowing you to take a video of something you want to ask questions about, including objects in motion. Rather than try to describe an issue you’re having with something, take a video and run the search to find relevant replies. The results will be presented as an AI Overview with steps and resources to help you troubleshoot. Searching with video will be available soon for Search Labs users in English in the U.S. and will be expanded to more regions over time.
Larger Context Window
Gemini Advanced will have access to Gemini 15 Pro, which offers a larger context window, new data analysis capabilities, connections to additional Google apps, and more customizable options. The context window can make sense of large documents that are up to 1,500 pages total or summarize up to 100 e-mails. It will soon also be able to handle an hour of video content or codebases with more than 30,000 lines. You can also upload files via Google Drive or directly from the device. This can help you figure out details about complex documents like a pet policy in your rental agreement or key arguments in long research papers. It will soon also be able to uncover insights and building custom visualizations and charts on the fly from uploaded data files like spreadsheets. You could also do things like take a photo of a dish at a restaurant to get a recipe or take a photo of a math problem to get step-by-step instructions on how to solve it. Gemini 1.5 Pro will be available to Gemini Advanced subscribers in more than 150 countries and over 35 languages.
Gemini Live With More Natural Conversations
There will be new ways to interact with Gemini more naturally, both while texting and talking. With Gemini in Google Messages, you can chat with Gemini in the same app you message your friends. In the coming months, Google is also rolling out Live for Gemini Advanced subscribers, a new mobile conversational experience that uses speech technology to make speaking with Gemini more intuitive. With Gemini Live, you can talk to Gemini and choose from a variety of natural-sounding voices with which it can respond. You can also speak at your own pace or interrupt mid-response with clarifying questions, just like you would when having a conversation with a human. Gemini can help you practice for a job interview, for example, even suggest skills you should highlight for the employer. Later this year you’ll be able to use your camera when you go Live, opening up conversations about what you see around you.
Trip Itineraries
Google Gemini Advanced can help with planning and organizing your next trip, going beyond just showing a list of activity options to creating full custom itineraries. Tell Gemini, for example, where you plan to go, what the members of your family like, from food to activities, then to pull your flight and hotel information from Gmail and help plan the entire weekend. It will take into account the time of your flight, meal preferences, information about local museums, and more, then use Google Maps to find the best spots close by. It will also use Google Search to recommend other activities to help fill the days. All of this information will combine together to create a customized itinerary that you can manually change or add details to at any time. This dynamic new planning experience is coming to Gemini Advanced in the coming months.
Create Customized Gems
Gemini Advanced subscribers will soon be able to create customized version of Gemini called Gems. These could be a virtual gym buddy, a sous chef, a coding partner, or even a creative writing guide. Describe what you want the Gem to do and how it should respond, and it will adjust accordingly. Tell it to be a positive and upbeat running coach, for example, that provides a daily running plan.
App Extensions
Extensions were launched for Gemini last year, integrating the service with various Google apps and services. There are now more extensions, including a YouTube Music extension rolling out now, which allows you to use Gemini to search for favourite music even without knowing the song title: just mention a favourite verse or featured artist, for example. Soon, Google will be connecting Gemini with other Google services, including Google Calendar, Tasks, and Keep. You will be able to do things like take a photo of your child’s school syllabus and ask Gemini to create a calendar entry for each assignment, or use a photo og a recipe to automatically add it to Keep as a shopping list.