You can now augment Google Lens photo searches with text

By Chuong Nguyen Published April 7, 2022

Google is looking to improve its search results by leveraging both the power of photos, along with additional text for context. The new experience is called multisearch, which will be available on phones and tablets as part of Google Lens inside the Google app.

Google says the feature combines visual and word searches together to deliver the best results possible, even when you can’t describe exactly what it is that you’re trying to search for.

Five screenshots that show how to search in Google using multiple elements. — Image used with permission by copyright holder

“At Google, we’re always dreaming up new ways to help you uncover the information you’re looking for — no matter how tricky it might be to express what you need,” Google explained of the new multisearch feature. “That’s why today, we’re introducing an entirely new way to search: Using text and images at the same time. With multisearch in Lens, you can go beyond the search box and ask questions about what you see.”

Recommended Videos

A practical example of where multisearch will be useful is online shopping. Fashionistas may like a particular style of dress but may not know what that style is called. In addition, rather than shopping from a catalog with that particular dress available in a specific color, by leveraging the power of multisearch, you can snap a picture of the dress and search for the color green or orange. Google will even suggest similar alternatives in the colors you want.

“All this is made possible by our latest advancements in artificial intelligence.”

In this sense, multisearch extends the Google Lens experience by not only identifying what you see but by entering additional search text — like the color green — your search becomes more meaningful.

“All this is made possible by our latest advancements in artificial intelligence, which is making it easier to understand the world around you in more natural and intuitive ways,” Google explained the technology powering multisearch. “We’re also exploring ways in which this feature might be enhanced by MUM — our latest AI model in Search — to improve results for all the questions you could imagine asking.”

To begin your multisearch experience, you’ll need the Google app, which can be downloaded as a free app on iOS and Android devices. After you download the app, launch it, tap on the Lens icon, which resembles a camera, and snap a picture or upload one from your camera roll to begin your search. Next, you’ll want to swipe up and click on the plus (+) icon to add to your search.

Some ways to use this new multisearch tool include snapping a picture of your dining set and adding the “coffee table” term to your search to find matching tables online, or capturing an image of your rosemary plant and adding the term “care instruction” to your search to find out how to plant and care for rosemary, Google said.

Multisearch is available now as a beta experience within the Google app. Be sure to keep your Google app updated for the best results.

Computing Writer

Silicon Valley-based technology reporter and Giants baseball fan who splits his time between Northern California and Southern…

Topics

Computing

Ancient Mayan city discovered via page 16 of Google search results

Proceeding to even the second page of Google search results is rare enough, but going all the way to page 16 and then selecting an entry that leads to the discovery of a huge Mayan city that was lost for centuries under a jungle canopy ... well, that’s really something.

“I was on something like page 16 of Google search and found a laser survey done by a Mexican organization for environmental monitoring,” Luke Auld-Thomas, a Ph.D. student at Tulane University in Louisiana, said in comments reported by the BBC.

Mobile

Motorola phones are getting Google’s excellent Circle to Search feature

The back of the Motorola Razr Plus 2024.

Days ago, we learned that many Motorola phones will be receiving the Android 15 update. We’ve now heard that a feature that initially debuted with the Samsung Galaxy S24 series and select Google Pixel phones will soon be available on Motorola devices as well.

According to Motorola, Circle to Search will be available on select Moto phones. Unfortunately, the company didn't answer the most important question: which Motorola phones will be getting the feature.

Computing

Google expands its AI search function, incorporates ads into Overviews on mobile

A woman paints while talking on her Google Pixel 7 Pro.

Google announced on Thursday that it is "taking another big leap forward" with an expansive round of AI-empowered updates for Google Search and AI Overview.
Earlier in the year, Google incorporated generative AI technology into its existing Lens app, which allows users to identify objects within a photograph and search the web for more information on them, so that the app will return an AI Overview based on what it sees rather than a list of potentially relevant websites. At the I/O conference in May, Google promised to expand that capability to video clips.
With Thursday's update, "you can use Lens to search by taking a video, and asking questions about the moving objects that you see," Google's announcement reads. The company suggests that the app could be used to, for example, provide personalized information about specific fish at an aquarium simply by taking a video and asking your question.
Whether this works on more complex subjects like analyzing your favorite NFL team's previous play or fast-moving objects like identifying makes and models of cars in traffic, remains to be seen. If you want to try the feature for yourself, it's available globally (though only in English) through the iOS and Android Google App. Navigate to the Search Lab and enroll in the “AI Overviews and more” experiment to get access.

You won't necessarily have to type out your question either. Lens now supports voice questions, which allows you to simply speak your query as you take a picture (or capture a video clip) rather than fumbling across your touchscreen in a dimly lit room.
Your Lens-based shopping experience is also being updated. In addition to the links to visually similar products from retailers that Lens already provides, it will begin displaying "dramatically more helpful results," per the announcement. Those include reviews of the specific product you're looking at, price comparisons from across the web, and information on where to buy the item.