At Google I / O, Google Vice President Prabhakar Raghavan announced new enhancements for its Lens multi-search toolwhich allows you to do a search with just a picture and a few words.
A new mode called “near me” will allow users to take a photo of an object and then find results locally. As Raghavan explained, you will be able to take a photo of a dish and then look for restaurants that serve that specific food. Google will then display a list of relevant restaurants near you. In order for this feature to occur, Google scans the relevant photos from websites, as well as those posted by reviewers, and then matches them with the one you uploaded. Near Me will be available in English later this year and will expand to more languages ”over time.”
Google is also developing something called scene research. This will allow users to move their camera and then enter a search phrase about the objects in front of them. When explaining the feature, Raghavan used the example of trying to find a nut-free chocolate bar in a supermarket. You’ll be able to scan an entire shelf of chocolate bars and then see overlays that provide “helpful insights,” such as reviews of each item. We think Raghavan’s description of the feature sums up this pretty nicely: “This is like having an overloaded Ctrl + F for the world around you.”
The search giant first launched a multi-search in April, but its goal was primarily related to shopping and searching for directions. For example, you can take a screenshot or picture of a dress you like and then type in the color name. Google’s multi-search will then display a list of results with similar-looking dresses in that color. Or you can take a photo of a specific type of plant and look for “care instructions” to find the relevant information.
But, as Lou Wang, Google’s director of product management, alluded to during its release, Google’s multi-search feature can be used for much more, and it seems that that additional functionality is starting to arrive.
To use multi-search, open the Google app on Android or iOS, and then tap the Lens icon to the right of the search bar. From there, you can upload a photo or screenshot from your gallery – or take a picture of an object in front of you – to begin your search. Then, swipe up on the screen and select “Add to your search” to search for the photo next to that word or phrase.
Updated May 11, 1:51 PM ET: Updated to add additional context of Prabhakar Raghavan.
Updated May 11, 2:12 PM ET: Updated to add the relevant Google blog post and more information related to the new features.