Google focuses Lens on combined image and text searching
(2022/04/07)
- Reference: 1649349009
- News link: https://www.theregister.co.uk/2022/04/07/google_multisearch/
- Source link:
Google's latest feature is making its Lens visual search tool mingle with text for image searches for those difficult-to-describe vague queries.
That may not sound like a big deal, but what Google calls [1]Multisearch "a feat of machine learning" could change the way people perform some of the most common searches and take a bite out those moments when you know the rough idea of what you're looking for but lack a suitable way to define it in text or image searches alone.
Retailers will salivate over the vague search option, for one, as highlighted by Belinda Zeng, Google Search product manager, who described several examples of how Multisearch could work: "How many times have you tried to find the perfect piece of clothing, a tutorial to recreate nail art or even instructions on how to take care of a plant someone gifted you – but you didn't have all the words to describe what you were looking for?"
[2]
Multisearch would address all of those situations: a photo of a dress could be queried with the name of a preferred color, photos of a finished set of nails could be searched with the phrase "how to," and unknown plans can be Googled with "care instructions."
[3]
An example of Google's Multisearch tool
In each of those cases, Google Search is addressing multiple unknowns: what the object in the image is, how the text request relates to the image, and how to combine the two into what the user wants.
In short, what you need is an expert on those particular subjects with the skill to infer, based on what it's presented with, the best course of action.
Say hi to MUM
Multisearch is possible thanks to Google's Multitask Unified Model, or MUM. MUM is an improvement on [4]BERT , Google's previous AI language model released in 2019. BERT improved Google search results by 10 percent for English speakers, Google said. MUM, it added, is 1,000 times more powerful than BERT.
Google's Pandu Nayak described the difference between BERT and MUM using the planning of a hiking trip as an example: ask an expert how to prepare for a hike and they'll take all sorts of things into account.
[5]Google makes outdated apps less accessible on Play Store
[6]Google Cloud previews new BigLake data lakehouse service
[7]Boeing spreads bets with AWS, Google, Microsoft trio
[8]Google snubs South Korea's app store payments law
"Today, Google could help you with this, but it would take many thoughtfully considered searches – you'd have to search for the elevation of each mountain, the average temperature in the fall, difficulty of the hiking trails, the right gear to use, and more," Nayak said.
BERT is what you'd use to make multiple searches (Nayak said eight on average for a query of that level of complication), but MUM is designed to respond like the aforementioned professional. "MUM could understand you're comparing two mountains, so elevation and trail information may be relevant. It could also understand that, in the context of hiking, to 'prepare' could include things like fitness training as well as finding the right gear," Nayak said.
[9]
As with all things search, Google isn't providing specifics about its algorithms, so the inner workings of MUM and its Multisearch capabilities are likely to remain a mystery. Google said that Multisearch is available in beta through its iOS and Android apps in the US only, though as of writing it doesn't appear to be available in the iOS version of the app.
Google said MUM-powered Multisearch currently works best for shopping-related searches (e.g. drapes with pattern X, a dress in a different color, etc.). Other searches will work, but Google's not vouching for their effectiveness. We've asked Google if it can explain the technology behind Multisearch and future applications in more detail.
[10]
In calendar 2021, 'Google Search and other' brought in $43.3 billion in revenue, up from $31.9 billion in the prior twelve months. The entire company turned over $75.3 billion for the year. ®
Get our [11]Tech Resources
[1] https://blog.google/products/search/multisearch/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://regmedia.co.uk/2022/04/07/google-new-multisearch.jpg
[4] http://theregister.com/2019/10/28/google_search_ai/
[5] https://www.theregister.com/2022/04/07/google_makes_outdated_apps_less/
[6] https://www.theregister.com/2022/04/06/google_previews_biglake_data_lakehouse/
[7] https://www.theregister.com/2022/04/06/boeing_signs_new_cloud_deals/
[8] https://www.theregister.com/2022/04/06/google_south_korea_app_payments_illegal/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://whitepapers.theregister.com/
That may not sound like a big deal, but what Google calls [1]Multisearch "a feat of machine learning" could change the way people perform some of the most common searches and take a bite out those moments when you know the rough idea of what you're looking for but lack a suitable way to define it in text or image searches alone.
Retailers will salivate over the vague search option, for one, as highlighted by Belinda Zeng, Google Search product manager, who described several examples of how Multisearch could work: "How many times have you tried to find the perfect piece of clothing, a tutorial to recreate nail art or even instructions on how to take care of a plant someone gifted you – but you didn't have all the words to describe what you were looking for?"
[2]
Multisearch would address all of those situations: a photo of a dress could be queried with the name of a preferred color, photos of a finished set of nails could be searched with the phrase "how to," and unknown plans can be Googled with "care instructions."
[3]
An example of Google's Multisearch tool
In each of those cases, Google Search is addressing multiple unknowns: what the object in the image is, how the text request relates to the image, and how to combine the two into what the user wants.
In short, what you need is an expert on those particular subjects with the skill to infer, based on what it's presented with, the best course of action.
Say hi to MUM
Multisearch is possible thanks to Google's Multitask Unified Model, or MUM. MUM is an improvement on [4]BERT , Google's previous AI language model released in 2019. BERT improved Google search results by 10 percent for English speakers, Google said. MUM, it added, is 1,000 times more powerful than BERT.
Google's Pandu Nayak described the difference between BERT and MUM using the planning of a hiking trip as an example: ask an expert how to prepare for a hike and they'll take all sorts of things into account.
[5]Google makes outdated apps less accessible on Play Store
[6]Google Cloud previews new BigLake data lakehouse service
[7]Boeing spreads bets with AWS, Google, Microsoft trio
[8]Google snubs South Korea's app store payments law
"Today, Google could help you with this, but it would take many thoughtfully considered searches – you'd have to search for the elevation of each mountain, the average temperature in the fall, difficulty of the hiking trails, the right gear to use, and more," Nayak said.
BERT is what you'd use to make multiple searches (Nayak said eight on average for a query of that level of complication), but MUM is designed to respond like the aforementioned professional. "MUM could understand you're comparing two mountains, so elevation and trail information may be relevant. It could also understand that, in the context of hiking, to 'prepare' could include things like fitness training as well as finding the right gear," Nayak said.
[9]
As with all things search, Google isn't providing specifics about its algorithms, so the inner workings of MUM and its Multisearch capabilities are likely to remain a mystery. Google said that Multisearch is available in beta through its iOS and Android apps in the US only, though as of writing it doesn't appear to be available in the iOS version of the app.
Google said MUM-powered Multisearch currently works best for shopping-related searches (e.g. drapes with pattern X, a dress in a different color, etc.). Other searches will work, but Google's not vouching for their effectiveness. We've asked Google if it can explain the technology behind Multisearch and future applications in more detail.
[10]
In calendar 2021, 'Google Search and other' brought in $43.3 billion in revenue, up from $31.9 billion in the prior twelve months. The entire company turned over $75.3 billion for the year. ®
Get our [11]Tech Resources
[1] https://blog.google/products/search/multisearch/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://regmedia.co.uk/2022/04/07/google-new-multisearch.jpg
[4] http://theregister.com/2019/10/28/google_search_ai/
[5] https://www.theregister.com/2022/04/07/google_makes_outdated_apps_less/
[6] https://www.theregister.com/2022/04/06/google_previews_biglake_data_lakehouse/
[7] https://www.theregister.com/2022/04/06/boeing_signs_new_cloud_deals/
[8] https://www.theregister.com/2022/04/06/google_south_korea_app_payments_illegal/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offprem/saas&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Yk9fFa@P1iNyB8SV2R2ECAAAAM8&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://whitepapers.theregister.com/
Terry 6
Absolutely. What (Google etc) search needs is a decent category restriction. If I want to find the specifications for [widget] I need search to give me that information. not a billion adverts for items that may or may not be said widget, with maybe a few thousand more for the search terms that I tried to use to limit the search results for the item, but entirely unrelated to my search. Google has, over the years, even made this harder- since it now tends to ignore search terms that seek to restrict results to something useful.
Neil Barnes
Like you, I am increasingly frustrated when I search and don't get anything that even includes the search word. A simple automatic AND would be useful - I'm sure they did it back in the dim and distant past.
zuckzuckgo
Lately, I have had Google respond a few times with: "It looks like there aren't many great matches for your search" with no results or just a short useless list. Repeat the same search with duckduckgo and get more results with something useful in the mix.
Is Google ignoring sites that don't produce revenue?
Oh goody. Yet another way to interpret search to find a lot of things I didn't ask for to bury the few that match - or hide the fact that there weren't any.