Data harvesting superapp admits it struggled to wield data – until it built an LLM
- Reference: 1727420167
- News link: https://www.theregister.co.uk/2024/09/27/grab_dataset_llm/
- Source link:
Grab offers ride-share services, food delivery, and even some financial services. In 2021 the biz [1]revealed it collects 40TB of data every day. Execs have [2]bragged that its fintech arm knows enough about its drivers that it can rate their suitability for a loan before they even bother applying.
In a Thursday [3]blog post , the developer admitted it has sometimes struggled to make sense of all that data.
[4]
"Companies are drowning in a sea of information, struggling to navigate through countless datasets to uncover valuable insights," the org wrote, before admitting it was no exception. "At Grab, we faced a similar challenge. With over 200,000 tables in our data lake, along with numerous Kafka streams, production databases, and ML features, locating the most suitable dataset for our Grabber's use cases promptly has historically been a significant hurdle."
[5]
[6]
Prior to mid-2024, Grab used an in-house tool called Hubble – built on top of the popular open source platform DataHub and utilizing open source search and analytics engine Elasticsearch – to sort through its giant data pile.
"While it excelled at providing metadata for known datasets, it struggled with true data discovery due to its reliance on Elasticsearch, which performs well for keyword searches but cannot accept and use user-provided context (ie it can't perform semantic search, at least in its vanilla form)," Grab's engineering blog explains.
[7]
Eighteen percent of searches were abandoned by staff users. Grab guessed the searches were abandoned because the Elasticsearch parameters provided by Datahub were not yielding helpful results.
[8]Grab – Asia's Uber – knows customers and drivers so well it can vet them for loans
[9]Ever wondered how much data web giants generate? Singaporean super-app Grab says 40TB a day
[10]Big Tech's maps led ride-sharing giant Grab astray
[11]Uber plans to ride out of stable Singapore, move APAC HQ to high-tension Hong Kong
But Elasticsearch wasn't the only problem to blame for laborious data discovery – oodles of documentation was missing. Only 20 percent of the most frequently queried tables had any descriptions.
The developer's data analysts and engineers were forced to rely on internal tribal knowledge in order to find the datasets they needed. Most reported it took days to find the right dataset.
Grab sought to rectify this through three initiatives: enhancing Elasticsearch; improving documentation; and creating an LLM-powered chatbot to catalog its datasets.
The Singaporean superapp enhanced Elasticsearch by boosting relevant datasets, hiding irrelevant ones, and simplifying the user interface.
[12]
Eventually it brought the number of abandoned searches to just six percent. It also built a documentation generation engine that used GPT-4 to produce labels based on table schemas and sample data. That effort increased the number of data sets with thorough descriptions from 20 to 70 percent.
And then it built the pièce de résistance: its own LLM. Called HubbleIQ, the LLM uses an off-the-shelf search tool called Glean to draw on its newly expanded descriptions and recommend datasets to its employees through a chatbot.
"We aimed to reduce the time taken for data discovery from multiple days to mere seconds, eliminating the need for anyone to ask their colleagues data discovery questions ever again," the superapp techies blogged.
The upgrades are a work in progress. Grab intends to work to improve the accuracy of its documentation and incorporate more dataset types into its LLM, in addition to other initiatives.
Grab's hyperlocalization strategy, which is enabled by its massive quantities of data, has given it the edge to know the ins and outs of Asia's people and roads – and frankly kept the business alive.
While its 2021 IPO results may have been [13]unquestionably disappointing , it did run [14]Uber out of town .
In Grab's Q2 2024 earnings, it [15]reported a record high of 41 million monthly transacting users, narrowing losses and 17 percent revenue growth.
"Features like mapping, hyper batching and just-in-time allocation, they're all unique to Grab and none of our competitors have that and we believe that makes us consistently more reliable as well as more affordable," [16]explained CEO Anthony Tan.
Consistently reliable, affordable … and drowning in datasets. ®
Get our [17]Tech Resources
[1] https://www.theregister.com/2021/08/03/grab_q1_2021/
[2] https://www.theregister.com/2022/09/27/grab_unconventional_data/
[3] https://engineering.grab.com/hubble-data-discovery
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZvaCRvii7QNBEmJqHXzkDgAAARM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZvaCRvii7QNBEmJqHXzkDgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZvaCRvii7QNBEmJqHXzkDgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZvaCRvii7QNBEmJqHXzkDgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2022/09/27/grab_unconventional_data/
[9] https://www.theregister.com/2021/08/03/grab_q1_2021/
[10] https://www.theregister.com/2022/06/01/grab_drops_google_maps/
[11] https://www.theregister.com/2020/05/27/uber_apac_hq_move/
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZvaCRvii7QNBEmJqHXzkDgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://www.theregister.com/2021/12/03/grab_ipo/
[14] https://www.theregister.com/2020/05/27/uber_apac_hq_move/
[15] https://investors.grab.com/news-releases/news-release-details/grab-reports-second-quarter-2024-results
[16] https://seekingalpha.com/article/4714870-grab-holdings-limited-grab-q2-2024-earnings-call-transcript
[17] https://whitepapers.theregister.com/
Re: 40TB a day
- Grab had over $2B revenue last year so $5,000 a week - $260,000 per year is nothing to them. I suspect the storage costs are of course many times higher - those disks need a chassis to put them in, power, cooling, networking etc...
- Biggest HDD's available today are 32TB. With 60 such drives in a 4U rack chassis, that's 1.92PB raw. Rack full of these can house more than a year's worth of data.
- There are also 60TB NVMe drives available as well. They're probably somewhat more expensive.
- That 40TB data probably isn't images, video or audio so it probably compresses and deduplicates well
In any case, any reasonable business would store this data with redundancy in mind so in 16 drive RAID6 they would lose 2 drives worth of space. Plus whatever spare drives they use.
...times two or three since the system is replicated to one or two other locations.
...plus the backups.
If they don't even know what they're collecting, then they can't know whether they should be collecting it. I guess Singapore doesn't have annoying regulations of that kind and the company apparently doesn't have a culture which asks such questions.
Do the maths..
41 million monthly transacting users generating 40TB a day is around 1 megabyte of data per user per day.
I'm going to go out on a limb here and suggest that the vast majority of what they're collecting is garbage. It sounds a lot like "data everywhere" culture has overtaken the company - and an LLM is a symptom not a solution.
That's not to say they don't have a serious scaling problem. Many companies would kill to have that many active users. And at that scale, any data insight can create useful improvements in efficiency. However, if the cost of finding that efficiency is a few million dollars worth of hardware and a large software team on the payroll, then you're just moving the inefficiency around rather than reducing it.
Perhaps not so surprising that they can have so many active users and still be struggling to break even..
40TB a day
If I'm not mistaken, that's over a petabyte of storage needed par month. The biggest HDDs you can get is apparently 16TB, so that's 3 disks a day, 21 per week. I hope they get a good price, because otherwise it's a budget that can reach over €5000 per week.
I checked SSD prices and capacity, but that would be insanely expensive at such a volume.
I'd like to see that storage room.