News: 1594741331

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

BigQuery analytics goes multi-cloud: Use our service without moving your data, says Google – but there is no magic

(2020/07/14)


Google's virtual Cloud Next event has kicked off with the introduction of BigQuery Omni, letting customers extend the tentacles of the analytics service to probe data across Google Cloud, AWS and Microsoft Azure.

The service is currently in private beta for AWS and "very soon BigQuery Omni is going to be available for Azure," said Google's Data Analytics GM Debanjan Saha.

[1]

BigQuery Omni lets users use Google’s business intelligence and big data analytics service across data in multiple clouds

The thinking behind [2]BigQuery Omni is that most enterprises, whether by accident or design, find themselves using more than one public cloud. This is true of more than 80 per cent of organisations, said Saha. "When you are in multiple clouds the data is siloed, and if you want to run analytics on that siloed data, you have to move data from one cloud to another which is cumbersome and expensive," he added.

Note that data is moved temporarily within AWS from your data storage to the BigQuery clusters running on Anthos to execute queries

BigQuery Omni lets you use Google's analytics service on the cloud where the data sits. Features of BigQuery include building and running ML (machine learning) models, using an in-memory analysis service, and linking to location data for geospatial analysis.

The way this works is that Google uses Anthos – an application platform based on Kubernetes – to run its BigQuery service on other clouds. Customers will not have a dedicated BigQuery service on AWS or Azure, but will use Google's managed BigQuery as a shared resource. This enables use of SQL queries and other BigQuery features on these third-party clouds.

In the case of AWS, Saha said: "Behind the scenes, BigQuery's query engine is running on our Anthos clusters within the BigQuery managed service. BigQuery gets the data from data storage within your account once you've authorized permissions via your other public clouds' IAM roles.

"Note that data is moved temporarily within AWS from your data storage to the BigQuery clusters running on Anthos to execute queries."

Google does not envisage that users will attempt to query data stored in multiple clouds in one operation. The latency issues and data transfer required for this would make it inefficient. The idea is that users can query data in one cloud and export the results to another, and then perform further analytics against the combined data.

[3]

The control plane runs on Google Cloud Platform, but queries are carried out on the cloud where the data resides. If you need to query across clouds you should export the results of one query to the cloud where you want to query the joined data

An example would be where a customer wants to combine advertising data from Google Analytics with logs from an ecommerce platform running on AWS. Users can run a query on AWS, export the results to Google Cloud Platform, and then "build a dashboard that allows you to visualise your audience behaviour and purchases alongside your advertising spend," said Saha.

There is no magic and if the result sets were large, such an operation would still require some export of data from one cloud to another. That said, BigQuery Omni does mean that users have a single analytics system to learn, all controlled from a single user interface or API on Google Cloud.

Procter and Gamble (P&G) is one BigQuery customer mentioned by Google, with the claim that P&G will "transform its business by shifting from mass market to ...mass personalization."

It added: "By continuing to expand the implementation of Google Cloud's BigQuery data analytics technology, P&G can combine siloed customer data to enrich the shopping journey for existing customers, while also attracting new consumers to its product."

Those seeking to push out an "enriched shopping journey" will surely be delighted. ®

Get our [4]Tech Resources



[1] https://regmedia.co.uk/2020/07/14/bigquery1.jpg

[2] https://cloud.google.com/blog/products/data-analytics/introducing-bigquery-omni

[3] https://regmedia.co.uk/2020/07/14/bigquery2.jpg

[4] https://whitepapers.theregister.com/

Procter & Gamble

Pascal Monett

In 2019, it made over [1]$67 billion in revenue.

I should hope that, with that kind of financial ability, a company can make Big Data work.

I don't see how this is going to translate to a company that only makes millions. Big Data is a toy for the grown-ups. Kindergarden-level shops that barely break a million can forget about it.

[1] https://www.macrotrends.net/stocks/charts/PG/procter-gamble/revenue

A man is born gentle and weak.
At his death he is hard and stiff.
Green plants are tender and filled with sap.
At their death they are withered and dry.

Therefore the stiff and unbending is the disciple of death.
The gentle and yielding is the disciple of life.

Thus an army without flexibility never wins a battle.
A tree that is unbending is easily broken.

The hard and strong will fall.
The soft and weak will overcome.