Pimp my PostgreSQL: Swarm64 paints go-faster stripes on open-source database challenger
- Reference: 1588078877
- News link: https://www.theregister.co.uk/2020/04/28/swarm64_postgresql/
- Source link:
Ranked number four in the database market by [1]DBEngines , PostgreSQL is often left in the shade of fellow open-source stalwarts MySQL and MariaDB.
The firm hopes a performance boost to the DB - allowing it to analyse data orders of magnitude faster than the basic open-source PostgreSQL download can - will tempt users of proprietary alternatives to consider the open-source database as a viable, cheaper option, said Swarm64 co-founder and CEO Thomas Richter.
"Not only do we want to give people who are using Postgres a wider scope and easy way out of their performance and scaling issues, but in addition, we want people to consider Postgres for replacement legacy data warehouses, or for new developments, where they may not traditionally have considered Postgres," he said.
"Postgres is fantastic in the transactional space: that's really where it has built its popularity, but when you're trying to run complicated reports, to run heavy analytics, to slice and dice large data sets it is where Postgres falls behind those proprietary offerings."
This is where Swarm64 DA 4.0 comes in, he said.
The product does not require changes to users' SQL or application code, the CEO told us. It supports greater parallel processing by rewriting query patterns to execute in parallel at every phase of the query. It parallelises scanning, filtering, joining and merging, and spins up to 64 parallel threads.
Swarm64 DA compresses data by a factor of five to 25, the company claimed, depending on the data type. Besides reducing storage costs, reading compressed data, along with columnar indexing, Swarm64 reduces I/O by a factor of 20 relative to standard Postgres, it claimed.
Among other new features, the accelerator provides optional support for field-programmable gate array (FPGA) processors on premises or in cloud-based systems. "FPGAs are a cost-efficient way to further increase the acceleration effects of Swarm64 DA to handle bigger databases, real-time analytics, or greater concurrency," the firm said.
MariaDB cuddles Kubernetes, snuggles ServiceNow and readies analytics with latest update to cloudy SkySQL [2]READ MORE
The accelerator may appeal to those trying to migrate away from legacy Oracle databases in particular, as the Postgres and Oracle databases have always been very close, Richter said.
Another application where the accelerator offers benefits is in IoT data, he said. "Wherever you have large amounts of data and a high degree of parallelism: that's where we see ourselves as the performance enhancers."
The accelerator also helps Postgres handle a hybrid between transactional and analytics data where the user is analysing live data. As such, it has completed IoT projects with Toyota in connected cars, wind-turbine optimisation and predictive maintenance, as well as ad-tech and fintech analytics.
But the rival market in open-source databases is not standing still - competition remains intense. Just last month, MariaDB, which forked from MySQL, also added [3]features to support transactions and analytics on the same data . ®
Sponsored: [4]How to Build Your Digital Experience Portfolio
[1] https://db-engines.com/en/ranking
[2] https://www.theregister.co.uk/2020/03/31/mariadb_skysql/
[3] https://www.theregister.co.uk/2020/03/31/mariadb_skysql/
[4] https://go.theregister.co.uk/tl/1936/-8578/how-to-build-your-digital-experience-portfolio?td=wptl1936
I agree. Postgres is really impressive, but not well used or understood.
And the technical documentation is an absolute joy to use compared with some of the stuff out there.
But companies like EnterpriseDB will sell you pretty swish support (for ridiculously cheap compared to Oracle prices), and even guarantee drop in compatibility for Oracle (you can make the Postgres DB appear to everything connecting to it that it is an Oracle DB), full PL/SQL support etc.
Like you say, this FPGA kit could still be cheaper than Oracle or DB2 if you are a bank or other big company.
Postgres has seen continued uptake since MySQL was sold to Sun and particularly since Sun was sold to Oracle. Though this is somewhat ironic because Oracle is definitely taking better care of MySQL than MySQL AB ever did. The uptake has allowed companies like Enterprise DB, 2nd Quadrant and others to invest in Postgres which has helped make releases better and more predictable and added very important functions for larger installs. It now also has good support on the various cloud platforms for those companies who don't want to invest in DBAs and aren't worried about losing control of their data.
But I guess it's always going to be hard to find developers who are prepared to try and understand how relatiional database systems are supposed to work. Note, this doesn't necessary mean learning all the underlying relational algebra, but being able to define relationships logically and understand the flexibility of making projections from normalised data.
I've always preferred PG simply because there's no *NONSENSE* in setting it up - although some Linux distros put files in odd places [it seems], once you find them, it's not hard to configure.
(actualy did this recently, set up PG databases for testing on several machines, one running Raspbian on an RPi, another using FreeBSD on an RPi - FreeBSD was easier)
I've never really liked MySQL since it has non-standard "features" (read: bugs) with things like embedded quote mark handling, and I don't like the overly complicated setup process. OK matybe it got fixed within the 15+ years since I last looked at it but PG always seems superior to me, seems to be THE most standards compliant, is easy to back up and restore, and so on.
There's also something to be said about WRITING EFFICIENT QUERIES. There. I said it.
EXPLAIN ANALYZE is your friend but only if you've set up your database correctly. How many queries rely on stupid "optimisations" because data is nor normalised properly or indexed reasonably?
That said, SQL doesn't necessarily always make writing efficient queries easy, though Common Table Expressions do help take some of the pain out of using subqueries.
Ranked number four in the database market by DBEngines, PostgreSQL is often left in the shade of fellow open-source stalwarts MySQL and MariaDB.
Only by journos who don't know the difference: Postgres is streets ahead of MySQL in terms or functionality and reliability and the only open source choice for those who want to get off the Oracle bus. And Postgres does this all with a single codebase, ie. no need for different table types for "real" data, no tricy licensing.
Swarm 64 is really impressive for companies that need this kind of throughput – let's face it any FGPA-based system isn't going to be cheap – so banks, etc.