News: 1729510571

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Developer pockets $2M in savings from going cloud-free

(2024/10/21)


The web software biz that decided to exit the cloud after racking up a huge bill says it has saved almost $2 million in its first "clean year" after making the switch to on-prem, and has already recouped the costs of the extra hardware it needed.

A couple of years ago, 37signals was aghast to find it had run up charges of $3,201,564 on cloud services, a large chunk of which was going to Amazon Web Services (AWS), as The Register [1]detailed previously .

The developer of project management platform Basecamp then embarked on a plan to invest in new servers and gradually migrate its applications and customer data to its own infrastructure instead.

[2]

In the latest [3]update on LinkedIn , CTO David Heinemeier Hansson said 37signals successfully moved seven cloud apps, including its email and calendar tool HEY, off AWS and onto its own hardware last summer.

[4]

[5]

As it took until the end of the year for various contract commitments to expire, 2024 has been the first clean year of savings, and according to Hansson, "we've been pleasantly surprised that they've been even better than originally estimated."

In fact, the cloud bill for 37signals now stands at about $1.3 million, a reduction of almost $2 million per year, and the savings are likely to be more than the company's original estimate of $7 million over five years as it managed to fit the new hardware into its existing datacenter racks and power restrictions.

[6]

Speaking of new hardware, 37signals bought about $700,000 of Dell systems to replace its cloud instances, but Hansson claims this cost was "entirely recouped" during 2023 as those contract commitments expired one by one.

"Think about that for a second. This is gear we expect to use for the next five, maybe even seven years! All paid off from savings accrued during the second half of 2023," he commented.

The remaining $1.3 million that 37signals still spends on cloud services is all going on AWS S3, as part of a four-year contract that doesn't expire until next summer, when the web software biz plans to ditch that as well.

[7]

While 37signals stores almost 10 PB of data in S3 currently, it plans to replace it with a dual datacenter configuration for redundancy based on Pure Storage with a combined 18 PB of capacity. This setup is expected to cost about the same for the initial hardware as a year's worth of AWS S3, according to Hansson.

[8]AWS claims customers are packing bags and heading back on-prem

[9]40k servers, 400k CPUs and 40 PB of storage later... welcome to Google Cloud

[10]Duke Uni libraries decamp from 37Signals' Basecamp over CTO's blogs

[11]37 Signals says cloud repatriation plan has already saved it $1 million

"This brings our total projected savings from the combined cloud exit to well over $10 million over five years, while getting faster computers and much more storage," he said.

Perhaps this is the kind of situation AWS had in mind when it told the UK's Competition and Markets Authority that it was [12]facing stiff competition from on-premises infrastructure .

That claim was in response to the regulator's investigation into Britain's cloud services market, and whether the big players such as Amazon engage in practices that may limit customer choice.

AWS argued that "building a datacenter requires significant effort, so the fact that customers are doing it highlights the level of flexibility that they have and the attractiveness of moving back to on-premises."

However, even Hansson conceded that when comparing cloud to on-prem, "it's never fully apples-to-apples," and will obviously depend on an individual organization's circumstances, such as what infrastructure it owns and the applications and services it requires.

One analyst previously told us that while such "cloud repatriation" projects are becoming more common, "we'd put the share of companies actively repatriating public cloud workloads in the single digit percentage sphere."

But it is still remarkable that 37signals has been able to secure such savings from leaving the cloud, Hansson noted.

"We've been out for just over a year now, and the team managing everything is still the same. There were no hidden dragons of additional workloads associated with the exit that required us to balloon the team, as some spectators speculated when we announced it." ®

Get our [13]Tech Resources



[1] https://www.theregister.com/2023/01/16/basecamp_37signals_cloud_bill/

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZxZ6pQrroCZoV3csRxf1cQAAAIU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[3] https://www.linkedin.com/posts/david-heinemeier-hansson-374b18221_our-cloud-exit-savings-will-now-top-ten-million-activity-7252755548859727874-k5V2

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZxZ6pQrroCZoV3csRxf1cQAAAIU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZxZ6pQrroCZoV3csRxf1cQAAAIU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZxZ6pQrroCZoV3csRxf1cQAAAIU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZxZ6pQrroCZoV3csRxf1cQAAAIU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[8] https://www.theregister.com/2024/09/17/aws_cma_investigation/

[9] https://www.theregister.com/2024/02/27/moving_thousands_of_servers_to_google_cloud/

[10] https://www.theregister.com/2023/12/01/duke_university_libraries_bandcamp/

[11] https://www.theregister.com/2023/09/18/37_signals_cloud_repatriation_savings/

[12] https://www.theregister.com/2024/09/17/aws_cma_investigation/

[13] https://whitepapers.theregister.com/



b0llchit

I wonder how much the BOFH charges to co-loc/on-prem the installation from the BOFH cloud datacenter to the BOFH co-loc/on-prem datacenter.

Controls and discipline

Anonymous Coward

Where are your cost alerts and someone to police cloud usage? I'm a firm believer in "horses for courses", you choose cloud, hybrid or onprem, but if you open cloud accounts to one and all then it's just giving an unlimited company credit card to all employees, don't complain when someone does something stupid with it!

We have sandboxes for our devs, max spend limits, we track costs and usage every 15-20 mins, we even restrict VM usage to max 2 CPUs and 16GB memory, max 2 machines per person. We don't allow databases, we dont' allow unlimited storage. Only about 6 people in the whole company are allowed into the test and prod cloud accounts out of 700 employees. We have hourly checks for costs exceeded, daily round ups, name and shame on spending, you spin it up and you're name will be on display for all to see so choose wisely.

Only 40 people in the company get a company credit card, like wise we don't let everyone in IT play in the cloud toybox!

Re: Controls and discipline

yoganmahew

It sounds like hell. Have you considered not having computers at all? Going back to pen and paper?

The problem for SMEs is that the cloud discounts are what makes cloud affordable for standard operations. Only the big players get the big discounts. Everyone else spends their lives chasing random cost instead of making new things (or fixing existing things). My own small cloud application (a few hundred TPS) costs next to nothing to run on GCP native services, mainly because it is part of a large GKE cluster and there are swinging discounts. If you're not operating at scale and you just lifted and shifted, you're almost certainly not going to get cost benefits.

Re: Controls and discipline

Roland6

Just moved a small low usage application off AWS to an independent, the typical £500~600.pcm AWS cost has been replaced by a flat rate £200 pcm.

Additional, support and maintenance savings will accrue as the migration took it off AWS services and onto our own LAMP platform (MariaDB instead of MySQL), which we can better manage the patch and version upgrades.

Yawn

Stu J

Businesses that run large, relatively stable, predictable workloads are likely to save money on-prem, long-term.

But they do sacrifice a degree of agility. If their customer base was to triple over the next three weeks, could they source, configure, and install enough hardware to cope with the additional load? Unlikely.

If they wanted to develop a new product with as yet unknown resource requirements, would they have enough spare capacity to do that, without affecting their production workloads, and without having to down tools to wait for new hardware to arrive? Because I've experienced this exact situation when relying on on-prem tin.

37Signals is literally at one end of the spectrum. The counter-argument at the other end of the spectrum is the UK charity Comic Relief, who provisioned serverless infrastructure that facilitated the processing of tens of millions of pounds in donations from hundreds of thousands of members of the public, for less than £100; their previous on-prem solution was IIRC 1000x more expensive, and sat doing nothing for the vast majority of the year.

Everyone else is somewhere in the middle. You don't have to run all of your workloads in the cloud; you don't have to run them all on-prem. There is a middle ground, but anti-cloud zealots like DHH seem to forget this and present their own experience as "look, this is true for us so cloud sucks".

Re: Yawn

Anonymous Coward

It should be a pretty well paid job to be an anti-cloud zealot like DHH if the savings can be that big.

Re: Yawn

Doctor Syntax

"But they do sacrifice a degree of agility. If their customer base was to triple over the next three weeks, could they source, configure, and install enough hardware to cope with the additional load?

...

Comic Relief, who provisioned serverless infrastructure that facilitated the processing of tens of millions of pounds in donations from hundreds of thousands of members of the public, for less than £100"

Comic Relief hold an annual events on dates decided well in advance. If they depend on agility to plan for that they're doing it wrong.

I have, in the past, worked with marketing who probably did hope for something like a tripling of turnover in three weeks. Having failed to work out that their campaign had a small flaw they were disappointed. (It was intended to let them sell direct instead of through distributors; the customers needed to arrange an installion, however, and the installers were also the distributors.)

Re: Yawn

Roland6

> 37Signals is literally at one end of the spectrum. The counter-argument at the other end of the spectrum is the UK charity Comic Relief

Odd spectrum, I suspect if you ignore the annual telethon, their normal day-to-day operations load is boring and predictable. Thus, just like 37signals there will be few benefits from running this load in the cloud, although with charity pricing, I expect they will be a big user of 365 and cloud-based application as a service systems for accounting, customer relationship etc.

Also I suspect the annual telethon is such that they can get their cloud provider to donate the costs…

Groo The Wanderer

Remember that cloud computing is a for profit business, and they certainly won't settle for making 10 cents per dollar.

The "cloud" is just using someone else's hardware. There really is no benefit for an organization with a functioning brain cell in management to go cloud.

But functioning brain cells is a big ask for most corporate management...

Pete Sdev

Depends on how bursty or stable the load is.

Also depends on access (and budget) to the necessary skilled employees.

Anonymous Coward

For most organisations with some critical mass (and a baseline workload), cloud is going to be more expensive for that element of what they do, but for smaller organisations, who can't fill a modern server, cloud likely is a good shout.

For bursty or massive spike workloads, cloud can be a great fit for elastic capacity on a temporary basis, but the MO needs to be "kick the arse out of it, then get back home" as soon as the workload drops.

To my mind, sizing on prem for the "once in a year" workload if you're Wimbledon, or putting all workloads in the cloud just because you're planning to run a big marketing campaign once a quarter, are equally bad approaches.

F. Frederick Skitty

My current employer have been doing a costing exercise to see if on prem could save us money versus our AWS setup. We don't have "bursty" activity, just a steady steam of data processing and web traffic, so estimating the hardware requirements is straightforward. It also helps that every AWS feature we use can be replaced with an on prem equivalent - PostgreSQL, OpenSearch, Redis and storage to replace S3 essentially. Had we bought into the serverless snake oil like Lambda or other dubious AWS features I'm not sure we'd be considering a transition.

Anonymous Coward

You make an excellent point, which I'm always surprised people miss - don't get locked in.

You'd have thought that IT had enough examples of people having buyers' remorse about lock-in being painful/expensive ***cough, ORACLE, cough***, that we'd have learned to stop doing that, but far too many people don't appear to pay enough attention to history and therefore are happy to take the short term easy option and suck it up/regret at their leisure when they're bent over by the vendor.

Pink fluffy handcuffs are still handcuffs......

Doctor Syntax

IT people have experienced those examples. You can recognise them. They're the ones burying their heads in their hands when the decision makers meet the salesmen.

What always surprises me is that the manglement are so often snake -oil merchants themselves so why can't they recognise the same thing when they see it?

It's not just about scale, it's also...

Anonymous Coward

...the value of somebody else handling the operational complexity.

Take S3, he says they want to do this themselves? I'll assume he's not actually talking about multi-regional - even maybe multi-zonal S3?

So let's say you do this only in one location... do you use a SAN? Probably not because then you have another expensive vendor contract you didn't want.

So lets implement it as a service on utilising storage across your on-premise servers - I take it you've got all the servers RAID configured properly and monitored - or are you replicating across multiple machines instead?

Either way hardware fails, so you've got your on-call processes sorted, you've got your contracts with your hardware suppliers locked down because you know you'll be replacing failing disks with some kind of regularity.

You've either trained the datacenter staff or you have staff present near your datacenter location, and their access sorted to be able to go in and replace those failing disks.

Depending on how much you're leaning on the on-premise provider (in which case how much money are you really saving?) you need an entire ops team likely with a different skillset from your existing cloudops team.

Re: It's not just about scale, it's also...

Anonymous Coward

"I'll assume he's not actually talking about multi-regional - even maybe multi-zonal S3?"

You can assume what you like, but considering his company provides SaaS apps as their product, and they tend not to work well when the infrastructure is down, I think the implied assumption he's going single site is bat-shit crazy.

He does actually cover lots of the concerns you raise in his article, and some others in the comments to his LinkedIn post.

They are still saving millions per year after paying for smart hands, architecting for in-site / cross-site redundancy and full support from vendors like Dell and Pure (so likely no disks to change - it will all be solid state, so most likely not failing with "some kind of regularity").

Finally, they didn't change the number of people supporting the infrastructure - so they're either super human, or the state of managing infrastructure has improved since you last did it.

Your "let's say" thought experiment is amusing, but I'm paying more attention to the guy who is already doing it for real and whose company is pocketing seven figures of savings per year (even before he does the same on the S3).

Re: It's not just about scale, it's also...

yetanotheraoc

"somebody else handling the operational complexity"

That's the dream, isn't it? No complexity anywhere, or as Hans says in _Die Hard_: "We'll be sitting on the beach, earning 20%." And as everybody knows, there are no mosquitos in paradise. Whereas in the real world, managing suppliers is just as much work as managing employees. The complexity moves around, but it doesn't go away. This must be true, because if it were possible to outsource all the complexity, then anybody could be in the same business as you. Er, wait, that's what MBAs think already. Never mind, puts hand down.

"It is hard to overstate the debt that we owe to men and women of genius."
-- Robert G. Ingersoll