Cloudflare Outage Knocks Many Popular Websites Offline
- Reference: 0180120719
- News link: https://tech.slashdot.org/story/25/11/18/120222/cloudflare-outage-knocks-many-popular-websites-offline
- Source link:
Update : In a statement after the outage was resolved, Cloudflare CTO [1]said :
> Earlier today we failed our customers and the broader Internet when a problem in Cloudflare network impacted large amounts of traffic that rely on us. The sites, businesses, and organizations that rely on Cloudflare depend on us being available and I apologize for the impact that we caused.
>
> Transparency about what happened matters, and we plan to share a breakdown with more details in a few hours. In short, a latent bug in a service underpinning our bot mitigation capability started to crash after a routine configuration change we made. That cascaded into a broad degradation to our network and other services. This was not an attack.
>
> That issue, impact it caused, and time to resolution is unacceptable. Work is already underway to make sure it does not happen again, but I know it caused real pain today. The trust our customers place in us is what we value the most and we are going to do what it takes to earn that back.
[1] https://x.com/dok2001/status/1990791419653484646
Critical Infrastructure (Score:2)
Step aside, US-1-East, there's another single point of failure to the internet!
(seriously though, have a fallback CDN. Much less effort than multi-cloud)
Re: (Score:2)
US-East-1
Brain not awake yet....
Re: (Score:3)
I'm posting 10 minutes after you and many sites that use cloudflare are still down despite you comment implying the outage has already been resolved. Some sites are back up, others aren't. In short it still isn't fixed.
Re: (Score:2)
congrats you can use twitter. that isn't the only website on the internet using cloudflare, but it is one of the most high profile ones. Its no surprise to me that it gets fixed first when they haven't gotten everything working again.
Re: (Score:3)
The majority of people don't care if X is down or not, we care about all the other sites that are down that are used for work.
Scheduled maintenence... (Score:2)
They have scheduled maintenance notices at several data centres that seem to coincide with the outage... coincidence or yet another self inflicted configuration screw up?
Re: (Score:3)
My money is on BGP cascade due to misconfiguration of reroutes for the maintenance
Re: (Score:3)
So, not DNS?
Too many eggs... (Score:5, Insightful)
... not enough baskets. Not the only place in tech where this is going massively wrong.
Re: (Score:1)
Time to run the cleaning ferret through the tubes again...
Re: (Score:2)
[1]https://bsky.app/profile/desig... [bsky.app]
n/t
[1] https://bsky.app/profile/designthinking.lol/post/3m5vux2an5s27
I can confirm for simons.ca (Score:1)
I tried to reload page opened yesterday and was greeted by bilingual message "Your security is our top priority. To continue on to our website, please check the box below." It lasted for a while but it looks like now access to Website is working. What a nasty piece of software.
Down in Boston ATM... (Score:2)
...and the error page helpfully provides a link to Cloudflare's site for more information...which is also unavailable
That explains it (Score:3)
I was wondering why a site I was on this morning decided to stop loading. Since other sites were still working I thought it might be the site itself. The hamsters got too tired and took a nap.
Touch dirt (Score:3, Insightful)
Seriously, that we have become so reliant on 100% 24x7x365 connectivity is chilling.
Re: Touch dirt (Score:3)
No, what's chilling is that with all the routing protocols we still have 2-3 single point of failure situations every year.
Re: (Score:2)
Electricity, water, transport... what's one more network?
What is chilling is that it's a single private subject that can bring this network down.
Re: Touch dirt (Score:2)
Well 2-3 private subjects but ok
Let's get the whole internet to us Cloudflare (Score:5, Insightful)
If all of the most popular websites use Cloudflare now the world's best hackers only have to take out one service instead of many.
X?! (Score:1)
Didn't musk brag about cancelling cloudflare?
Re: X?! (Score:3)
It shouldnâ(TM)t really be surprising that the compulsive liar was lying.
Carrington Event (Score:2)
You can tell that the next giant solar flare is going to flatten this whole house of cards.
Re: Carrington Event (Score:2)
That will be quite a day.
downdetector is also down (Score:1)
we need a downdetectordown site
Another internal DNS update failure (Score:2)
Those issue takes time to for the fix to propogate.
GTA got it right (Score:2)
[1]This radio ad from GTA 5 [youtube.com] sums it up perfectly:
> The future is now!
> The future is in the cloud!
> Cloud computing!
> GIRL: what's cloud computing?
> Imagine a computer you share with everyone
> imagine your private data spread around the world
> being shared equally with everyone
> it's the cloud
> GIRL: I'm in the cloud!
> It's utopia
> nothing can possibly go wrong
> imagine instead of your own computer
> it's a giant one we all share together
> your data is safe
> it's in the cloud!
> GIRL: Everyone is in the cloud!
> Live life lsurrounded by the mists
[1] https://www.youtube.com/watch?v=9GP0KDuzgBc
It's like the old unix fortune says: (Score:2)
# fortune
You attempt things you do not plan because of your extreme stupidity.
# _
Post-mortems (Score:2)
At least with Cloudflare, you know that there's going to be a thorough RCA and then post-mortem will be public. They might have gotten too big and too much of the internet is dependent on them, but the reasons why they got that big are pretty clear. There should be more competitors paying attention and emulating, as they do a lot right and having more people be open about good engineering would make for a healthy and more competitive space.
Having trouble with Slashdot too (Score:3)
I just had trouble looking at a comment on one of my posts yesterday because I can't get through the Cloudflare bot detector. I'm not sure why that is only used when looking at my comment history. It's funny that I first have that problem this morning on Slashdot just to see this story at the top of my news feed on the same site.
Re:Having trouble with Slashdot too (Score:5, Funny)
> ... including ChatGPT and X
Is there somewhere I can donate to keep this going?
Burning Expectations. (Score:2)
>> ... including ChatGPT and X
> Is there somewhere I can donate to keep this going?
I hear they burn mostly coal for fuel. Work to keep them and the Black Friday marketing offline! Coal for Christmas I say! A lump in every stocking should put a dent in their supply, and a damper on childhood arrogance.
Say it with me now! Ho Hell Nooo! Cooal for Christmas!
Re: (Score:2)
No need to donate, just install Windows XP.
Re: (Score:3)
I can confirm, slashdot isn't working for me either.
/jk , relax Maurice.
Re: (Score:1)
ChatGPT, Claude, Perplexity, and a bunch of others are down. The panic is hilarious.