News: 1721371592

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

CrowdStrike code update bricking Windows machines around the world

(2024/07/19)


UPDATED An update to a product from infosec vendor CrowdStrike is bricking computers running Windows.

The Register has found numerous accounts of Windows 10 PCs crashing, displaying the Blue Screen of Death, then being unable to reboot.

“We're seeing BSOD Org wide that are being caused by csagent.sys, and it's taking down critical services. I'll open a ticket, but this is a big deal,” [1]wrote one user.

[2]

Forums [3]report that Crowdstrike has issued an advisory with a URL that includes the text "Tech-Alert-Windows-crashes-related-to-Falcon-Sensor-2024-07-19" – but it's behind a regwall that only customers can access.

[4]

[5]

An apparent [6]screenshot of that article reads "CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor. Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon Sensor."

CrowdStrike's engineers are working on the issue.

[7]

Falcon Sensor is an agent that CrowdStrike [8]claims "blocks attacks on your systems while capturing and recording activity as it happens to detect threats fast."

Right now, however, the sensor appears to be the threat.

This is a developing story and The Register will update it as new info comes to hand. ®

Updated at 0730 UTC to add:

Brody Nisbet, CrowdStrike's chief threat hunter, has confirmed the issue and on X [9]posted the following:

There is a faulty channel file, so not quite an update. There is a workaround... 1. Boot Windows into Safe Mode or WRE. 2. Go to C:\Windows\System32\drivers\CrowdStrike 3. Locate and delete file matching "C-00000291*.sys" 4. Boot normally.

In a later post he wrote "That workaround won't help everyone though and I've no further actionable help to provide at the minute".

[10]More here ...

Get our [11]Tech Resources



[1] https://twitter.com/McCurdy1987/status/1814164537815585049

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zpo5ObNNJNkpiq3ASKX6UwAAAcQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[3] https://www.reddit.com/r/crowdstrike/comments/1e6vmkf/bsod_error_in_latest_crowdstrike_update/

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zpo5ObNNJNkpiq3ASKX6UwAAAcQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zpo5ObNNJNkpiq3ASKX6UwAAAcQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[6] https://twitter.com/Xaaavier_8613/status/1814180533108400569

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zpo5ObNNJNkpiq3ASKX6UwAAAcQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://www.crowdstrike.com/products/trials/try-falcon-prevent/

[9] https://x.com/brody_n77/status/1814185935476863321

[10] https://www.theregister.com/2024/07/19/crowdstrike_shares_sink_as_global/

[11] https://whitepapers.theregister.com/



Related?

Andy 68

https://www.bbc.co.uk/news/articles/cv2g5lvwkl2o Multiple major Aussie systems down....?

Re: Related?

BartyFartsLast

Yeah, The Guardian and a couple of others are reporting it as a massive MS Windows issue and linking to the Crowdstrike fail.

https://www.theguardian.com/australia-news/article/2024/jul/19/microsoft-windows-pcs-outage-blue-screen-of-death?CMP=share_btn_url

Re: Related?

rafff

https://www.bbc.co.uk/news/live/cnk4jdwp49et Planes grounded as mass worldwide IT outage hits airlines, media and banks

Re: Related?

Phil O'Sophical

Who needs Y2K when you've got Microsoft?

Re: Related?

Peter-Waterman1

Let's test patches before we roll them out....nah, testing is for pussies.

Re: Related?

0laf

Testing? That needs testers fk that expense we have bonuses to think about.

Re: Related?

Valeyard

oh hey boss didn't know you were here

Re: Related?

Doctor Syntax

I'd hope that bonuses at Crowdstrike manglement are going to become a distant memory once all the damage claims have been paid but suspect the EULA will have protected them.

Re: Related?

Paul 195

To be fair, this one is not directly Microsoft's fault, it's Crowdstrike who've pushed something without adequate testing.

Re: Related?

Peter-Waterman1

Yes, crappy update, it happens; Microsoft release them all the time, and that's why you patch development and not go to production first without testing

Re: Related?

Anonymous Coward

It's Crowdstrikes Falcon sensor - i.e. a see security tool.

How many organisations test their security updates versus letting them be rolled out to all devices asap?

Re: Related?

Doctor Syntax

You can list them by discovering the businesses that have been hit today.

Re: Related?

Anonymous Coward

... and a Very Stable™ operating system?

Re: Related?

Alan Bourke

It's a Crowdstrike issue, not Microsoft. Try reading the article.

Re: Related?

Anonymous Coward

I've never heard of Crowdstrike before today. Is it something you get cheap with Azure or 365?

Re: Related?

Anonymous Coward

It's an IBM product. Basically the McAffee for enterprise - at least that's the sales pitch.

Re: Related?

Doctor Syntax

It's the Windows monoculture that has turned this into a global outage.

Re: Related?

Mr Dogshit

It's fuck all to do with Microsoft, you dodo.

It's clearly a kernel mode crash. No OS can cope with that.

Re: Related?

ttlanhil

It'd be possible to have a kernel that can catch a subset of crashes in kernel space (and maybe eject a driver or something), but more to the point:

kernel crash means a bug in kernel space.

Either MS's code has bugs itself, or has a security hole that allows other software to crash the kernel (yes, in reality it's both).

Unless you're using a non-MS kernel, MS has something to do with this.

And yes, the same could happen with other kernels, sure, but it usually happens with Windows

Re: Related?

katrinab

And most of the UK train operators

https://www.nationalrail.co.uk/service-disruptions/networkissues-20240719/

More being added to the list every time I refresh the page.

Re: Related?

gforce

Who would notice if UK trains were not running. Strike laden networks with abysmal track (sic) records of operation & punctuallity.

Re: Related?

Anonymous Coward

yeah, that selling everything cheaply to make it private owned really fucking helped, fuck thatcher and all her descendants and sycophants (blaire and starmer included)

Anonymous Coward

I remember when Starmer sold off Microsoft and Crowdstrike too.

Clown.

Re: Related?

vilemeister

And this is relevant to the current discussion how?

Just to get a mouth frothing political rant in?

Re: Related?

The Onymous Coward

Thatcher knew the railways should remain in public hands and refused to sell them. We've got Major to thank for that terrible decision.

And Blair/Brown

Steve Davies 3

Did nothing to rectify that mistake. They had 13 years to do it but no, they didn't.

All politicians are liars and hypocrites.

Re: And Blair/Brown

Evil Auditor

My memory's a bit blurry and may be wrong but I seem to remember that Blair not only didn't rectify but happily jumped and pushed on the privatisation bandwagon.

How the heck did we end up here, coming from bricked Windows machines?

Re: And Blair/Brown

Androgynous Cupboard

How would you rectify it exactly? Once it's sold it's sold, unless you renationalise. For governments that always an option, but given you're clearly no fan of Labour I presume you're to their right so I don't imagine this is what you had in mind...

Re: Related?

Phil O'Sophical

If you think it's bad now, wait until it's renationalised. However bad some companies are today, they're worlds better (with far higher passenger numbers now) than they were in the 1970s/80s.

Re: Related?

sedregj

"wait until it's renationalised"

Perhaps I will wait, rather than rely on your crystal ball.

Re: Related?

Steve Button

Yeah, over £100 to get into London from Suffolk. That's so much "better". I know a lot of things have gone up in price, but train tickets are highway robbery, but on rails.

Re: Related?

Mike 137

" train tickets are highway robbery "

Railway robbery, surely?

Re: Related?

bud-weis-er

yeah, not so fast, that "privatisation is great" argument has worn thin worldwide.

And the trains and service were way better back then.

Old Smith and Jones sketch

Mishak

Scene: Smith and Jones are sitting at a table on a British Rail train service. The drinks trolley approaches.

Trolly person: "Tea or coffee sir?"

Smith: "Darned if I can tell".

Re: Related?

nagi

Meanwhile the german Win 3.11 rail still runs probably :D

Re: Related?

Anonymous Anti-ANC South African Coward

Vunderschon! Ist sehr gut, ja?

*makes choo-choo noises*

Re: German rail

Steve Davies 3

A lot of the fans who went to Euro 2024 and traveled by train would disagree with you on the German rail system. Late or cancelled trains, overcrowding...

Does that seem familiar? DB in Germany, is just like BR in the UK, has/was been starved of investment by their government.

There was a long piece on the Euro 2024 issues on the BBC News site on Monday.

Re: German rail

Andy 68

DB now owns a significant chunk of the UK rail and bus services..... coincidence?

Re: German rail

Anonymous Coward

It makes headlines when one of the UK operators manages to run an efficient service.

I think many users

goldcd

will appreciate knowing the reason why there isn't a train this time.

Definitely an improvement on BAU

Re: Related?

richardcox13

Now top of the front page... doing a live update page: https://www.bbc.co.uk/news/live/cnk4jdwp49et

Re: Related?

abend0c4

Just had a call from my GP to cancel a vaccination I had booked for this morning (with a lead time of almost 4 weeks) as they're unable to access the patient record system and were using a printed list to phone round and cancel all non-urgent appointments.

I suggested that, for a scheduled vaccination, they could perhaps administer it anyway and make a paper note and update their patient records when they came back online but it seems to be beyond their comprehension that medicine might be practised in the absence of a functioning computer. And they could offer no assurance that it might not be another 4 weeks before an alternative appointment could be made.

I think my local surgery use EMIS, but I have no idea whether this is merely a local problem or more widespread.

However, given the increasing frequency of major system outages, it's depressing that service providers have such poor contigency plans when in many cases they could perhaps continue to function with a little forethought.

Re: Related?

Missing Semicolon

The Cloud has taught service providers that all these 9s of uptime are optional. "Computer says no" is now a valid excuse.

Apparently affecting MS worldwide.

Hazmoid

https://www.abc.net.au/news/2024-07-19/global-it-outage-crowdstrike-microsoft-banks-airlines-australia/104119960

appears this is world wide. I suspect this company is going to be in a world of hurt when the washout of this comes out. Fortunately it is late Friday afternoon for the East coast in Australia so I suspect there are going to be a lot of early marks taken. However you may be struggling to get a beer unless you have cash as a lot of banks and POS machines have been hit as well.

Re: Apparently affecting MS worldwide.

alain williams

However you may be struggling to get a beer unless you have cash as a lot of banks and POS machines have been hit as well.

Except for those idiot pubs that refuse to take cash and only take card payments.

Re: Apparently affecting MS worldwide.

Anonymous Coward

We went to one of those annoying pubs last week. Ordered 2 beers. Barman poured them. I offered cash. He said it was cards only. I gave him my Amex card. "We don't take Amex", he said. "All I've got", I lied and watched him pour the beer down the sink before I left and went to a decent pub. Can't wait to go back.

Re: Apparently affecting MS worldwide.

BartyFartsLast

Yeah, that really pisses me off too, I've walked out of a few places that refuse cash (which I believe used to be illegal?)

Re: Apparently affecting MS worldwide.

Dog Eatdog

The worst one for me is places that don't accept cash, and *still* add a 1.5% card fee.

Re: Apparently affecting MS worldwide.

I ain't Spartacus

Last Summer, I came across an ice cream van in Soho. It was hot. Got my 99 with Flake. Tried to pay with coins. Card only. We really are living in the future when ice cream vans are card only. How are kids supposed to pay?

Jay 2

Yep, this is pretty bad. Got a call from my colleague in HK (I'm in UK) at 0700 to ask if I could log in as they were having problems. I couldn't either. So I'll just twiddle my thumbs for now and let the Windows guys figure it all out. Hopefully it hasn't messed up all my Linux boxes!

All people are born alike -- except Republicans and Democrats.
-- Groucho Marx