CrowdStrike deja vu as 'performance issue' leaves systems sluggish
- Reference: 1724350510
- News link: https://www.theregister.co.uk/2024/08/22/crowdstrike_deja_vu/
- Source link:
"CrowdStrike hits again," [1]noted one admin on Reddit, along with: "At least it's not on a Friday."
While it likely needs no explanation, the customer is referring to that [2]fatal Friday in July when a faulty file update inadvertently led to what may well be the [3]largest IT outage in history.
[4]
Another admin reports their organization "had performance issues being reported all day," including "delays and slowness when running things."
[5]
[6]
Luckily for the embattled security vendor and its [7]customers , however, there was no [8]blue screen of death this time around, nor does it appear that this remediation will [9]ruin any admins' weekend plans. CrowdStrike says it has now fixed the problem, and there's nothing to worry about.
The security biz posted its original [10]alert at 0850 UTC on Thursday:
CrowdStrike identified a performance issue with a cloud service that may cause degraded performance and boot times for some customer systems. This issue impacted customers in our EU-1 cloud. CrowdStrike Falcon customers are still protected.
CrowdStrike has scaled the impacted cloud service. Sensor performance is stabilizing. CrowdStrike is continuing to monitor the situation.
A status update from 1220 UTC said performance was returning to normal, and a spokesperson told The Register that it had fixed the issue.
"CrowdStrike identified and resolved a cloud performance issue this morning that had caused system delays for a small segment of EU cloud customers," a company spokesperson said. "This is not related to the Channel File 291 incident, and all customers have remained protected throughout."
[11]
The spokesperson declined to say how many customers were affected by the issue: "A nominal number of customers were affected."
[12]CrowdStrike president cheered after accepting 'Epic Fail' Pwnie award
[13]Angry admins share the CrowdStrike outage experience
[14]The months and days before and after CrowdStrike's fatal Friday
[15]CrowdStrike hires outside security outfits to review troubled Falcon code
The [16]Channel File 291 incident , of course, refers to the [17]flawed Falcon endpoint defense software update that [18]boot-looped millions of Windows computers worldwide last month.
CrowdStrike is now facing [19]legal threats from Delta Air Lines, which claims the IT meltdown cost it more than $500 million, as well as a [20]class-action lawsuit from investors for making false and misleading statements about its software.
Earlier this month at DEF CON, CrowdStrike President Michael Sentonas [21]accepted the Pwnie Award for Most Epic Fail and admitted, "we got this horribly wrong." ®
Get our [22]Tech Resources
[1] https://www.reddit.com/r/sysadmin/comments/1eyfex6/at_least_its_not_on_a_friday/
[2] https://www.theregister.com/2024/07/25/crowdstrike_timeline/
[3] https://www.theregister.com/2024/07/23/crowdstrike_ceo_to_testify/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zse1A4CrCEaDy3krv7MmoAAAARY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zse1A4CrCEaDy3krv7MmoAAAARY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zse1A4CrCEaDy3krv7MmoAAAARY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2024/07/19/admin_crowdstrike_update_mess/
[8] https://www.theregister.com/2024/07/19/crowdstrike_falcon_sensor_bsod_incident/
[9] https://www.theregister.com/2024/07/19/crowdstrike_windows_kettle/
[10] https://www.reddit.com/r/crowdstrike/comments/1eyia2o/tech_alert_eu1_performance_degradation_20240822/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zse1A4CrCEaDy3krv7MmoAAAARY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2024/08/12/crowdstrike_president_cheered_following_acceptance/
[13] https://www.theregister.com/2024/07/19/admin_crowdstrike_update_mess/
[14] https://www.theregister.com/2024/07/25/crowdstrike_timeline/
[15] https://www.theregister.com/2024/08/07/crowdstrike_full_incident_root_cause_analysis/
[16] https://www.theregister.com/2024/08/07/crowdstrike_full_incident_root_cause_analysis/
[17] https://www.theregister.com/2024/07/19/crowdstrike_shares_sink_as_global/
[18] https://www.theregister.com/2024/07/29/microsoft_crowdstrike_kernel_mode/
[19] https://www.theregister.com/2024/08/08/delta_crowdstrikes_offer_for_help/
[20] https://www.theregister.com/2024/08/01/crowdstrike_lawsuit/
[21] https://www.theregister.com/2024/08/12/crowdstrike_president_cheered_following_acceptance/
[22] https://whitepapers.theregister.com/
Re: Hmmm…
From personal experience in times past, I refuse to believe that any endpoint protection software can have more of a performance impact than Symantec/Norton Antivirus.
*shudder*
cloud issue?
Why would Crowdstrike be using 'cloud services' on a customer's machines? I thought they provided low-level anti-malware scanning and suchlike. They shouldn't be exposing their code to stuff in the cloud IMHO.
Re: cloud issue?
> Why would Crowdstrike be using 'cloud services' on a customer's machines?
It's cheaper than maintaining their own hardware. Besides, isn't it possible for the innovators to design an OS that's runs from read-only media. With the apps run on a virtual machine. That disappears into the æther on reboot.
Re: cloud issue?
The article does rather make it sound as though CS Falcon is signature-matching files against white/blacklists in the cloud, meaning that slow cloud performance slows down local performance.
I'm sure - or at least, before July's outage exposed their incompetence, I would have been sure - there's some caching of those lists in play though.
Hmmm…
Crowdstrike is a permanent performance issue in my experience. I wish our organisation would drop them.