News: 1723211954

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Software innovation just isn't what it used to be, and Moxie Marlinspike blames Agile

(2024/08/09)


black hat There's a rot at the heart of modern software development that's destroying innovation, and infosec legend Moxie Marlinspike believes he knows exactly what's to blame: Agile development.

Marlinspike opened the second day of Black Hat with a talk that was ostensibly supposed to be a fireside chat with Black Hat founder Jeff Moss, but the Signal founder stole the show with an opening chat laying out a case for reclaiming the "magic" of software development that's been lost after 20 years. That loss, he argued, was due to stuffing developers into "black box abstraction layers" that strip them of the freedom needed to be innovative.

"Anybody who is managing an engineering organization will have some kind of management philosophy that is in some way downstream of, derivative of, in the zone of, or somehow related to agile," Marlinspike said.

[1]

Moxie Marlinspike (left) at Black Hat with event founder Jeff Moss (right) - Click to enlarge

Instead of allowing developers to operate from the bottom up in a way that lets them combine engineering expertise with the vision to see new capabilities in existing technology, agile teams end up siloed, working separately from each other, and without much visibility into what other teams are doing, he argued.

These black box teams also tend to lack visibility into some of the fundamentals of what makes their own products work, Thistle Technologies founder and CEO Window Snyder added later, during the Black Hat Locknote wrapup session.

[2]

Programming students aren't learning low-level languages, or how to interact with machine code, Snyder said - just high-level languages that make app development smoother, but leave engineers without needed context to understand how their pieces of the puzzle fit into a larger, vastly interconnected whole.

[3]

[4]

That, as Marlinspike explained in the morning, has left software engineers unable to do more than be derivative.

"The picture that I'm trying to paint here is that we spent the past 20 years onboarding people into software by putting them into black box abstraction layers, and then putting them into organizations composed of black box abstraction layers," Marlinspike said.

[5]Study finds 268% higher failure rates for Agile software projects

[6]Fragile Agile development model is a symptom, not a source, of project failure

[7]Agile Manifesto co-author blasts failure rates report, talks up 'reimagining' project

[8]Study backer: Catastrophic takes on Agile overemphasize new features

Understanding, Marlinspike asserted, "is the basis for most of the important developments and the history of discovery in software."

And where can such understanding be found, if not among the "ballooned" ranks of the large engineering organizations? Security researchers - duh.

Infosec pros to the rescue

While software engineering has spent the past few decades struggling to become quicker, more flexible and, by extension, more abstracted, security researchers have been doing the opposite, said Marlinspike.

"Security is the process of looking through abstractions in order to actually understand how things work, what's beneath them, and sometimes understand them better than the people who built them to begin with," he argued.

[9]

"What I'm trying to say is that without knowing it, I think you, the people in this room, have actually inherited their Earth," he continued.

[10]Study backer: Catastrophic takes on Agile overemphasize new features

[11]Agile Manifesto co-author blasts failure rates report, talks up 'reimagining' project

[12]ViperSoftX variant spotted abusing .NET runtime to disguise data theft

[13]US Army: We want to absorb private-sector AI 'as fast as y'all are building them'

There is magic in software development, Marlinspike maintained, saying that understanding how it works is analogous to mastering wizardry in the world of Harry Potter, where the talented can change the world with nothing but knowledge and a wand.

Infosec people, he continued, are akin to Hogwarts students who didn't actually hate their homework, unlike some main characters we could mention.

"[Security people] are the ones who've been sitting in the library, learning the spells, actually understanding how all of this works … like the Harry Potter world, the only thing that you need to make use of this knowledge is a computer," Marlinspike added. "And it doesn't even have to be a good one." ®

Get our [14]Tech Resources



[1] https://regmedia.co.uk/2024/08/09/moxie-marlinspike-jeff-moss-black-hat.jpg

[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/blackhatanddefcon&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZrY9H7e24v-rEphUv--RVwAAANI&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/blackhatanddefcon&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZrY9H7e24v-rEphUv--RVwAAANI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/blackhatanddefcon&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZrY9H7e24v-rEphUv--RVwAAANI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[5] https://www.theregister.com/2024/06/05/agile_failure_rates/

[6] https://www.theregister.com/2024/06/10/agile_opinion_column/

[7] https://www.theregister.com/2024/07/16/jon_kern/

[8] https://www.theregister.com/2024/08/07/agile_catastrophes_risk_undermining_the/

[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_specialfeatures/blackhatanddefcon&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZrY9H7e24v-rEphUv--RVwAAANI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[10] https://www.theregister.com/2024/08/07/agile_catastrophes_risk_undermining_the/

[11] https://www.theregister.com/2024/07/16/jon_kern/

[12] https://www.theregister.com/2024/07/10/vipersoftx_malware_dot_net/

[13] https://www.theregister.com/2024/07/02/us_army_private_sector_ai/

[14] https://whitepapers.theregister.com/



Misleading headline?

Anonymous Coward

Sounds like he's blaming over-abstraction, and happened to mention Agile for some reason.

I don't think Agile mandates anything about abstraction.

Re: Misleading headline?

JohnSheeran

Agile done extremely poorly without the benefit of all the components is what we are seeing I believe. In the corporate world, agile dev is a joke.

He's right about black boxes and over-abstraction though

may_i

I find a lot of the M$ C# development patterns totally impenetrable, web development in particular.

How anything works is hidden behind so many levels of abstraction and hidden code that you can't see and don't know how it works that doing anything with this stuff amounts to little more than making magic incantations in a language you don't understand and hoping you don't summon a major demon.

Tradeoffs

pip25

You have an idea for a service, or found a market gap you want to fill with your own solution. Obviously you don't want to release some unstable crap, but time to market is still of critical importance. Unsurprisingly, in such cases you won't be reaching for some kind of bottom-up, close-to-bare-metal solution. Your software will be much slower than what it could be, but that's the price you pay for getting something done as quickly as possible.

Not all projects are like this. But few of them lack deadlines altogether. Without the abstractions and "black boxes" that speed up development, these projects would not be better or worse, they would simply not exist.

Re: Tradeoffs

James Anderson

But developers are pushed to use frameworks which may make development a bit quicker but

make debugging and tuning a nightmare.

Who knows what really goes on inside ORM generated code?

Re: Tradeoffs

Rich 2

“But developers are pushed to use frameworks…”

l have written embedded stuff, multi-platform distributed PC apps, some pretty large and complicated web apps, etc etc and I have never used a “framework”

People use frameworks because they are either lazy or because they need to go back to school. Coding OpenGL from scratch is a nightmare, but STILL doable and you will learn a lot.

Has "Agile" become synonymous with "Development management" ?

JoeCool

I can agree on the issue of Silos, but there's no non-coincidental link to Agile.

For the Nth time, the Agile manifesto never stated

"do stupid, self limiting things" those are coming from other causes.

If we do not change our direction we are likely to end up where we are headed.