'Agile' F-35 fighter software dev techniques failed to speed up supersonic jet deliveries
- Reference: 1616692649
- News link: https://www.theregister.co.uk/2021/03/25/f35_gao_report_fy2020_software_woes/
- Source link:
The US Government Accountability Office (GAO) said in its annual report into F-35 design and development that software development practices within the F-35 Joint Project Office (JPO) and jet manufacturer Lockheed Martin were below par – and had hindered the supersonic stealth fighter's progress.
[1]
"The program's primary reliance on the contractor's monthly reports, often based on older data, has hindered program officials' timely decision making," said the GAO. "The program office has also not set software quality performance targets, inconsistent with another key practice. Without these targets, the program office is less able to assess whether the contractor has met acceptable quality performance levels."
The program office has also not set software quality performance targets, inconsistent with another key practice
[2]
The GAO recommended that Lockheed and the US Department of Defence both pull their socks up, sighing: "For over 20 years, we have consistently emphasized the need for organizations to collect and use data about program performance to help inform and measure organization operations and results."
Lockheed and the JPO adopted an Agile-style methodology that the US military branded C2D2, or Continuous Capability Development and Delivery. This has been less than stellar, though its intent was to allow iterative development of the jet's capabilities, rather than delivering an all-singing all-dancing aircraft all at once – and taking decades to do so. So far C2D2 has cost $14bn, a portion of which will have been paid by F-35 customers including the UK, which intends buying at least 48 of the jets to fly from its Queen Elizabeth-class aircraft carriers.
Knock-on effects from these poor software development practices were very real. Ground-based simulators used for pilot training weren't a good enough representation of the real aircraft, the GAO [3]said [PDF] in its annual report, while developers "routinely underestimated the amount of work needed" to develop new software loads for the world's most expensive supersonic stealth fighter.
[4]China compromised F-35 subcontractor and forced expensive software system rewrite, academic tells MPs
[5]So long, Top Gun... AI software waxes US F-16 pilot's tail 5-0 during virtual dogfight drills
[6]US Air Force wants to pit AI-powered drone against its dogfighting hotshots in battle of the skies next year
[7]ALIS through the looking glass: F-35 fighter jet's slurpware nearly made buyers pull out – report
[8]Easy-to-hack combat systems, years-old flaws and a massive bill – yup, that's America's F-35
US website Defense News [9]reported that the F-35 JPO's plans to push half-yearly software updates for the F-35 fell into disarray after planned "increments" within those builds spiralled out of control.
While each build was supposed to consist of four "increments", the June 2020 edition came with no fewer than 10 – and contractors let slip to the watchdog that four of those "were to address software defects" instead of adding new functionality as promised.
October 2020's build had eight increments, of which four were purely bug-squashing items.
Separately from the GAO report, US magazine Aviation Week [10]reported , citing its own sources, that a bug which would have disabled the radars on all F-35As came "uncomfortably close to being released" to the US Air Force early last year.
[11]
In domestic F-35 news, the UK seems set to [12]slash its order for 138 jets as defence chiefs start eyeing up [13]domestically built unmanned aircraft instead. ®
Get our [14]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YF0WD5XIBhHfZNkgP2dVaQAAABA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YF0WD5XIBhHfZNkgP2dVaQAAABA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[3] https://www.gao.gov/assets/gao-21-226.pdf
[4] http://www.theregister.com/2020/11/12/china_compromised_f35_software_claim/
[5] http://www.theregister.com/2020/08/21/ai_beats_f16_pilot/
[6] http://www.theregister.com/2020/06/09/us_military_drones/
[7] http://www.theregister.com/2019/06/13/f35_alis_us_data_slurping_fears/
[8] http://www.theregister.com/2019/03/28/f35_software_fail/
[9] https://www.defensenews.com/air/2021/03/23/f-35-program-not-moving-quick-enough-to-get-software-out-on-time-congressional-watchdog-finds/
[10] https://aviationweek.com/defense-space/aircraft-propulsion/software-bugs-rattle-us-air-force-f-35-block-4-progress
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YF0WD5XIBhHfZNkgP2dVaQAAABA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://ukdefencejournal.org.uk/uk-reportedly-to-cut-f-35b-order-by-65-percent/
[13] https://www.naval-technology.com/news/royal-navy-project-vixen-exploring-potential-carrier-uas/
[14] https://whitepapers.theregister.com/
Which is often what agile ends up delivering.
I think it has its place if done in a certain way but it's often sold as a silver bullet and then implemented religiously and inappropriately.
I also get a bit fed up being told that traditional "waterfall" projects spent a year analysing what was needed and then came back to the business three years later with something they no longer wanted. What a load of rubbish, I can only think of one government project which came close to that. All other projects kept in regular contact with the business and failures were usually due to issues in the business... moving goal posts, changing management whims, planned resources being put on other work at the last minute etc. etc.
The whole waterfall image also present a false image of one slow thing after another. Again, not true in my experience. You always look at what can be done in parallel, or if things can be moved around if you're blocked on a point.
Oh well, rant over :-)
Basics
Maybe somebody should adopt whatever the IT people used getting the Apollo program to the moon. Just a suggestion. Who knows, might stop aircraft falling from the skys. Complexity doesn't have to equate to impossibility, it just sometimes take care and a bit of time.
Re: Basics
The agile methodology can't fix graft and fraud. The Apollo project succeeded in part because of a genuine will to succeed. The F-35 project was designed to fail by going over-budget. When that became clear and the project wasn't cancelled, they doubled down. The later it is the more money they make on it.
Unfortunately because the lack of urgency isn't just impacting cost, what is being built has slipped in terms of both time and capability. It's decades late instead of it's development being sped up, it can't meet the requirements that were laid out 20 years ago, and by the time it is operating semi-reliably, it will be obsolete in almost every meaningful way. The trade-offs of fighter performance vs obervability by dotcom era radar systems don't look good 20 years later. And it will still be eye wateringly expensive.
By the time the F-35 is combat ready, it's operators would be better served having put the money into combat UAVs and support craft.
Re: Basics
from Wikipedia "The Apollo flight computer was the first computer to use silicon IC chips. ... The computer had 2048 words of erasable magnetic-core memory and 36 kilowords of read-only core rope memory. Both had cycle times of 11.72 microseconds. The memory word length was 16 bits: 15 bits of data and one odd-parity bit."
Which is to say, nowhere near as powerful as an Apple-IIe or the original IBM-PC. Probably coded by one person or a handful of people in assembler. Probably not capable of managing most of the subsystems in a 1999 Toyota Camry much less those in a modern fighter-jet.
We've moved on a bit.
Not that there isn't a lot to be said for keeping things a simple as possible.
As for Agile. Haven't tried it or seen it tried, but creating satisfactory mission critical software is not one of the things I would expect it to be capable of.
"domestically built unmanned aircraft"
Much better suited to future UK armed forces staffing levels. There'll be enough Air Commodores to fly the RAF versions and enough Admirals to fly the Naval ones. Finding enough ground staff for servicing might by tricky.
>Finding enough ground staff for servicing might by tricky.
Which was solved by cancelling the landing gear option
>>>Air Commodores
SX-64s ?
Here's a more sobering stateside opinion of HMS WhiteElephant:
https://thehill.com/opinion/national-security/542290-royal-navy-in-the-pacific-an-ally-against-china-where-we-need-it
"and U.S. pilots will comprise between a third and a half of the Brits’ strike-capable air wing."
Rule Britannia.
Figures
And the more I see the results of "design on the fly" (pardon the pun) in mainstream products like all of the 157 Microsoft portals, it's clear to me "agile", "CI/CD", and all the other cool buzzwords around this philosophy simply hide the fact that no one is thinking about the end before they begin. At least with waterfall you had to consider what "finished" might look like. Now? It's a dogs breakfast with constantly changing UI, levels of abstraction that make no sense, duplicate ways of doing similar things - none of them well thought out - and generally no cohesiveness at all to the product. And that's just O365, but I've seen it in other "go fast, break things" products as well.
Good engineering and design is hard. Being able to change your mind - and the product - every time it becomes clear you haven't thought something through, or living in your own little world and not thinking about how what you're doing interacts with the larger whole, well, that's easy. It doesn't require thought or imagination and it allows you to keep pushing difficult design decisions down the road until there's no way out but to start all over now that you have some clue what you are doing. How about we just admit that "fast" isn't the end-all. Real engineering, real design matters too. And thinking about a usable end state and what it will take to get there needs to be part of the plan.
I used to work with guys that thought of their designs as something that could be "elegant", and I saw that elegance more than once. Haven't seen it out of this new methodology, and I doubt that I ever will. Now get off my lawn.
Re: Figures
..and that elegance often leads to adaptability meaning cost savings in the long run.
Re: How about we just admit that "fast" isn't the end-all.
Agree with the rest of your comment but I'd argue Agile, despite the name is slower vs. waterfall if you know what you want. It is suited to projects where the goals/capabilities/behaviours of the resulting product are unclear at the start. If you know what you need just design and build it without all the iteration and wasted time.
For a fighter plane, surely it would have been quicker overall to decide what it could do and design and implement according to that?
UnAgile
Too many forget that you need to think seriously and actually have a solid idea before having code wranglers at a project. I have some nominally 'agile' projects come past me with virtually no specifications were someone has thought through the scenarios and have a general idea of how they should be handled. It makes for code that is impossible to properly maintain when it is found out that major use cases were never considered in the design given to the programmers. Use cases never considered by the programmers because they did not know about them.
Agile does not mean there are not design documents and preplanning but that groups are in contact with each other throughout the process. Issues are dealt with as they arise not after the fact.
No blaming software will ever making up for borked procurement practise. A supply chain of tens of thousands of employees leads to high unit cost. Keep it simple, keep it cheap
Phew!
Lucky we didn't design an entire aircraft carrier so it could only fly this type of fighter
Re: Phew!
>we didn't design an entire aircraft carrier so it could only fly this type of fighter
But then built a pair of aircraft carriers that couldn't actually fly this type of fighter
Re: Phew!
You didn’t design _an_ entire aircraft carrier. You designed _two_ entire aircraft carriers. And will be living with the decision for 30-40 years.
Come, cheer up, me lads, for ‘tis to glory we steer…
Re: Phew!
... heart of oak are our ships, brains of oak are the MoD...
At least only costs $36,000 / hour to operate
It's only tax payers money after all.
Hey, let's reinvent the wheel using untested code and stick Uncle Sam with the costs. And be sure to put so much bloat in the code it will take years to find the buggy bits. Our training at the microsoft school of programming has really paid off.
From what I have seen, the development methodology they used was not actually 'Agile'. It was one similar in principle to Agile but mostly reserved for tasks relating to student groupwork. Its technical name goes by 'Clusterfsck'.