Home Office slams PNC tech team: 'Inadequate testing' of new code contributed to loss of 413,000 records
- Reference: 1622017810
- News link: https://www.theregister.co.uk/2021/05/26/home_office_slams_pnc_tech/
- Source link:
Although the lost data has all been recovered, [1]according to Kit Malthouse, minister for crime and policing , an independent investigation into the incident, led by Lord Bernard Hogan-Howe, has found that established procedures, such as reviews, were only loosely followed or in some cases not followed at all, during the introduction of the faulty script. Hogan-Howe is a former head of London's Metropolitan Police.
The incident, [2]which left Prime Minister Boris Johnson floundering in Parliament in February , was the result of a scripting error in code designed to weed out records to be deleted from the PNC, a 43-year-old system running on a Fujitsu BS2000/OSD SE700-30 mainframe-based at a data centre in Hendon, London.
[3]
[4]
[5]
The team introducing the code did not have fully defined business requirements and was not maintaining an accurate record of tests undertaken, the report found.
Meanwhile, there was a failure to "design effective and complete tests, including for affected systems, and what appears to be a significant failure of the manager whose responsibility it was to thoroughly review the testing procedures prior to approval. It appears this was not done," the report said.
"Testing of the new code was inadequate and specific processes within the change were not tested at all. It is concerning that the supporting documentation for these recent actions cannot be found," according to the report, produced on 19 March.
[6]
Also among the criticism was an apparent cosiness among the ageing PNC’s operational team. "The team who operate it have worked together over a long period of time. The expertise and closeness of the teams involved in running the PNC increased the risk that their work would be accepted rather than checked by a leadership that were in a poor position to challenge their decision making. The PNC services team has very limited police experience in the team and have limited understanding of how the police operate," the report said.
However, the report said the age of the PNC should be considered. Its replacement, the National Law Enforcement Data Programme (NLEDP), which enables the decommissioning of PNC, is undergoing a fundamental reset, although some projects in the programme were meant to have gone live last year.
[7]Big fat doubt hovers over UK.gov's Making Tax Digital, customs declaration IT projects
[8]IT bod flings £1m sueball at Met Police for wrongly listing him as a convicted fraudster
[9]UK Home Office seeks secret settlements over unlawful DNA retention
[10]Dratted 'housekeeping', eh? 150k+ records deleted off UK’s Police National Computer database
According to a [11]review of Home Office systems submitted to Parliament NLEDP “is progressing against its new plan and is looking to deliver incrementally so that PNC and [Police National Database] can be decommissioned during 2022 and 2023 respectively."
As such, investment in the PNC should not be ignored, Lord Hogan-Howe's report
[12]PDF
said. The Home Office should create a plan, with investment, to sustain PNC for its full life."Within 8 weeks, the Home Office must provide the Police with feasibility options, including estimated timescales, costs and risks for either replacing or sustaining the PNC," the report said.
[13]
Other recommendations include introducing PNC user representatives to help oversee developing and running the PNC, making a technical architect responsible for the end to end technical design system, and creating product management lifecycle techniques for the PNC service. ®
Get our [14]Tech Resources
[1] https://www.theregister.com/2021/05/25/pnc_missing_files/
[2] https://www.theregister.com/2021/01/21/police_national_computer_pmqs/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YK4cQby2WlO7R@dra8tWmAAAAMo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YK4cQby2WlO7R@dra8tWmAAAAMo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YK4cQby2WlO7R@dra8tWmAAAAMo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YK4cQby2WlO7R@dra8tWmAAAAMo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2019/07/18/making_tax_digital_and_customs_it_among_raft_of_gov_projects_rated_at_major_risk/
[8] https://www.theregister.com/2019/05/15/man_sues_met_1m_fake_conviction_pnc_entry/
[9] https://www.theregister.com/2016/04/05/home_office_settlements/
[10] https://www.theregister.com/2021/01/15/pnc_records_deleted/
[11] https://committees.parliament.uk/publications/1063/documents/8878/default/
[12] http://data.parliament.uk/DepositedPapers/Files/DEP2021-0433/PNC_Independent_Review_Summary_of_Lessons_Learned_Report.pdf
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/storage&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YK4cQby2WlO7R@dra8tWmAAAAMo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://whitepapers.theregister.com/
Re: Realities
T'was ever this....
"Where's your documentation?
"..."
"..."
"Oh. In that case we're Agile"
Re: Realities
Completely agreed on Agile, but is it clear what methodologies were used here? The article talks about the team having been there for a long time, so they might not be Agile or indeed have any methodology at all.
So, something without fully defined requirements isn't tested properly? Colour me surprised. But TBF it sounds like they couldn't be arsed testing it anyway and should therefore should be first up against the wall in the subsequent search for scapegoats.
And before people start: this kind of thing isn't restricted to government projects - I've seen plenty of it in the private sector as well...
Common. that was just an innovative way to do DevOps: putting dev directly into production without prior testing.
WTF, blaming experience now!
"The team who operate it have worked together over a long period of time. The expertise and closeness of the teams involved in running the PNC increased the risk that their work would be accepted rather than checked by a leadership that were in a poor position to challenge their decision making. The PNC services team has very limited police experience in the team and have limited understanding of how the police operate," the report said.
so because the PNC Services team have no clue about what they are doing its the fault of the experienced team that keep the PNC/PND running on a day to day basis.
this is nothing but a call for more outsourcing by a bunch of wits that know nothing about what they are talking about.
Hogan-Howe would have no clue about the detail of what happened so he got help from an "independent" consultancy group. Whats the betting that that group has deep links with outsourcers.
They have no clue about the hard work that go's on to maintain systems, let alone ageing systems that are hard to replace because the people doing the replacement have no working knowledge of what the system needs to do and why, or they are trying to emulate a bespoke system on an off the shelf replacement.
Re: WTF, blaming experience now!
I interpret it as the skilled team that look after the PNC made a change that they were asked to make, but it was so poorly specified that it didn't do what the requester intended. But because the requester knows the ins-and-outs of how the police operate (rather than how the database operates), they can't be blamed for what's clearly a database problem.
Re: WTF, blaming experience now!
At least its not supported by the Horizon support team..
Just remote in and adjust that database record...
Looks like the article is missing an affective spell checker
You got there before me.
To quote the article (and also the top of this forum page): lack of reviews and affective testing
And to quote again: UK’s Police National Computer (PCN)
That said, PCN (or is it PNC) records should be held to a higher standard than Reg journalism posts.
Realities
' The team introducing the code did not have fully defined business requirements and was not maintaining an accurate record of tests undertaken [...] Meanwhile, there was a failure to "design effective and complete tests..." '
The common (erroneous) interpretation of agile strikes again.
The Agile Manifesto did not recommend not bothering to plan, test, or keep records. That's been added later by those who saw agile merely as an opportunity to save themselves effort.