News: 1637929391

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Government-favoured child safety app warned it could violate the UK's Investigatory Powers Act with message-scanning tech

(2021/11/26)


A company repeatedly endorsed by ministers backing the UK's Online Safety Bill was warned by its lawyers that its technology could breach the Investigatory Powers Act's ban on unlawful interception of communications, The Register can reveal.

SafeToNet, a content-scanning startup whose product is aimed at parents and uses AI to monitor messages sent to and from children's online accounts, had to change its product after being warned that a feature developed for the government-approved app would break the law.

SafeToNet was hailed this week by senior politicians as an example of "new tech in the fight against online child abuse," having previously featured in announcements from the Department for Digital, Culture, Media and Sport over the past 12 months.

[1]

Chief exec Richard Pursey recounted, during an online seminar at the [2]CogX conference in March this year, how his company's lawyers warned SafeToNet its technology was unlawful.

[3]

[4]

"I don't think I've ever broken out in such a sweat in all my life," Pursey told the seminar.

I'll never forget our CFO and I, we were called into a meeting with our lawyers – we've got some pretty heavyweight lawyers in London – and we told them about how our technology worked and one of the things we were doing, is we were intercepting incoming messages, without the authority of the person that had sent it in the first place.

Intercepting data without the permission of the sender is a civil offence under [5]section 3 of the Investigatory Powers Act 2016 (aka the Snoopers' Charter). Breaches are investigated and judged by the Information Commissioner's Office.

Pursey continued: "We were doing it for good, you know, it's a social impact, we were doing it to safeguard children, until the lawyers said... 'you realise you could go to prison for doing that'. And so what seemed a pretty obvious thing to do – why wouldn't you be allowed to do that – you know, it just put the fear of God in me."

Such interception may also breach [6]section 1 of the Computer Misuse Act 1990 , which criminalises accessing data without authorisation.

[7]

Pursey told The Register "development of that particular feature" had stopped when the company was told of the legal compliance problem, adding:

We consider ourselves pioneers of safety tech and were concerned about existing and future data protection legislation (GDPR) and so asked for a legal review of our proposed tech architecture to ensure we would be fully compliant. The review led to a detailed analysis against a range of legislation from the Computer Misuse Act, the Defamation Act, RIPA, Data Protection etc. It became clear that our plans for interception could have been challenged as illegal – albeit there would have been a defence that related to the way spam emails are filtered before the user sees them.

This is a reference to section [8]3(61 (2)(b) of the Snoopers' Charter, which allows message interception with the "express or implied consent" of a system owner or manager.

[9]MI5 still risks breaking the law on surveillance data through poor controls – years after it was first warned

[10]Snowden was right, rules human rights court as it declares UK spy laws broke ECHR

[11]UK terror law reviewer calls for expanded police powers to imprison people who refuse to hand over passwords

[12]Thou shalt not hack indiscriminately, High Court of England tells Britain's spy agencies

SafeToNet's website says today: "For legal reasons, SafeToNet does not analyze incoming messages before a child has read them."

Pursey added that Britain's unique tech law environment throws this problem up regularly, telling us: "We were a very young startup then but it worries me that those that don't have the finance to get professional advice will cut corners and innocently/naively breach laws like [the Computer Misuse Act] etc. We see that all the time, especially with international safety tech providers entering the UK market. They often have no idea these laws exist."

The revelation that a government-approved company's product fell foul of Britain's laws highlights the [13]ongoing campaign to reform the Computer Misuse Act , and may well prompt further reforms of Britain's convoluted surveillance legislation.

Civil servants and government ministers are engaged in a bitter war against social media platforms' moves towards end-to-end encryption (E2EE) for user messages. Law enforcement bodies such as the National Crime Agency claim that wider adoption of E2EE will stop them from detecting paedophiles preying on children through messaging apps. Tech platforms and privacy advocates say E2EE is a vital tool to prevent and deter [14]unlawful surveillance .

[15]

Avoiding the E2EE problem by scanning messages on children's devices after delivery seems like it might help preserve adults' internet privacy while allowing police agencies to focus on actual harms instead of indiscriminate platform surveillance.

Part of the government war on social media platforms [16]is the Online Safety Bill , renamed from Online Harms Bill at the last moment perhaps in a crude attempt to disassociate it from mountains of well-informed criticism.

While Britain's current surveillance laws were [17]designed to place GCHQ and other spy agencies above and beyond the criminal law (following former NSA sysadmin [18]Edward Snowden 2013's revelations about the extent of nation-states' spying and public revulsion at unchecked domestic mass surveillance), their authors may not have intended to cause difficulties for people making child safety apps. ®

Get our [19]Tech Resources



[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YaESv0ED34oD7IxKo0gpXgAAAEA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[2] https://cogx.live/event/safety-by-design/

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaESv0ED34oD7IxKo0gpXgAAAEA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaESv0ED34oD7IxKo0gpXgAAAEA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[5] https://www.legislation.gov.uk/ukpga/2016/25/part/1/crossheading/prohibitions-against-unlawful-interception/enacted

[6] https://www.legislation.gov.uk/ukpga/1990/18/section/1

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YaESv0ED34oD7IxKo0gpXgAAAEA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://www.legislation.gov.uk/ukpga/2016/25/part/3/enacted

[9] https://www.theregister.com/2021/06/22/mi5_legal_compliance_ipco_reports/

[10] https://www.theregister.com/2021/05/25/echr_ruling_uk_ripa_surveillance_laws/

[11] https://www.theregister.com/2021/03/29/terror_cops_password_law_change_call/

[12] https://www.theregister.com/2021/01/11/equipment_interference_privacy_international_judgment/

[13] https://www.theregister.com/2021/11/03/computer_misuse_act_defence_principles_cyberup/

[14] https://www.theregister.com/2021/10/07/pegasus_malware_princess_haya/

[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YaESv0ED34oD7IxKo0gpXgAAAEA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[16] https://www.theregister.com/2021/10/25/online_harms_dont_need_dangerous/

[17] https://www.theregister.com/2014/10/07/britains_snooping_powers_are_too_weak_says_nca_chief/

[18] https://www.theregister.com/2013/06/14/nsa_whistleblower_used_usb_thumb_drive/

[19] https://whitepapers.theregister.com/



For goodness' sake...

nichomach

...don't encourage them to overhaul surveillance laws with the Home Sec currently in full Bond Villain mode!

https://inews.co.uk/opinion/priti-patel-anti-protest-powers-stuffed-policing-bill-1316830

Our Lord and Savior Rahl

I am not certain that it tracks fully, as presumably they'd have the consent of the Childs legal guardian and by definition, that would be sufficient.

I'm fairly sure that would apply right up until the age of majority, which means the only people affected would be at the tail end of sixth form.

iron

If you had read the article, they require the consent of the sender of the message so consent from the parent or guardian of the child recieving the message does not help.

Doctor Syntax

That might apply to outbound messages but TFA says specifically that the sender's permission is required, not the receiver's. On this interpretation it makes it illegal for inbound messages.

But wait...what are the consequences for ordinary email spam filtering?

Blazde

This is a ridiculous interpretation of the law. Surely if you have permission from the recipient to intercept then you are simply acting as their agent and the law applies as if the intended recipient 'intercepted' it, so to speak. They might need some extra legalese to nail down that relationship, but all kinds of similar arrangements would be unlawful if you couldn't do this.

('Aah that is good coffee. Now I'm ready for you to read me my email Jeeves. Do the silly voices again would you. And skip the parts where mother tells me to get a job, good chap.')

SPARKESFRANKIE66

IANAL, but it's not that simple.

Section 3 creates the offence of unlawful interception and refers to Section 6 for the definition of "lawful authority". Section 3(2) doesn't apply as a parent cannot control the other party's use of the messaging service.

Section 6 specifies when a person has lawful authority. Warrants referenced in 6(1)(a) and 6(1)(c) do not apply here, so we fall back to 6(1)(b), which refers to Sections 44–52.

Of these, 44 refers to interception with consent; 45–48 refer to interception for business or enforcement purposes by the postal services, OFCOM, or businesses; 49–51 refer to interception in institutions such as prisons, immigration detention centres, and psychiatric hospitals; and 52 is in accordance with overseas requests.

Section 44 is the only one that reasonably applies and says that, except for an authorisation under Part 2 of RIPA where only one party need consent, both parties must consent to the interception.

In any case, Section 1 of the Computer Misuse Act would remain a sticky wicket if the sender withheld consent, and because the CMA is written so broadly (for better or worse), adding an exception to Section 1 where the legal guardian of a child has consented would cause more problems than it solved, especially if the intercepter for whatever reason turned rogue.

Blazde

(IANAL either) but I don't see recipient-authorised email-scanning passes Section 3(2), parent involved or not.

I suppose it hinges on whether the relevant 'private telecommunication system' is 'my email address' in which case I surely have the right to control it's operation and any service providers I designate to scan my email, trash it, forward it to me, or automatically broadcast it straight on Twitter are simply operating it with my authority.

Or whether the 'private/public telecommunication system' is 'the creaking email system in general' or even 'the internet', in which case the sender could somehow have an expectation that what..? only 1990s email technology were being used to transmit the email and no scanning takes places? (Or the switches and SMTP servers are unlawful too and the magic IPA-immune fairies are supposed to transmit it?). That still doesn't make sense, but conceivably it's an interpretation a supremely tech-illiterate judge might reach forcing costly appeals and an over-cautious lawyer might advise about that.

Roads

The Man Who Fell To Earth

"The road to tyranny is paved with good intentions."

Pascal Monett

Paving's looking mighty fine 'round here.

Solution

Yet Another Anonymous coward

Declare all children terrorists (personal experience of nephews confirms this) then you can spy on them all you want without any rules

Re: Solution

tiggity

I'm probably declared a terrorist for holding the idea that Palestine has a right to exist & Palestinians should not be murdered at will by an occupying state...

Given that Hamas effectively control the Gaza strip & UK regard Hamas (political wing, not just military wing) as terrorist groups then I'm sure that's enough excuse for them to define me as a terrorist, as they will be more than happy to conflate supporting the human rights of Palestinians with support for military wing of Hamas.

.. So, UK govt can readily find an excuse for most people with a scintilla of compassion to be labelled a terrorist if they feel like it.

Anonymous Coward

So, this is an example of RIPA doing the right thing?

Jonathon Green

“…we were intercepting incoming messages, without the authority of the person that had sent it in the first place.”

[…]

“And so what seemed a pretty obvious thing to do – why wouldn't you be allowed to do that – you know, it just put the fear of God in me."

I was going to say that words failed me. But then it turned out that words hadn’t failed me at all as words like “Good.” and phrases like ‘Have you considered applying for a job in the home office?” Immediately came to mind.

Yet Another Anonymous coward

So I sign my 5 year old up for "My Little Pony Online" and click TOS to agree that her messages are censored, but I can't have the system block dick pics until she opens them because it violates the privacy of the person sending the dick pics ?

Ah, there it is : think of the children

Pascal Monett

" Law enforcement bodies such as the National Crime Agency claim that wider adoption of E2EE will stop them from detecting paedophiles preying on children through messaging apps "

Because of course they do.

I'd like to know how many paedos have been caught thanks to this indispensable privacy-violating attitude.

This is the favorite excuse for snooping, but I've never heard an official declaration stating "We have caught X criminals with this technology".

So, out with it. How many are now behind bars because of your snooping in everyone's lives ?

Re: Ah, there it is : think of the children

Anonymous Coward

The Home Office periodically trots out figures on the number of terrorist attacks thwarted... it's always a low number... and you always assume that most of them are down to stupidity, at least in bringing themselves to the attention of the authorities

V for Vendetta / 1984 incoming

CountCadaver

The more time goes on the more convinced I am that we are rapidly spiralling towards some hideous hybrid of various dystopias including 1984 and V for Vendetta....

Though instead of mandating telescreens, they've persuaded the proles to voluntarily buy them through the medium of "home assistants" something that could be useful with proper privacy safeguards but instead have become an electronic spy in the homes of its users, just waiting to unleash a treasure trove of context free information for the authorities to "prove" wrongdoing.....

Please, please intercept this communication...and much good may it do you!

Anonymous Coward

Quote: "...unlawful interception of communications..."

*

Some of us don't care! Because we like our personal privacy and our personal security, we encrypt everything before any message goes near a public channel. Take your pick...AES, IDEA, Blowfish.....or my favourite, our own private triple encrypted book cipher (sample below). Snoops are welcome to tell EL Reg readers what the message says in plain text (i.e. you are authorised!!).

*

uv2JG5ef4PcVi1mzCh2Ns54L8T0R6VqNihSR2XqNoxyxINgZwRoZOV2paXSNWvcVWjwXiNgfKNiJ

eXYVW3ulCHujGXE1gDyl4dOHQN2nc5O3ElKNk907KfmrKDQnkDo1U5UFCFIroTURqREFUVUhqD2z

YZ4Rgdul0JqhiJSNC5yZ6tMluno3ePaRklwx674b07SjIJuZ8ZyfePYxOHqTCPY3a1M9KdWj0TQz

YBG5glQ3uv8BsTW5sdaLC5mdufIfYdKJa7CBqFSfqTw94t0BcfKvyz4RydKDSXgLMTWtkXUVgXUn

6bGZ0jOZY1OfgvM1mhCxMfSPgX0rqB8naDK36D4Ze5I7eRW14TglwFg3E9cTMjoVs9AJyFwjOReH

G5o50ZKviXCJYjiFQ7apUXwdIdwR6Z09UFkLAXghm3yjELQXgNQp63gPSBEDKbAVqNOLOBUJcNwr

Q5uhwlgD0vWrkrkLuPYZ4f2j0xmn0ZWt0juxU5ipsRwTqL4hevYby5UJmhIVu5WH81YXit8doTSV

6501gNqvIrCjqRMHKHghiHm5k9YpsFA7

*

tiggity

Any software praised by "senior politicians" wrt "child abuse" was almost bound to be privacy infringing - MPs love their privacy but don't want anyone else to have any and think of the children is always their go to excuse for shafting citizens rights* / privacy.

UK govt (and Labour have been equally as bad when they had the chance - stares at Blunkett for example) seem to have a target of UK plebs* to have less privacy and rights than East Germans did in the Berlin wall days.

* rules only to apply to the "little people", just like taxes

Chris G

Every time I hear about governments and cops talking about the detection of paedophiles via online snooping, the thing that is missing is any figures that indicate why they are so concerned.

Some years back, a member of my family attempted suicide as a young teen, it turned out she had been abused by a neighbour for about four years.

During the investigation on talking to one of the senior officers, she explained that often abusers are either family members or friends and neighbours of the family, given that, how often are children targeted by paedophiles online and what are the statistics?

I have a feeling that if authorities and police really wanted to tackle the problem, one of the first things would be via education, after all crime prevention is supposed to be the prime mover for police forces, being able to snoop on everyone at will sounds more like a fishing expedition to me.

Of course if things have changed and child molesters have moved online, then a major think tank needs to look at practical ways to deal with it without destroying society's basic freedoms.

not what it seems

FuzzyTheBear

The real problem is not what it seems. it's about education. it's about being responsible adults that educate the kids. Surprised ?

If you snoop .. you snoop .. email or listening on the telephone or anything else , snooping is snooping. stop it.

If you educate your kids , teach them how to say " hey .. dad .. something's not right .. " hey mom .. i got an invite , can you check it for me ?

and teach him to rely on your judgment when something is odd to them well you done your job.

You got to build trust between you and your kid.

Snooping just shows you don't trust the kid.

Teach , be a parent and all this is useless and you will have given the kids the tools they need to get by in life.

Really .. snooping in " parental controls " is just for parents who don't do their jobs of educating their kids.

Period. Discussion closed.

Which part of the process don't you understand?

Anonymous Coward

@tiggity

@Chris_G

*

Surely you don't want our political class to soil their hands with ANALYSIS?

*

Oh no........they TELL us what's wrong.....they TELL us what they plan (not) to do about it.....

*

......and we plebs just SNAP TO ATTENTION!!

*

Which part of the process don't you understand?

Vegetarians beware! You are what you eat.