Fool me OnePlus, shame on me: Chinese phone firm fingered for fiddling with performance figures – again
(2021/07/08)
- Reference: 1625765528
- News link: https://www.theregister.co.uk/2021/07/08/oneplus_benchmark_fiddling/
- Source link:
Chinese smartphone maker OnePlus has had the shine taken off its latest launches with tests that indicate it is once again trying to fiddle the figures on benchmark results, throttling real-world performance considerably compared to synthetic workloads.
OnePlus hit the market in 2014 with the launch of the OnePlus One, under the tagline "Never Settle". Designed to be a flagship phone priced as a mid-range device, the handset was the first to launch with third-party Android fork CyanogenMod as standard.
Sales have been strong, no thanks to [1]supply issues and an initial but since-abandoned invite-only purchase model, and the success gave the company cause to [2]abandon CyanogenMod in favour of building its own Android fork, dubbed OxygenOS.
[3]
Its second handset [4]had a hyped launch but [5]proved disappointing – though that didn't stop the company releasing [6]device after [7]device after [8]device after [9]device .
[10]
[11]
OnePlus is now up to the OnePlus 9, and its more expensive OnePlus 9 Pro stablemate, but members of the company's [12]faithful congregation don't seem to be getting what they were promised: the company has been fiddling benchmark results, making the handsets seem more powerful than they are in daily use.
First spotted by [13]AnandTech , the problem lies in how the handsets balance performance and power draw. As with most mobile devices, the OnePlus 9 family adjust the voltage and clock speed of their processors in order to maximise battery life. When you're not doing much, the processor is down-clocked to save power; fire up a game or other demanding application and the processor roars into life.
[14]
At least, that's how it's supposed to work. What OnePlus has allegedly been doing, it transpires, is hard-coding a variety of applications into its throttling logic – having them run at a lower-than-usual performance level while excluding common benchmarking apps in order to provide a false but impressive-sounding readout of the handsets' day-to-day speeds.
[15]OnePlus accused of installing cheat codes for benchmarks with new handset
[16]OnePlus privacy shock: So, the cool Chinese smartphones slurp an alarming amount of data
[17]Heads up: OnePlus phones have a secret root backdoor and the password is 'angela'
[18]OnePlus Android mobes' clipboard app caught phoning home to China
Testing has found that the majority of apps have their performance limited on the handsets, including web browsers – with the top-end OnePlus 9 Pro pushing around a quarter of the performance of its rival ZenFone 8 in an in-browser benchmark. Dedicated benchmarking apps, alongside selected games, appear exempt from the throttling – giving an overinflated impression of how fast the phone will be in general use.
"It's disappointing to see OnePlus handsets making performance decisions based on application identifiers rather than application behaviour. We view this as a form of benchmark manipulation," benchmarking site Geekbench announced via [19]Twitter following the discovery. "We've delisted the OnePlus 9 and OnePlus 9 Pro from our Android Benchmark chart.
"We will also test the other OnePlus handsets in our performance lab to see if these handsets also manipulate performance in the same way. If they do, we will delist them from the Android Benchmark chart."
The problem even extends to screen refresh rates. The company's latest handsets come with the promise of a 120Hz display, but the majority of Android browsers are artificially limited to a 60Hz refresh rate – unless the user is willing to fiddle around with hidden settings to unlock the promised performance.
[20]
The really embarrassing thing for OnePlus: this isn't even the first time it's been caught doing exactly the same thing. The company was first found to be cheating benchmarks [21]back in 2017 on its OnePlus 3 and OnePlus 5 devices.
At the time, OnePlus co-founder Cal Pei defended the move. "When users run benchmark apps – which I agree aren't a useful proxy for real-life performance – we believe that they want to see the full potential of their device without interference from tampering," Pei claimed without a trace of irony. "That's what we've unlocked."
The company's customers have had other cause for complaint, too. Concerns were raised with about the [22]amount of data gathered and siphoned off to OnePlus's native China, the discovery of a [23]backdoor to root on its handsets, a clipboard app which [24]phoned home , a breach of its website responsible for the theft of [25]40,000 credit card details , and a camera that could [26]see through clothes – though the fix for the last likely left a fair few pervs sorely disappointed.
OnePlus did not respond to a request for comment from El Reg , but did provide one to mobile-centric site [27]XDA which skirts the issue – claiming the differences spotted by reviewers is down to a post-launch firmware update designed to "improve the devices' battery life and heat management." The behaviour's confirmed presence on launch-day hardware, though, brings that claim into question. ®
Get our [28]Tech Resources
[1] https://www.theregister.com/2014/10/21/oneplus_phone_open_for_allfor_an_hour/
[2] https://www.theregister.com/2015/02/13/oneplus_oxygenos_team/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2015/08/02/oneplus_fanbois_flock_to_paw/
[5] https://www.theregister.com/2015/09/25/oneplus_2_review/
[6] https://www.theregister.com/2016/02/20/oneplus_x_review/
[7] https://www.theregister.com/2017/07/27/oneplus_5_review/
[8] https://www.theregister.com/2017/11/20/oneplus_5t_review/
[9] https://www.theregister.com/2018/05/30/oneplus_6_perfect_porridge/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2018/05/17/oneplus6_religion/
[13] https://www.anandtech.com/show/16794/oneplus-9-performance-examination/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://www.theregister.com/2017/06/21/oneplus_cheat_codes_for_benchmarks/
[16] https://www.theregister.com/2017/10/12/oneplus_privacy_concerns/
[17] https://www.theregister.com/2017/11/14/oneplus_backdoor/
[18] https://www.theregister.com/2018/01/12/oneplus_handsets_clipboard_privacy/
[19] https://twitter.com/geekbench/status/1412433598200823810
[20] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[21] https://www.theregister.com/2017/06/21/oneplus_cheat_codes_for_benchmarks/
[22] https://www.theregister.com/2017/10/12/oneplus_privacy_concerns/
[23] https://www.theregister.com/2017/11/14/oneplus_backdoor/
[24] https://www.theregister.com/2018/01/12/oneplus_handsets_clipboard_privacy/
[25] https://www.theregister.com/2018/01/19/oneplus_credit_card_hack/
[26] https://www.theregister.com/2020/05/19/oneplus_x_ray_camera_filter/
[27] https://www.xda-developers.com/oneplus-9-pro-performance-throttling-popular-apps/#update1
[28] https://whitepapers.theregister.com/
OnePlus hit the market in 2014 with the launch of the OnePlus One, under the tagline "Never Settle". Designed to be a flagship phone priced as a mid-range device, the handset was the first to launch with third-party Android fork CyanogenMod as standard.
Sales have been strong, no thanks to [1]supply issues and an initial but since-abandoned invite-only purchase model, and the success gave the company cause to [2]abandon CyanogenMod in favour of building its own Android fork, dubbed OxygenOS.
[3]
Its second handset [4]had a hyped launch but [5]proved disappointing – though that didn't stop the company releasing [6]device after [7]device after [8]device after [9]device .
[10]
[11]
OnePlus is now up to the OnePlus 9, and its more expensive OnePlus 9 Pro stablemate, but members of the company's [12]faithful congregation don't seem to be getting what they were promised: the company has been fiddling benchmark results, making the handsets seem more powerful than they are in daily use.
First spotted by [13]AnandTech , the problem lies in how the handsets balance performance and power draw. As with most mobile devices, the OnePlus 9 family adjust the voltage and clock speed of their processors in order to maximise battery life. When you're not doing much, the processor is down-clocked to save power; fire up a game or other demanding application and the processor roars into life.
[14]
At least, that's how it's supposed to work. What OnePlus has allegedly been doing, it transpires, is hard-coding a variety of applications into its throttling logic – having them run at a lower-than-usual performance level while excluding common benchmarking apps in order to provide a false but impressive-sounding readout of the handsets' day-to-day speeds.
[15]OnePlus accused of installing cheat codes for benchmarks with new handset
[16]OnePlus privacy shock: So, the cool Chinese smartphones slurp an alarming amount of data
[17]Heads up: OnePlus phones have a secret root backdoor and the password is 'angela'
[18]OnePlus Android mobes' clipboard app caught phoning home to China
Testing has found that the majority of apps have their performance limited on the handsets, including web browsers – with the top-end OnePlus 9 Pro pushing around a quarter of the performance of its rival ZenFone 8 in an in-browser benchmark. Dedicated benchmarking apps, alongside selected games, appear exempt from the throttling – giving an overinflated impression of how fast the phone will be in general use.
"It's disappointing to see OnePlus handsets making performance decisions based on application identifiers rather than application behaviour. We view this as a form of benchmark manipulation," benchmarking site Geekbench announced via [19]Twitter following the discovery. "We've delisted the OnePlus 9 and OnePlus 9 Pro from our Android Benchmark chart.
"We will also test the other OnePlus handsets in our performance lab to see if these handsets also manipulate performance in the same way. If they do, we will delist them from the Android Benchmark chart."
The problem even extends to screen refresh rates. The company's latest handsets come with the promise of a 120Hz display, but the majority of Android browsers are artificially limited to a 60Hz refresh rate – unless the user is willing to fiddle around with hidden settings to unlock the promised performance.
[20]
The really embarrassing thing for OnePlus: this isn't even the first time it's been caught doing exactly the same thing. The company was first found to be cheating benchmarks [21]back in 2017 on its OnePlus 3 and OnePlus 5 devices.
At the time, OnePlus co-founder Cal Pei defended the move. "When users run benchmark apps – which I agree aren't a useful proxy for real-life performance – we believe that they want to see the full potential of their device without interference from tampering," Pei claimed without a trace of irony. "That's what we've unlocked."
The company's customers have had other cause for complaint, too. Concerns were raised with about the [22]amount of data gathered and siphoned off to OnePlus's native China, the discovery of a [23]backdoor to root on its handsets, a clipboard app which [24]phoned home , a breach of its website responsible for the theft of [25]40,000 credit card details , and a camera that could [26]see through clothes – though the fix for the last likely left a fair few pervs sorely disappointed.
OnePlus did not respond to a request for comment from El Reg , but did provide one to mobile-centric site [27]XDA which skirts the issue – claiming the differences spotted by reviewers is down to a post-launch firmware update designed to "improve the devices' battery life and heat management." The behaviour's confirmed presence on launch-day hardware, though, brings that claim into question. ®
Get our [28]Tech Resources
[1] https://www.theregister.com/2014/10/21/oneplus_phone_open_for_allfor_an_hour/
[2] https://www.theregister.com/2015/02/13/oneplus_oxygenos_team/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://www.theregister.com/2015/08/02/oneplus_fanbois_flock_to_paw/
[5] https://www.theregister.com/2015/09/25/oneplus_2_review/
[6] https://www.theregister.com/2016/02/20/oneplus_x_review/
[7] https://www.theregister.com/2017/07/27/oneplus_5_review/
[8] https://www.theregister.com/2017/11/20/oneplus_5t_review/
[9] https://www.theregister.com/2018/05/30/oneplus_6_perfect_porridge/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2018/05/17/oneplus6_religion/
[13] https://www.anandtech.com/show/16794/oneplus-9-performance-examination/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://www.theregister.com/2017/06/21/oneplus_cheat_codes_for_benchmarks/
[16] https://www.theregister.com/2017/10/12/oneplus_privacy_concerns/
[17] https://www.theregister.com/2017/11/14/oneplus_backdoor/
[18] https://www.theregister.com/2018/01/12/oneplus_handsets_clipboard_privacy/
[19] https://twitter.com/geekbench/status/1412433598200823810
[20] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOd1gYXPXKeJnA-WQ2ysPwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[21] https://www.theregister.com/2017/06/21/oneplus_cheat_codes_for_benchmarks/
[22] https://www.theregister.com/2017/10/12/oneplus_privacy_concerns/
[23] https://www.theregister.com/2017/11/14/oneplus_backdoor/
[24] https://www.theregister.com/2018/01/12/oneplus_handsets_clipboard_privacy/
[25] https://www.theregister.com/2018/01/19/oneplus_credit_card_hack/
[26] https://www.theregister.com/2020/05/19/oneplus_x_ray_camera_filter/
[27] https://www.xda-developers.com/oneplus-9-pro-performance-throttling-popular-apps/#update1
[28] https://whitepapers.theregister.com/
Bugger, shattering my bubble...
Chris 239
I'm typing this on my 4 year old OnePlus 5 and have to say until now I've been more than happy with it.
It still pisses all over my <1 yr old Samsung A50 work mobile in performance, battery life, charging speed and reliability.
It's been updated several times and is on Android 10 so support seems ok to me.
Re: Bugger, shattering my bubble...
DF118
Same. Work Samsung A40 is infuriatingly slow, plus the Samsung bloatware, unwanted apps and OS overlay are all, frankly, horrible.
Personally idc if OnePlus have been fiddling the numbers, although with the prices they're at now I'll almost certainly never buy one new again. Luckily I just picked a mint 8 Pro for £250 and if that lasts me as long as my 3 did (still going strong when it got nicked) then I'll be happy.
The only thing the Sammy has going for it over both my OnePlus handsets is its size. If OnePlus made a smaller handset I'd go for that in a heartbeat. The big screen is nice for browsing and PDF manuals but the ability to have it in a pocket and not feel like I'm going to snap it in two every time I have to shoehorn myself in under some desk or the back of an equipment rack would be nicer.
Cyanogen Mod?
TeeCee
Cobblers, cobblers and thrice cobblers.
OnePlus originally used Cyanogen OS , Cyanogen's commercial offering based on the hobbyist Mod. Unfortunately, Cyanogen then came over all venally stupid and did an exclusive deal with some Indian mob. OnePlus were forced to change or face being shut out of the Indian market.
OnePlus had the last laugh when Cyanogen went comprehensively titsup.com when their Indian connection proved to be rather less lucrative than they'd been led to believe.
mark l 2
I have been considering getting a Oneplus phone as my next handset as they are pretty well supported by LineageOS so you can extend their life for quite a few years after OnePlus have stopped providing updates.
this isn't even the first time it's been caught doing exactly the same thing.
clearly enough idiots keep buying into this 0.001 percentage 'advantage' shit, if the brand keeps doing it. On one side, I don't care, on the other hand, to see a brand lying and getting away with it...