Qualcomm Is Cheating On Their Snapdragon X Elite/Pro Benchmarks (semiaccurate.com)
- Reference: 0173627042
- News link: https://it.slashdot.org/story/24/04/24/1432240/qualcomm-is-cheating-on-their-snapdragon-x-elitepro-benchmarks
- Source link: https://www.semiaccurate.com/2024/04/24/qualcomm-is-cheating-on-their-snapdragon-x-elite-pro-benchmarks/
> Qualcomm is cheating on the Snapdragon X Plus/Elite benchmarks given to OEMs and the press. SemiAccurate doesn't use these words lightly but there is no denying what multiple sources are telling us. [...] Then there were the actual 'briefings' for the X Pro SoC. To call them pathetic is giving them more than their due. The deck was 11 slides, three of which were empty/fluff, five 'benchmark' slides with woefully inadequate disclosure, and two infographic summary slides. The last was the slide below with the 'deep technical' stats [screenshots in the linked article], much of which we told you about last week. [2]And more .
>
> The rest of the 'disclosure' for Snapdragon X Pro was a list of features that all fall under the guise of exactly what you would expect. The rest was filled with deep 'details' like the GPU capabilities of 3.8TFLOPS. That's it. No specs, no capabilities, no nothing. It was truly pathetic. But wait there is more, or less really, with statements like it having AV1 encode and decode. Trivialities like frame rates and resolutions were seemingly not needed for such technical briefs. See what we mean by pathetic? Those 10 cores are arranged how again? That 42MB of cache is what level? Shall I go on about the bare minimum basics or do you get the point now? SemiAccurate was planning to ask Qualcomm about their cheating on benchmarks at the promised briefing but, well, they lied to us and cut us out of the pathetic bits they did brief on. We honestly would have liked to know why they were cheating but we kind of think they will do their usual response to bad news and pretend it never happened like last time. If they actually do explain things we will of course update this article as we always do.
[1] https://www.semiaccurate.com/2024/04/24/qualcomm-is-cheating-on-their-snapdragon-x-elite-pro-benchmarks/
[2] https://www.semiaccurate.com/2024/04/19/what-is-qualcomms-purwa-x-pro-soc/
Cheating on benchmarks? (Score:2)
A vendor cheating on benchmarks to make their product look good? I be shocked if they weren't !
Apple (Score:2)
Is this the new M4-killer that I keep hearing about?
Or is that a different project?
Cheating how? (Score:2)
Um, if your title starts with Qualcomm is cheating, shouldn't you somehow mention what the cheating is, or how the cheating is happening? The summary mentions criticisms of the slides, seemingly unrelated to any cheating, but we are told the slides are pathetic. How is being pathetic equal to cheating?
Hypocritical. (Score:2)
Article complains in inflammatory terms about Broadcom's slides lacking technical details and "without the minimum disclosure needed to check those claims.", then goes on to make accusations while providing no technical details and without the minimum disclosure needed to check those claims. All based on unspecified sources.
[1]Yellow journalism [wikipedia.org] at its finest.
[1] https://en.wikipedia.org/wiki/Yellow_journalism
Stay tuned, truth to come soon (Score:2)
As posted by others, TFA is a strange opinion piece- main grievance is that Qualcomm did not provide information as promised (as part of that, their slides suck). And the claim that some performance numbers are not possible with given settings (but not too much detail on that either). Author is confident on this point.
The there is [1]this article [arstechnica.com] which suggests the moment for Windows-on-ARM is finally here, probably.
I guess we will see. Should be interesting.
[1] https://arstechnica.com/gadgets/2024/04/is-the-arm-version-of-windows-ready-for-its-close-up/
Re: (Score:2)
It *is* strange.
What does "settings" even imply in this context?
Why the weird part about them accidentally testing with x86 binaries? (assumed, since the "fix" was to use Arm native)
How on FSM's green earth do these people consider themselves qualified to evaluate what is "possible" for some set of "settings" on a new piece of silicon?
This article was a pile of shit. Even if it ends up being completely accurate- the article is still a flaming pile of shit.
Prove it (Score:4, Insightful)
That's some seriously stupid clickbait chasing just because he didn't like the slides.
He might as well say this proves P=NP.
Re: (Score:2)
The OEMs built prototypes with these chips and got terrible performance. I would believe that the sources are good. The problem is that the lead in to the article (and the summary) focuses on how the benchmarks were cooked, but gives no other details than the real world performance doesn't match. It was a very long article that says "these run slower than they said they would but we don't know what happened."
Re: (Score:2)
Aren't a lot of "benchmarks" sorta bullshit today, like they can really diverge from actual use cases?
I feel like every manufacturer can goose the numbers on synthetics to get the results they want for press releases.
Re: (Score:2)
Good benchmarks are based around a mix of standard use cases packaged up in a way that is perfectly repeatable.
You can definitely tweak performance to favor higher benchmark scores but that can only go so far without actually improving real world use.
Re: (Score:2)
I think the part that irked me, is that they even mentioned the part where they were testing x86 binaries at "SUB 50%".
Well no shit. That's a your bad. Maybe you should leave that out, because now it looks like perhaps you are trying to taint perception.
Re: (Score:2)
It's hard to tell if the accusation is accurate or not. The summary is so full of hyperbolae that I can't bring myself to read the full article.
Re: (Score:2)
Ya, it's really fucking bad.
It screams like the kind of tortured logic someone who has been personally offended would employ.
I think this person is pissed off at QC, and they decided to lash out in an article, stretching what they really had to look as bad as possible.