Microsoft quantum lab retracts published paper: Readings that cast doubt on crucial discovery went AWOL
(2021/03/09)
- Reference: 1615280108
- News link: https://www.theregister.co.uk/2021/03/09/quantum_paper_withdrawn/
- Source link:
A paper published in Nature two years ago and spearheaded by a Microsoft scientist has been retracted after it emerged that the data presented simply didn't add up.
The work was produced at a quantum computer lab set up by Microsoft and QuTech, a research center co-founded by the Delft University of Technology (TU Delft) in the Netherlands. The study, led by Microsoftie and TU Delft Professor Leo Kouwenhoven, reported the discovery of a theoretical quasiparticle the academics believed would prove useful for future quantum computers.
[1]
"A 2018 academic paper published in Nature and led by one of our scientific directors, primarily in his capacity as a Professor at TU Delft, was retracted,” Zulfi Alam, a Microsoft Quantum unit veep, told The Register on Monday.
“As part of proposing the retraction, the authors of the paper took feedback from the scientific community, re-analyzed the data, wrote a new paper based on the analysis, and embraced the paper’s examination by independent experts in the field. This is an excellent example of the scientific process at work.
[2]
"We remain confident in our topological approach to scaled quantum computing, as we nurture a culture of continuous improvement to innovate across every component of the Quantum Machine.”
What went wrong?
That 2018 paper [3]described a lab experiment that attempted to detect Majorana zero mode particles by measuring the conductance of superconductors at cryogenic temperatures.
That's important because scientists [4]want to use Majorana zero mode particles as qubits in quantum computers as they believe they will be more robust at storing information than today's qubits. Majorana zero mode particles bring stability, in other words.
It would be a significant milestone in quantum computing to find and use these qubit candidates, and the Microsoft-Delft team claimed they had discovered these ghostly quasiparticles in their superconductor.
“Majorana zero-modes — a type of localized quasiparticle — hold great promise for topological quantum computing,” they wrote in their 2018 paper.
“Tunnelling spectroscopy in electrical transport is the primary tool for identifying the presence of Majorana zero-modes, for instance as a zero-bias peak in differential conductance.”
Microsoft: We've made a coding language for a quantum computer that may or may not exist [5]READ MORE
Multiple graphs appeared to show the presence of these wonder particles. But it was too good to be true. The retraction note, [6]published in Nature this week, said that after the team was told of shortcomings in their paper, they were unable to replicate their study’s results, and so decided they hadn't found the Majorana particles after all.
The note, headed by Microsoft's Kouwenhoven, admitted the team had left out measurements that would have cast doubt on their findings. It also acknowledged two researchers – Sergey Frolov and Vincent Mourik – who had scrutinized the paper and highlighted errors.
“Several inconsistencies were pointed out by Sergey Frolov and Vincent Mourik between the raw measurement data that was made available to them and the figures that were published in the paper,” it said.
“We therefore re-analysed all the existing raw data for our original measurements and rebuilt the original experimental setup for a recalibration of the conductance values ... When the data are replotted over the full parameter range, including ranges that were not made available earlier, points are outside the 2-sigma error bars. We can therefore no longer claim the observation of a quantized Majorana conductance, and wish to retract this letter.”
Frolov, an associate professor at the department of physics and astronomy at the University of Pittsburgh in the US, and Mourik, a research associate at the University of New South Wales in Australia, however, were a bit more blunt about the whole debacle.
We have found that the original source experimental data may have been manipulated, namely cut, as well as cut out and pasted together. Furthermore, entire datasets that contradict the central claim of the Nature paper were suppressed
“We have found that the original source experimental data may have been manipulated, namely cut, as well as cut out and pasted together. Furthermore, entire datasets that contradict the central claim of the Nature paper were suppressed,” they said in a [7]series of slides showing how the data had been apparently massaged.
The pair, which had requested and studied the raw data from the team's experiments, noticed data points had been shifted on graphs' axes as well as an omission of evidence that contradicted the suggestion of a Majorana quasiparticle discovery.
"Taking the Nature paper and the additional data together, no evidence of zero-bias conductance quantization exists in this work," Frolov and Mourik concluded.
TU Delft conducted its own probe into the study, and while its Scientific Integrity Committee, made up of independent experts, agreed with the results of Frolov and Mourik’s academic sleuthing, we're told the panel found “no evidence that [the data manipulation] was intentional.”
The authors were caught up in the enthusiasm of the moment and thus did not pay enough attention to data that did not suit their purpose
“The experts consider the most plausible explanation to be that the authors were caught up in the enthusiasm of the moment and thus did not pay enough attention to data that did not suit their purpose,” the university said in a statement.
There were rumblings that something was amiss with the 2018 paper when a study [8]shared on arXiv last month, led by Kouwenhoven and 21 co-authors, admitted the quantum computing lab had failed to find Majorana particles, Wired [9]first reported . The team stated their original study was due to be formally retracted.
“The retraction of the article is obviously a setback in Majorana research towards the development of a quantum computer,” said Lieven Vandersypen, a scientific director at QuTech and TU Delft.
[10]
“Reflection on the methods used, must now run its course within the scientific community. Within QuTech we have already started that discussion. At the same time, we are continuing to work hard on our various lines of research towards the realization of a quantum computer and a quantum Internet.” ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[3] https://www.nature.com/articles/nature26142
[4] https://www.aps.org/publications/apsnews/201804/hunt.cfm
[5] https://www.theregister.com/2017/09/25/microsoft_talks_quantum_computing_and_ai/
[6] https://www.nature.com/articles/s41586-021-03373-x#ref-CR2
[7] https://zenodo.org/record/4587841
[8] https://arxiv.org/abs/2101.11456
[9] https://www.wired.com/story/microsoft-win-quantum-computing-error/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://whitepapers.theregister.com/
The work was produced at a quantum computer lab set up by Microsoft and QuTech, a research center co-founded by the Delft University of Technology (TU Delft) in the Netherlands. The study, led by Microsoftie and TU Delft Professor Leo Kouwenhoven, reported the discovery of a theoretical quasiparticle the academics believed would prove useful for future quantum computers.
[1]
"A 2018 academic paper published in Nature and led by one of our scientific directors, primarily in his capacity as a Professor at TU Delft, was retracted,” Zulfi Alam, a Microsoft Quantum unit veep, told The Register on Monday.
“As part of proposing the retraction, the authors of the paper took feedback from the scientific community, re-analyzed the data, wrote a new paper based on the analysis, and embraced the paper’s examination by independent experts in the field. This is an excellent example of the scientific process at work.
[2]
"We remain confident in our topological approach to scaled quantum computing, as we nurture a culture of continuous improvement to innovate across every component of the Quantum Machine.”
What went wrong?
That 2018 paper [3]described a lab experiment that attempted to detect Majorana zero mode particles by measuring the conductance of superconductors at cryogenic temperatures.
That's important because scientists [4]want to use Majorana zero mode particles as qubits in quantum computers as they believe they will be more robust at storing information than today's qubits. Majorana zero mode particles bring stability, in other words.
It would be a significant milestone in quantum computing to find and use these qubit candidates, and the Microsoft-Delft team claimed they had discovered these ghostly quasiparticles in their superconductor.
“Majorana zero-modes — a type of localized quasiparticle — hold great promise for topological quantum computing,” they wrote in their 2018 paper.
“Tunnelling spectroscopy in electrical transport is the primary tool for identifying the presence of Majorana zero-modes, for instance as a zero-bias peak in differential conductance.”
Microsoft: We've made a coding language for a quantum computer that may or may not exist [5]READ MORE
Multiple graphs appeared to show the presence of these wonder particles. But it was too good to be true. The retraction note, [6]published in Nature this week, said that after the team was told of shortcomings in their paper, they were unable to replicate their study’s results, and so decided they hadn't found the Majorana particles after all.
The note, headed by Microsoft's Kouwenhoven, admitted the team had left out measurements that would have cast doubt on their findings. It also acknowledged two researchers – Sergey Frolov and Vincent Mourik – who had scrutinized the paper and highlighted errors.
“Several inconsistencies were pointed out by Sergey Frolov and Vincent Mourik between the raw measurement data that was made available to them and the figures that were published in the paper,” it said.
“We therefore re-analysed all the existing raw data for our original measurements and rebuilt the original experimental setup for a recalibration of the conductance values ... When the data are replotted over the full parameter range, including ranges that were not made available earlier, points are outside the 2-sigma error bars. We can therefore no longer claim the observation of a quantized Majorana conductance, and wish to retract this letter.”
Frolov, an associate professor at the department of physics and astronomy at the University of Pittsburgh in the US, and Mourik, a research associate at the University of New South Wales in Australia, however, were a bit more blunt about the whole debacle.
We have found that the original source experimental data may have been manipulated, namely cut, as well as cut out and pasted together. Furthermore, entire datasets that contradict the central claim of the Nature paper were suppressed
“We have found that the original source experimental data may have been manipulated, namely cut, as well as cut out and pasted together. Furthermore, entire datasets that contradict the central claim of the Nature paper were suppressed,” they said in a [7]series of slides showing how the data had been apparently massaged.
The pair, which had requested and studied the raw data from the team's experiments, noticed data points had been shifted on graphs' axes as well as an omission of evidence that contradicted the suggestion of a Majorana quasiparticle discovery.
"Taking the Nature paper and the additional data together, no evidence of zero-bias conductance quantization exists in this work," Frolov and Mourik concluded.
TU Delft conducted its own probe into the study, and while its Scientific Integrity Committee, made up of independent experts, agreed with the results of Frolov and Mourik’s academic sleuthing, we're told the panel found “no evidence that [the data manipulation] was intentional.”
The authors were caught up in the enthusiasm of the moment and thus did not pay enough attention to data that did not suit their purpose
“The experts consider the most plausible explanation to be that the authors were caught up in the enthusiasm of the moment and thus did not pay enough attention to data that did not suit their purpose,” the university said in a statement.
There were rumblings that something was amiss with the 2018 paper when a study [8]shared on arXiv last month, led by Kouwenhoven and 21 co-authors, admitted the quantum computing lab had failed to find Majorana particles, Wired [9]first reported . The team stated their original study was due to be formally retracted.
“The retraction of the article is obviously a setback in Majorana research towards the development of a quantum computer,” said Lieven Vandersypen, a scientific director at QuTech and TU Delft.
[10]
“Reflection on the methods used, must now run its course within the scientific community. Within QuTech we have already started that discussion. At the same time, we are continuing to work hard on our various lines of research towards the realization of a quantum computer and a quantum Internet.” ®
Get our [11]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[3] https://www.nature.com/articles/nature26142
[4] https://www.aps.org/publications/apsnews/201804/hunt.cfm
[5] https://www.theregister.com/2017/09/25/microsoft_talks_quantum_computing_and_ai/
[6] https://www.nature.com/articles/s41586-021-03373-x#ref-CR2
[7] https://zenodo.org/record/4587841
[8] https://arxiv.org/abs/2101.11456
[9] https://www.wired.com/story/microsoft-win-quantum-computing-error/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/science&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YEdVSlO6VWBJRXGr4PxqgAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://whitepapers.theregister.com/
Re: Not very reassuring
Red Ted
"did not pay enough attention to data that did not suit their purpose"
There's a really fine line between that and manipulating the results, if there is even a line at all...
Caught up in the enthusiasm of the moment
Warm Braw
If only we'd known about the propensity of quanta for entanglement.
What I don't understand
Chris G
Aside from quantum physics, is why anyone who is producing a paper they know is going to be peer reviewed by people who are at least on par with them, would massage the results?
I would be inclined to give them the benefit of the doubt.
Re: What I don't understand
sreynolds
Errrm, peer review doesn't mean that someone is actually going to rely on what you said and/or even bother looking at it unless they have a vested interest. Sure some people will skim over it.
Wasn't the paper that convincingly statistically at least proved that autism was a direct result of the MMR jab? Peer review, surely you jest.
So you can't tell if there's a qubit in the box
Neil Barnes
Until you open it?
Looking in the pockets, obv. -->
Re: So you can't tell if there's a qubit in the box
sreynolds
Or until, it would appear you cut and paste data that you like. I am sorry, is that right? Are they saying that the data was cut and pasted? And if so from where?
Not very reassuring
The future of quantum computing apparently relies on the discovery of a theoretical particle that is its own antiparticle. That seems to be quite the barrier to entry.
On the other hand, " they were unable to replicate their study’s results " means that the study was clearly not conducted in a scientific manner. When you're a scientist, you're supposed to check, double check and, in a case like this, triple check your data and your results. When you can reliably replicate your results, then you can publish.
So this was not a proper scientific study. It was, however, a perfectly valid Microsoft study : throw something on the wall, see if sticks, publish.
Way to go.