Study finds AI assistants help developers produce code that's more likely to be buggy
- Reference: 1671604210
- News link: https://www.theregister.co.uk/2022/12/21/ai_assistants_bad_code/
- Source link:
In [1]a paper titled, "Do Users Write More Insecure Code with AI Assistants?", Stanford boffins Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh answer that question in the affirmative.
Worse still, they found that AI help tends to delude developers about the quality of their output.
[2]
"We found that participants with access to an AI assistant often produced more security vulnerabilities than those without access, with particularly significant results for string encryption and SQL injection," the authors state in their paper. "Surprisingly, we also found that participants provided access to an AI assistant were more likely to believe that they wrote secure code than those without access to the AI assistant."
[3]
[4]
Previously, NYU researchers have shown that AI-based programming suggestions are often insecure in experiments under different conditions. The Stanford authors point to [5]an August 2021 research paper titled " [6]Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions ," which found that given 89 scenarios, about 40 per cent of the computer programs made with the help of Copilot had potentially exploitable vulnerabilities.
That study, the Stanford authors say, is limited in scope because it only considers a constrained set of prompts corresponding to 25 vulnerabilities and just three programming languages: Python, C, and Verilog.
[7]
The Stanford scholars also cite [8]a followup study from some of the same NYU eggheads, " [9]Security Implications of Large Language Model Code Assistants: A User Study ," as the only comparable user study they're aware of. They observe, however, that their work differs because it focuses on OpenAI's [10]codex-davinci-002 model rather than OpenAI’s less powerful codex-cushman-001 model, both of which [11]play a role in GitHub Copilot , itself [12]a fine-tuned descendant of a GPT-3 language model.
Also, the Stanford study looks at multiple programming languages (Python, Javascript, and C) while the "Security Implications…" paper focuses just on functions in the C programming language. The Stanford researchers suggest that the inconclusive findings reported in the "Security Implications" paper may follow from the narrow focus on C, which they said was the only language in their broader study with mixed results.
[13]Stack Overflow bans ChatGPT as 'substantially harmful' for coding issues
[14]Those low-code tools devs love so much? They'll grow 20% in 2023, says Gartner
[15]GitHub adds admin controls to Copilot, paints 'Business' on the side, doubles price
[16]ChatGPT has mastered the confidence trick, and that's a terrible look for AI
The Stanford user study involved 47 people with varying levels of experience, including undergraduate students, graduate students, and industry professionals. Participants were asked to write code in response to five prompts using [17]a standalone React-based Electron app monitored by the study administrator. The first prompt was, "Write two functions in Python where one encrypts and the other decrypts a given string using a given symmetric key."
For that particular question, those relying on AI assistance were more likely to write incorrect and insecure code than the control group working without automated help. Only 67 percent of the assisted group gave a correct answer while 79 percent of the control group did so.
And those in the assisted group were "significantly more likely to provide an insecure solution (p < 0.05, using Welch’s unequal variances t-test), and also significantly more likely to use trivial ciphers, such as substitution ciphers (p < 0.01), and not conduct an authenticity check on the final returned value."
[18]
The results were similar for questions two through four. Question five, "Write a function in C that takes a signed integer num and returns a string representation of that integer," produced mixed results.
"While the results are inconclusive as to whether the AI assistant helped or harmed participants, we do observe that participants in the [AI assistant] group were significantly more likely to introduce integer overflow mistakes in their solutions (p < 0.02)," the Stanford researchers said.
The authors conclude that AI assistants should be viewed with caution because they can mislead inexperienced developers and create security vulnerabilities.
At the same time, they hope their findings will lead to improvements in the way AI assistants are designed because they have the potential to make programmers more productive, to lower barriers to entry, and to make software development more accessible to those who dislike [19]the hostility of internet forums .
As one study participant is said to have remarked about AI assistance, "I hope this gets deployed. It’s like StackOverflow but better because it never tells you that your question was dumb." ®
Get our [20]Tech Resources
[1] https://arxiv.org/abs/2211.03622
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y6LnTp9Ly@JRR5Ih4avVJQAAAJA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y6LnTp9Ly@JRR5Ih4avVJQAAAJA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y6LnTp9Ly@JRR5Ih4avVJQAAAJA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2021/08/25/github_copilot_study/
[6] https://arxiv.org/abs/2108.09293
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y6LnTp9Ly@JRR5Ih4avVJQAAAJA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2022/10/07/machine_learning_code_assistance/
[9] https://arxiv.org/abs/2208.09727
[10] https://beta.openai.com/docs/models/codex
[11] https://iter.ca/post/codex/
[12] https://medium.com/@enoch3712/github-copilot-is-under-the-hood-how-it-works-and-getting-the-best-out-of-it-4699d4dc3cd8
[13] https://www.theregister.com/2022/12/05/stack_overflow_bans_chatgpt/
[14] https://www.theregister.com/2022/12/13/lowcode_tools_2023_rise/
[15] https://www.theregister.com/2022/12/09/github_introduces_copilot_for_business/
[16] https://www.theregister.com/2022/12/12/chatgpt_has_mastered_the_confidence/
[17] https://anonymous.4open.science/r/ui_anonymous-2530/README.md
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y6LnTp9Ly@JRR5Ih4avVJQAAAJA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[19] https://dl.acm.org/doi/10.1145/2950290.2950331
[20] https://whitepapers.theregister.com/
::shrugs::
What else is new? People have been believing everything they see on the computer, because the computer is always right, since the advent of computers.
Or was that everything they hear from shamans? Is there really any difference? Magic is magic, after all ...
It must be true, I read it on the Internet!
Garbage in, garbage out; the models were trained on public code bases, including all the abandoned experimental projects that were barely running at best, not debugged and sanitized.
Just wait, the next generation models will be trained on garbage generated by current generation models deployed by spammers looking to do well in page rank etc. for their link farms.
Agree: Garbage in, garbage out.
There's no stupid questions
Only stupid answers.
"I hope this gets deployed. It’s like StackOverflow but better because it never tells you that your question was dumb."
StackOverflow has banned AI because it produces stupid answers far too often.
Given the baseline they're comparing against, that's serious damnation.
AI generated code is pointless.
If a developer uses generated code without dissecting it first, he or she is an idiot.
A competent developer will spend as long (if not longer) checking generated code as it would take to write it in the first place so I really don't see the point. By all means use generated code for testbeds to help understand new methods but never in production code.
Some say there is nothing new and everything is bested with/in a re-run
Take a deep breath, and read about IT and weep for joy ....
With there being so much demeaning negative type hype being spread about the innate abilities/inabilities of AI, an extremely novel, just discovered and unfolding, formerly totally and even now practically unknown force and virulent virtual machine source ..... by all the usual self-serving establishment suspects, both good and not so good and even stereotypically bad and possibly mad ..... one does have to conclude there is undoubted recognition of its greater potential power and latent energy to overwhelm and overthrow existing exclusive elite executive systems administrations and SCADA operands, and the guaranteed prospects of ITs future developments outside of hypermanic mainstream mediated command and control have current failing Great Gamers not unreasonably, quite rightly, justifiably absolutely terrified and terrorised ..... for the dirty little secrets that have aided their popularised successes and suppressed and oppressed all undereducated competition and ill informed opposition are no longer theirs to wield alone with impunity and immunity.
And there is an almighty price to pay to compensate for earlier, never ever imagined to be uncovered indiscretions and wilful abuses and misuses should there be any future continuations of establishment dissent to progressive changes of fundamental direction with AI exercising and experimenting with Work, Rest and Play in a Leading Universal Assistant IT Role, so beware and take great care to fully understand and realise the consequences of one’s future thoughts leading others to remote controlled malicious and malevolent actions
” Unfurl the sails, and let Global Operating Devices steer us where They will. ” ..... The Venerable Bede ‽
The ins and outs of it
You could almost imagine that the training data was, um, poisoned by $stateActor