GitHub Copilot auto-coder snags emerge, from seemingly spilled secrets to bad code, but some love it
(2021/07/06)
- Reference: 1625574495
- News link: https://www.theregister.co.uk/2021/07/06/github_copilot_autocoder_caught_spilling/
- Source link:
Early testers of GitHub's Copilot, which uses AI to assist programmers to write code, have found problems including alleged spilled secrets, bad code, and copyright concerns, though some see huge potential in the tool.
GitHub Copilot was [1]released as a limited "technical preview" last week with the claim that it is an "AI pair programmer." It is powered by a system called Codex, from OpenAI, a company which went into [2]partnership with Microsoft in 2019, receiving a $1bn investment.
[3]
How it works: using public code as a model for AI-assisted development
According to its website, the Codex model is trained by "public code and text on the internet" and "understands both programming and human languages."
An extension to Visual Studio Code "sends your comments and code to the GitHub Copilot service, which then users OpenAI Codex to synthesize and suggest individual lines and whole functions."
What could go wrong?
One developer [4]tried an experiment , writing some code to send an email via the Sendgrid service and prompting Copilot by typing "apiKey :=". Copilot responded with at least four proposed keys, according to his screenshot and bug report. He reported it as a bug under the name, "AI is emitting secrets."
But were the keys valid? GitHub CEO Nat Friedman [5]responded to the bug report , stating that "these secrets are almost entirely fictional, synthesized from the training data."
A Copilot maintainer added that "the probability of copying a secret from the training data is extremely small. Furthermore, the training data is all public code (no private code at all) so even in the extremely unlikely event a secret is copied, it was already compromised."
[6]
While reassuring, even the remote possibility that Copilot is prompting coders with other user's secrets is perhaps a concern. It touches on a key issue: is Copilot's AI really writing code, or is it copy-pasting chunks from its training sources?
[7]
[8]
GitHub attempted to address some of these issues in [9]a FAQ . "GitHub Copilot is a code synthesizer, not a search engine," it said. "The vast majority of the code that it suggests is uniquely generated and has never been seen before."
According to its own study, however, "about 0.1 per cent of the time, the suggestion may contain some snippets that are verbatim from the training set."
[10]
This 0.1 per cent (and some early users think it is higher) is troublesome. GitHub's proposed solution, as given in [11]this paper , is that when the AI is quoting rather than synthesizing code it will give attribution. "That way, I’m able to look up background information about that code, and to include credit where credit is due," said GitHub machine learning engineer Albert Ziegler.
The problem is that there are circumstances where Copilot may be prompting developers to do the wrong thing, for example with code that is open source but protected by copyright.
In the case of GPL code, which is [12]copyleft , the inclusion of the code could impact the licensing of the new work. It is confusing, since the Copilot FAQ [13]states that "the suggestions GitHub Copilot generates, and the code you write with its help, belong to you, and you are responsible for it;" but an attributed block of code would be an exception.
[14]
GitHub also said that "training machine learning models on publicly available data is considered fair use across the machine learning community," pre-empting concerns about the AI borrowing other people's code. There is some uncertainty.
GitHub's CEO [15]said on Twitter that "we expect that IP and AI will be an interesting policy discussion around the world in the coming years, and we're eager to participate."
Developer Eevee [16]said that "GitHub Copilot has, by their own admission, been trained on mountains of gpl code, so I'm unclear on how it's not a form of laundering open source code into commercial works."
The only way is ethics
Since Copilot will be a paid-for product, there is an ethical as well as a legal debate. That said, open source advocate Simon Phipps [17]has said : "I'm hearing alarmed tech folk but no particularly alarmed lawyers. Consensus seems to be that training the model and using the model are to be analysed separately, that training is Just Fine and that using is unlikely to involve a copyright controlled act so licensing is moot."
OpenAI has a [18]paper [PDF] on the matter which argued that "under current law, training AI systems constitutes fair use," although it added that "Legal uncertainty on the copyright implications of training AI systems imposes substantial costs on AI developers and so should be authoritatively resolved."
Does it work?
Another issue is whether the code will work correctly. Developer Colin Eberhardt has been trying the preview and [19]said "I'm stunned by its capabilities. It has genuinely made me say “wow” out loud a few times in the past few hours."
Read on though, and it seems that his results have been mixed. One common way to use Copilot is to type a comment, following which the AI may suggest a block of code. Eberhardt typed: //compute the moving average of an array for a given window size
and Copilot generated a correct function. However, when he tried: //find the two entries that sum to 2020 and then multiply the two numbers together
the generated code looked plausible, but it was wrong.
Careful examination of the code, combined with strong unit test coverage, should defend against this kind of problem; but it does look like a trap for the unwary, especially coming from GitHub as an official add-on for the world's most popular code editor, Visual Studio Code.
One could contrast with copy-pasting code from a site like StackOverflow, where other contributors will often spot coding errors and there is a kind of community quality control. With Copilot, the developer is on their own.
"I think Copilot has a little way to go before I'd want to keep it turned on by default," concluded Eberhardt, because of "the cognitive load associated with verifying its suggestions."
[20]GitHub Copilot is AI pair programming where you, the human, still have to do most of the work
[21]Good news: Google no longer requires publishers to use the AMP format. Bad news: What replaces it might be worse
[22]What’s the big deal with service meshes? Think of them as SDN at Layer 7
[23]Y'all ready to get back to the office this October, Facebook tells staff in the US
He also observed that suggestions were sometimes slow to appear, though this may be addressed by some sort of busy indicator. Eberhardt nevertheless said he believes that many enterprises will subscribe to Copilot because of its "wow factor" – a disturbing conclusion given its current shortcomings, though bear in mind that it is a preview.
Much of programming is drudge work and few problems are unique to one project, so in principle applying AI to the task could work well. Microsoft's [24]IntelliCode , which uses machine learning to improve code completion, is fine; it can improve productivity without increasing the risk of errors.
AI-generated chunks of code is another matter and the story so far is that there is plenty of potential, but also plenty of snags. ®
Get our [25]Tech Resources
[1] https://www.theregister.com/2021/06/30/github_ai_copilot/
[2] https://openai.com/blog/microsoft/
[3] https://regmedia.co.uk/2021/07/06/copilot.jpg
[4] https://twitter.com/pkell7/status/1411058236321681414
[5] https://twitter.com/azjezz/status/1412054859877126151/photo/1
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://copilot.github.com/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://docs.github.com/en/github/copilot/research-recitation
[12] https://www.gnu.org/licenses/copyleft.en.html
[13] https://copilot.github.com/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[15] https://twitter.com/natfriedman/status/1409914420579344385
[16] https://twitter.com/eevee/status/1410037309848752128
[17] https://twitter.com/webmink/status/1410943130979942407
[18] https://www.uspto.gov/sites/default/files/documents/OpenAI_RFC-84-FR-58141.pdf
[19] https://blog.scottlogic.com/2021/07/03/github-copilot-first-thoughts.html
[20] https://www.theregister.com/2021/06/30/github_ai_copilot/
[21] https://www.theregister.com/2021/06/28/google_amp_core_web_vitals/
[22] https://www.theregister.com/2021/06/25/service_mesh_sdn_layer_7/
[23] https://www.theregister.com/2021/06/10/facebook_office_work/
[24] https://devblogs.microsoft.com/visualstudio/ai-assisted-intellisense-for-your-teams-codebase/
[25] https://whitepapers.theregister.com/
GitHub Copilot was [1]released as a limited "technical preview" last week with the claim that it is an "AI pair programmer." It is powered by a system called Codex, from OpenAI, a company which went into [2]partnership with Microsoft in 2019, receiving a $1bn investment.
[3]
How it works: using public code as a model for AI-assisted development
According to its website, the Codex model is trained by "public code and text on the internet" and "understands both programming and human languages."
An extension to Visual Studio Code "sends your comments and code to the GitHub Copilot service, which then users OpenAI Codex to synthesize and suggest individual lines and whole functions."
What could go wrong?
One developer [4]tried an experiment , writing some code to send an email via the Sendgrid service and prompting Copilot by typing "apiKey :=". Copilot responded with at least four proposed keys, according to his screenshot and bug report. He reported it as a bug under the name, "AI is emitting secrets."
But were the keys valid? GitHub CEO Nat Friedman [5]responded to the bug report , stating that "these secrets are almost entirely fictional, synthesized from the training data."
A Copilot maintainer added that "the probability of copying a secret from the training data is extremely small. Furthermore, the training data is all public code (no private code at all) so even in the extremely unlikely event a secret is copied, it was already compromised."
[6]
While reassuring, even the remote possibility that Copilot is prompting coders with other user's secrets is perhaps a concern. It touches on a key issue: is Copilot's AI really writing code, or is it copy-pasting chunks from its training sources?
[7]
[8]
GitHub attempted to address some of these issues in [9]a FAQ . "GitHub Copilot is a code synthesizer, not a search engine," it said. "The vast majority of the code that it suggests is uniquely generated and has never been seen before."
According to its own study, however, "about 0.1 per cent of the time, the suggestion may contain some snippets that are verbatim from the training set."
[10]
This 0.1 per cent (and some early users think it is higher) is troublesome. GitHub's proposed solution, as given in [11]this paper , is that when the AI is quoting rather than synthesizing code it will give attribution. "That way, I’m able to look up background information about that code, and to include credit where credit is due," said GitHub machine learning engineer Albert Ziegler.
The problem is that there are circumstances where Copilot may be prompting developers to do the wrong thing, for example with code that is open source but protected by copyright.
In the case of GPL code, which is [12]copyleft , the inclusion of the code could impact the licensing of the new work. It is confusing, since the Copilot FAQ [13]states that "the suggestions GitHub Copilot generates, and the code you write with its help, belong to you, and you are responsible for it;" but an attributed block of code would be an exception.
[14]
GitHub also said that "training machine learning models on publicly available data is considered fair use across the machine learning community," pre-empting concerns about the AI borrowing other people's code. There is some uncertainty.
GitHub's CEO [15]said on Twitter that "we expect that IP and AI will be an interesting policy discussion around the world in the coming years, and we're eager to participate."
Developer Eevee [16]said that "GitHub Copilot has, by their own admission, been trained on mountains of gpl code, so I'm unclear on how it's not a form of laundering open source code into commercial works."
The only way is ethics
Since Copilot will be a paid-for product, there is an ethical as well as a legal debate. That said, open source advocate Simon Phipps [17]has said : "I'm hearing alarmed tech folk but no particularly alarmed lawyers. Consensus seems to be that training the model and using the model are to be analysed separately, that training is Just Fine and that using is unlikely to involve a copyright controlled act so licensing is moot."
OpenAI has a [18]paper [PDF] on the matter which argued that "under current law, training AI systems constitutes fair use," although it added that "Legal uncertainty on the copyright implications of training AI systems imposes substantial costs on AI developers and so should be authoritatively resolved."
Does it work?
Another issue is whether the code will work correctly. Developer Colin Eberhardt has been trying the preview and [19]said "I'm stunned by its capabilities. It has genuinely made me say “wow” out loud a few times in the past few hours."
Read on though, and it seems that his results have been mixed. One common way to use Copilot is to type a comment, following which the AI may suggest a block of code. Eberhardt typed: //compute the moving average of an array for a given window size
and Copilot generated a correct function. However, when he tried: //find the two entries that sum to 2020 and then multiply the two numbers together
the generated code looked plausible, but it was wrong.
Careful examination of the code, combined with strong unit test coverage, should defend against this kind of problem; but it does look like a trap for the unwary, especially coming from GitHub as an official add-on for the world's most popular code editor, Visual Studio Code.
One could contrast with copy-pasting code from a site like StackOverflow, where other contributors will often spot coding errors and there is a kind of community quality control. With Copilot, the developer is on their own.
"I think Copilot has a little way to go before I'd want to keep it turned on by default," concluded Eberhardt, because of "the cognitive load associated with verifying its suggestions."
[20]GitHub Copilot is AI pair programming where you, the human, still have to do most of the work
[21]Good news: Google no longer requires publishers to use the AMP format. Bad news: What replaces it might be worse
[22]What’s the big deal with service meshes? Think of them as SDN at Layer 7
[23]Y'all ready to get back to the office this October, Facebook tells staff in the US
He also observed that suggestions were sometimes slow to appear, though this may be addressed by some sort of busy indicator. Eberhardt nevertheless said he believes that many enterprises will subscribe to Copilot because of its "wow factor" – a disturbing conclusion given its current shortcomings, though bear in mind that it is a preview.
Much of programming is drudge work and few problems are unique to one project, so in principle applying AI to the task could work well. Microsoft's [24]IntelliCode , which uses machine learning to improve code completion, is fine; it can improve productivity without increasing the risk of errors.
AI-generated chunks of code is another matter and the story so far is that there is plenty of potential, but also plenty of snags. ®
Get our [25]Tech Resources
[1] https://www.theregister.com/2021/06/30/github_ai_copilot/
[2] https://openai.com/blog/microsoft/
[3] https://regmedia.co.uk/2021/07/06/copilot.jpg
[4] https://twitter.com/pkell7/status/1411058236321681414
[5] https://twitter.com/azjezz/status/1412054859877126151/photo/1
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://copilot.github.com/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[11] https://docs.github.com/en/github/copilot/research-recitation
[12] https://www.gnu.org/licenses/copyleft.en.html
[13] https://copilot.github.com/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOR@HrmfBECPtPAwYjwYCwAAAME&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[15] https://twitter.com/natfriedman/status/1409914420579344385
[16] https://twitter.com/eevee/status/1410037309848752128
[17] https://twitter.com/webmink/status/1410943130979942407
[18] https://www.uspto.gov/sites/default/files/documents/OpenAI_RFC-84-FR-58141.pdf
[19] https://blog.scottlogic.com/2021/07/03/github-copilot-first-thoughts.html
[20] https://www.theregister.com/2021/06/30/github_ai_copilot/
[21] https://www.theregister.com/2021/06/28/google_amp_core_web_vitals/
[22] https://www.theregister.com/2021/06/25/service_mesh_sdn_layer_7/
[23] https://www.theregister.com/2021/06/10/facebook_office_work/
[24] https://devblogs.microsoft.com/visualstudio/ai-assisted-intellisense-for-your-teams-codebase/
[25] https://whitepapers.theregister.com/
The final stage
Yet Another Anonymous coward
Copying code from Stackoverflow, as a Service by AI
Contrast with copy-pasting code from ... StackOverflow
Warm Braw
I'm not convinced that "like StackOverflow, only worse" is quite the AI we were promised.
Shy
elsergiovolador
The industry desperately does not want to pay the right money for the skill and talent.
This probably came from managers doing a coding bootcamp once or twice and thought writing a hello world microservice make them as good as anyone else at programming and so the AI not smarter than a slug could do that too.
Fast forward few years on. Microsoft will be trying to sell this to organisations promising they'll be able to grab anyone from the street to do "coding". When in reality they will be giving themselves a competitive advantage. While most companies trying to use that will be spending resources on training and then fighting fires created by this tool, Microsoft and other will be advancing their technology widening the gap.
In the 2-5 years, we will have "The Great Rewrite", where companies will be scrambling for specialists able to remove all copilot nonsense and rewrite the systems.
10-digit salaries will become the norm for developers.
The feedback problem
Anonymous Coward
This is a bad trend for the industry. The AI generates plausible looking but wrong code, which is then used in spite of it's bugs and so finds it's way onto GitHub. Next it is declared public domain and used for the future AI which is tainted by it's own effluence. It doesn't help that the first AI was trained on imperfect inputs to begin with. It's like a photocopy of a photocopy of a facsimile. The AI doesn't learn to avoid it's mistakes because it has no way of knowing what they were, instead it consumes it's mistakes and multiplies them. The only corrective feedback in this system is when a company goes bust because of it's crappy products/processes, and maybe not even then if GitHub preserves the crappy code in perpetua.
That's the problem with AI
fidodogbreath
All it knows is what you train it on. "Spilled secrets, bad code, and copyright concerns" are hallmarks of copypasta development.
problems including alleged spilled secrets, bad code, and copyright concerns
JDX
problems including alleged spilled secrets, bad code, and copyright concerns, though some see huge potential in the tool
Potential that this is just like the code their humans are already creating?
I don't know. IntelliCode has been helpful a couple of times. For example, I once had to edit a whole bunch of lines in a way that was very similar, but not so similar that it could be handled by a find/replace. After I did that for three or four lines, IntelliCode popped up with a suggestion that changed all of them in exactly the right way. That was nice.
However, in the vast majority of cases, IntelliCode suggestions are obviously-broken crap. Most of the time, its suggestions wouldn't even parse, which looks weird to me; surely they could at least filter out suggestions that don't pass the parser?
Now, I did read the article and I know that we are not talking about IntelliCode here, but I suspect the same basic problems apply. How many times you get a "wow" moment, compared to a crap moment? If the ratio is bad enough, you'll be better off not using it.