news
GitHub Copilot tops AI code assistant research report
GitHub topped research firm Gartner's first-ever Magic Quadrant report for AI code assistant vendors, coming in at the top for both completeness of vision and ability to execute.
This may not come as a surprise, as GitHub Copilot was the first AI coding assistant that broke the constraints of machine learning about three years ago, helping to usher in the GenAI era with its deep learning and natural language processing (NLP) capabilities.
Gartner currently describes the market as follows:
Gartner defines an AI code assistant as a tool that helps generate and analyze software code and configuration. The assistant uses an underlying model, such as a large language model (LLM), optionally fine-tuned for the code, or program understanding technologies, or a combination of both. Software developers instruct the code assistant to generate, analyze, debug, fix, refactor code, document, and convert code between languages. Code assistants are integrated into developer tools such as code editors, command line terminals, and chat interfaces; some can be customized to an organization's specific codebase and documentation.
In general, these tools have advanced rapidly since Copilot's launch, evolving beyond the simple code completion suggestions of the early days. Gartner now says that AI code assistants work for a variety of use cases.
Code generation: Developers use the AI Code Assistant code editor to auto-complete code and generate functions, helping them complete programming tasks faster. Debug code: Developers use the AI Code Assistant to find and fix bugs in their code, helping them resolve errors without asking colleagues or searching the internet for solutions. Modernize code: Developers use the AI Code Assistant to understand complex dependencies across many programs, helping them reduce technical debt and modernize their code. Build and test artifacts: Developers use the AI Code Assistant to generate acceptance tests from user stories (e.g., in Gherkin format) and generate unit tests. Explain code: Developers use the AI Code Assistant to get natural language explanations of their code, helping them understand complex and unfamiliar code.
The first report to look at the AI-powered code suggestions and contextual assistance that GitHub Copilot provides states, “The company's business is geographically diversified, and its customers tend to be large organizations across various sectors. GitHub offers GitHub Copilot for free to active maintainers in the open source community, as well as teachers and students. GitHub extends Copilot with capabilities such as Copilot Workspace for a collaborative, AI-native development environment, Copilot Extensions for seamless tool integrations, and enhanced security and compliance.”
Joining GitHub Copilot in the Leaders quadrant are Google Cloud, Amazon Web Services (AWS), and GitLab.
(Click image to enlarge.) Magic Quadrant for AI Code Assistants (Source: Gartner).
Gartner's must-have capabilities for this emerging market include:
Code completion from natural language (e.g. comments). Multi-line mid-entry code completion with the ability to plug in multiple code editor integrations. Ability to use code assistance across multiple vendor ecosystems. Ensuring that base models are not trained on customer code or documentation (except for approved tweaks). A conversational chat interface integrated into the development environment.
Other common features include everything from on-premise or private cloud instances to filters for biased code, explicit language and images.
“The vision behind GitHub Copilot is simple: harness the power of generative AI to fuel the innate human creativity in every developer,” GitHub, a Microsoft subsidiary, said in a blog post releasing the report last week. “Our goal isn't to create technology for technology's sake. It's to make all developers happier and more productive by enabling them to work longer, work faster, and lower the barrier to entry. With millions of developers and more than 77,000 organizations using Copilot, we feel we're making rapid progress toward that goal and delivering on our vision.”
Considering that two of the four leaders are cloud giants, and GitHub is also owned by a cloud giant, the only leader outside of the cloud giants is GitLab, which offers an AI-powered DevSecOps platform.
“AI code assistants are more than just code generation and completion,” GitLab said in its post celebrating the report. “They're collaborative partners that make developers more efficient by improving code quality and continuously learning. By automating routine tasks and providing intelligent suggestions, assistants like GitLab Duo – our suite of AI-powered capabilities – free up developers' time to focus on solving higher-level problems.”
Duo is GitLab's offering in this space, while Google was recognized for its Gemini Code Assist tool and AWS for Amazon Q Developer, formerly known as Codewhisperer.
In addition to the vendor assessment, the report provides several strategic planning assumptions that illuminate Gartner's market outlook.
By 2027, the number of platform engineering teams using AI to enhance all phases of the software development life cycle (SDLC) will increase from 5% to 40%. By 2027, 80% of enterprises will integrate AI-augmented testing tools into their software engineering toolchain. This is a significant increase from approximately 15% in early 2023. By 2027, 25% of software defects leaking into production will be due to lack of human oversight of AI-generated code. This is a significant increase from less than 1% in 2023. By 2028, 90% of enterprise software engineers will use AI code assistants. This is an increase from less than 14% in early 2024. By 2028, the use of generative AI (GenAI) will reduce legacy application modernization costs by 30% from 2023 levels.
While Gartner typically charges for its reports, this report, and many of its Magic Quadrant reports, are available free of charge from evaluated vendors who are authorized to provide distribution license copies, which can be found with a quick Internet search.
About the Author
David Ramel is an editor and writer at Converge360.
