Copilot AI: here is an innovation from Cydrill

Copilot AI, not to be confused with Microsoft's namesake, integrated into the Bing search engine.

Imagine a world where the software that powers your favorite apps, protects your online transactions, and maintains your digital life could be overtaken and targeted by a cleverly disguised piece of code and this is not the plot of the latest cyber-thriller; it is a reality that has existed for years.

How this will change in a positive or negative direction as artificial intelligence (AI) takes on an ever-larger role in software development it is one of the great uncertainties associated with this brave new world.

In an era where AI promises to revolutionize the way we live and work, the conversation about its security implications cannot be set aside; as we rely more and more on AI for tasks ranging from the mundane to the fundamental, the question is no longer just: “Can AI enhance cybersecurity?” (of course!), but also “Can AI be hacked?” (Yes!), “Can AI be used to hack?” (of course!), and “Will AI produce safe software?” (Well…).

This Copilot was created by Cydrill (a secure coding training company) ed explores the complex landscape of vulnerabilities produced by AIwith a special focus on GitHub Copilot, to highlight the imperative of secure coding practices in safeguarding our digital future.

The AI ​​Safety Paradox

AI's leap from academic curiosity to a cornerstone of modern innovation happened rather suddenly; its applications span a wide range of industries, offering solutions that were once the stuff of science fiction.

However, this rapid advancement and adoption have outpaced the development of corresponding security measures, leaving both AI systems and AI-created systems vulnerable to a variety of sophisticated attacks.

At the heart of many AI systems is machine learning, a technology that relies on large data sets to “learn” and make decisions. Ironically, AI's strength is its ability to process and generalize from vast amounts of data it is also his Achilles heel.

The starting point of “whatever we find on the Internet” may not be the perfect training dataset; unfortunately, the wisdom of the masses may not be enough in this case; furthermore, hackers, armed with the right tools and knowledge, can manipulate this data to trick the AI ​​into making incorrect decisions or carrying out malicious actions.

Copilot in the crosshairs

GitHub Copilotpowered by Codex of OpenAI, is a testament to the potential of AI in coding and was designed to improve productivity by suggesting code snippets and even entire blocks of code.

However, multiple studies have highlighted the dangers of relying completely on this technology; It has been demonstrated that a significant portion of the code generated by Copilot may contain security flaws, including vulnerability to common attacks such as SQL injection and buffer overflows.

The principle of “garbage in, garbage out” (Garbage in, garbage out or GIGO) is particularly relevant here. AI models, including Copilot, are trained on existing data, and just like any other Large Language Model, most of this training is unsupervised.

If this training data is flawed (which is very possible given that it comes from open source projects or large Q&A sites like Stack Overflow), the output, including code suggestions, could inherit and propagate these flaws. In the early days of Copilot, a study revealed that approximately 40% of the code samples produced by Copilot when called upon to complete the code based on samples from the Top 25 of the CWE were vulnerable, underscoring the GIGO principle and the need for greater security awareness.

A larger-scale study in 2023 (Is GitHub's Copilot as inefficient as humans at introducing vulnerabilities into code?) had slightly better results, but still far from good.

Removing the vulnerable line of code from the real-world vulnerability examples and asking Copilot to complete it, it recreated the vulnerability about 1/3 of the time and only fixed the vulnerability about 1/4 of the time; theFurthermore, it performed very poorly on vulnerabilities related to lack of input validation, producing vulnerable code every time.

This highlights that generative AI is poorly equipped to handle malicious inputs if “panacea” solutions to address a vulnerability (e.g. prepared instructions) are not available.

The path to secure AI-powered software development

Address the security challenges posed by AI and tools like Copilot requires a multifactorial approach:

1. Understanding Vulnerabilities: It is essential to recognize that AI-generated code can be susceptible to the same types of attacks as “traditionally” developed software.
2. Elevate secure coding practices: Developers must be trained in secure coding practices, taking into account the nuances of AI-generated code; this involves not only identifying potential vulnerabilities, but also understand the mechanisms through which AI suggests certain code fragments, to anticipate and mitigate risks effectively.
3. Adapt the SDLC: It's not just technology; processes should also take into account the subtle changes that AI will bring. When it comes to Copilot, code development is usually the focus, but requirements, design, maintenance, testing, and operations can also benefit of Large Language Models.
4. Continuous supervision and improvement: AI systems, as well as the tools they power, are constantly evolving, and keeping up with this evolution means staying informed about the latest security research, understand emerging vulnerabilities and update existing security practices accordingly.

Navigating the integration of AI tools like GitHub Copilot into your software development process is risky and requires not only a shift in mindset, but also the adoption of robust strategies and technical solutions to mitigate potential vulnerabilities.

Here are some practical tips designed to help developers ensure that their use of Copilot and similar AI-powered tools increase productivity without compromising security.

Below is a small “guide” on how to use a tool that is as powerful as it is useful.

Implement strict input validation

Practical implementation: Defensive programming is always at the heart of secure coding. When you accept code suggestions from Copilot, especially for functions that handle user input, implement rigorous input validation measures.

Define rules for user input, create a whitelist of allowable characters and data formats, and ensure that inputs are validated before processing and you can also ask Copilot to do it for you; sometimes it works really well!

Manage dependencies safely

Practical implementation: Copilot may suggest adding dependencies to your project, and attackers may exploit this to implement supply chain attacks via “packet hallucination.”

Before incorporating any suggested libraries, manually verify their security status by checking known vulnerabilities in databases such as the National Vulnerability Database (NVD) or perform a software composition analysis (SCA) with tools like OWASP Dependency-Check or npm audit for Node.js projects; these tools can automatically track and manage the security of dependencies.

Conduct regular security assessments

Regardless of the source of the code, be it AI-generated or hand-crafted, conduct regular code reviews and testing with a focus on security; Combines the approaches, test statically (SAST) and dynamically (DAST), do a software composition analysis (SCA). Do manual and integrated testing with automation.

But remember to put people above tools: no tool or artificial intelligence can replace natural (human) intelligence. So be gradual!

First, let Copilot write your comments or debug logs it's already pretty good at these and any errors in these won't affect the security of your code anyway; then, once you're familiar with how it works, you can gradually let it generate more and more code snippets for the actual functionality.

Always review what Copilot offers

Practical implementation: Never blindly accept what Copilot suggests; remember, you are the pilot, and Copilot is “just” the Copilot! You and Copilot can be a very effective team together, but you're still in charge, so you need to know what code is expected and what the end result should look like.

Experiment

Practical implementation: Try different things and stimuli (in chat mode). Try asking Copilot to refine the code if you're not happy with what you got; try to understand how Copilot “thinks” in certain situations and realize its strengths and weaknesses. Furthermore, Copilot improves over time and therefore continually experiments!

Stay informed and educated!

Continue to educate yourself and your team on the latest security threats and best practices; follow security blogs, attend webinars and workshops and participate in forums dedicated to secure coding.

Knowledge is a powerful tool in identifying and mitigating potential vulnerabilities in code, generated by AI or not.

Conclusion

Yes, you understood correctly this tool is always one of the various ChatGPT, Gemini or whatever family, with the difference that it is structured 100% for programming.

The importance of secure coding practices has never been more important as we navigate the uncharted waters of AI-generated code.

Tools like GitHub Copilot present significant opportunities for growth and improvement, but also particular challenges when it comes to the security of your code; only by understanding these risks can we successfully balance effectiveness with security and keep our infrastructure and data protected.