When ChatGPT was launched in November 2022, the entire AI ecosystem changed for people and companies. Subsequently, we witnessed a plethora of useful AI applications that significantly impacted the tech ecosystem. However, one segment was notably absent – Software Engineering. An AI tool capable of creating software, writing code, debugging it, and deploying it to production. With the introduction of Devin AI, the long wait came to an end as we now have a new AI tool acting as a software engineer – Devin.
What is Devin?
Cignition, a US-based applied AI lab, has introduced the world’s first AI software engineer – Devin. Devin can create software, code, debug, and deploy it to production instances. In fact, Devin has successfully passed practical engineering examinations and interviews conducted by numerous companies and institutions. It has also solved real problems posted on various freelancing platforms like Upwork and Fiverr.
What Can Devin Do?
Devin possesses the ability to strategize and execute complex engineering tasks involving multiple decisions. It can retain relevant information throughout the process, adapt and improve based on experience, and effectively rectify errors.Additionally, Devin is equipped with standard developer tools such as a shell, code editor, and browser in a secure computational environment – providing all necessary tools for human-like work.The creators stated that Devin can engage in real-time collaboration with users. It provides continuous updates on its progress, welcomes feedback, and collaborates with users on design decisions when necessary.
Example of Devin’s Work
An example showcased on the Cognition Labs blog demonstrates how Devin developed a game under a few instructions and even fixed its own bug. The example involved creating an interactive website simulating the Game of Life and deploying it to Netlify.
Devin vs Other AI Tools
In an official blog post about Devin, the creators evaluated its performance on SWE-bench, a challenging benchmark that requires agents to resolve real-world GitHub issues from open-source projects like Django and scikit-learn. Devin successfully resolves 13.86%* of these issues end-to-end, surpassing the previous state-of-the-art performance of 1.96%. Even when provided with exact files to edit, previous models could only resolve 4.80% of issues.
*Devin was evaluated on a random 25% subset of the dataset. Devin was unassisted, whereas all other models were assisted (meaning the model was told exactly which files need to be edited). | Source – Cognition Labs Blog
How to Access Devin?
Currently in early access stage, Cognition Lab is continuously enhancing Devin’s capabilities. If you wish to use Devin for engineering work, you can reach out to the Devin team by filling out this form.
My Thoughts on Devin
Having sought such a tool for some time now, I find the emergence of an AI software engineer like Devin truly groundbreaking. With AI’s increasing power and capabilities, this tool serves as a problem solver opening up numerous possibilities for the future.In their official blog conclusion, they mentioned that Devin is just the first step. It will be intriguing to see how this tool evolves and brings about transformative changes in the field.
