Microsoft researchers show how deep learning can be used to identify and fix bugs in software codes

Researchers at Microsoft have created a Deep Learning (DL) model called BugLab that has been trained on a game instead of labelled data to detect and fix bugs in software codes. The researchers presented their findings at the 2021 Conference on Neural Information Processing Systems (NeurIPS 2021).  

A subset of Machine Learning (ML), DL is a neural network with three or more layers. Unlike ML models that are used on structured and labelled data, DL models are used on unstructured and more complex datasets.  

“The goal of our work is to develop a better Artificial Intelligence (AI) that can find and repair bugs that seem simple but are hard to find,” Miltos Allamanis, principal researcher and Marc Brockschmidt, senior principal research manager at Microsoft said in a blog post.  

Also read: Microsoft disrupts Chinese hack targeting cos across 28 nations

However, training ML models to automatically identify bugs is not that simple. It's further complicated by lack of quality data, the researchers lamented. It is what led them to turn to a "hide and seek" game to train the model. They said that the game is inspired by Generative Adversarial Networks (GAN) but is not entirely based on it.  

Commonly used to create Deepfakes, GAN is an advanced subset of ML that takes two models and pits them against each other. One model is used as a generator and trained to create new examples while the other model is used as a discriminator and trained to identify fake examples. The objective is to make the first model better by using the second.  

On their part, Microsoft researchers also created a bug selector model (generator) to introduce a bug and a bug detector (discriminator) model to identify, locate and fix the bug.  

“Our bug selector does not generate a new code snippet from scratch and rewrites an existing piece of code that is believed to be correct,” they added.  

Unlike GAN which is used to find a good generator, the focus of Microsoft’s research was to find a better discriminator, where the selector trains the detector to identify and fix bugs.  

Microsoft researchers said they put the detector to detect and fix bugs in real-life code. They manually annotated a small dataset of bugs from the Python Package Index and found that the ML model trained on the “hide-and-seek” game was up to 30% more accurate at detecting bugs as compared to others.  

They claim that the detector has already found 19 previously unknown bugs in real-life open-source GitHub code.  

Researchers feel their AI model would allow developers to focus on the more critical elements of software development. This also resonates with the larger developer community. According to Rollbar’s State of Software Code report, published February 2021, fixing bugs and errors is the biggest pain point for 44% of developers while 37% said they spend over 25% of their time on it.  

Failure to detect and fix bugs in the initial stages of the software’s development can further heighten the risk of future cyberattacks. Once multiple layers of codes are added, detection of bugs becomes even more difficult. According to a WhiteHat Security report, published June 2021, the average time taken to fix critical vulnerabilities was 205 days.