How swarm learning can improve AI collaboration without compromising data privacy

India is witnessing another surge in coronavirus cases in recent weeks.  The country, like many others globally, has invested extensively in every aspect of basic science, preclinical development, and clinical trials for vaccines. The results and data availability of vaccine trials directly affected the decisions of healthcare providers, the public, and policymakers evaluating its efficacy. Many regulations and data security requirements were loosened to help with the development of COVID-19 vaccines, making the medical research field realize the value of data sharing for coordinated solutions. 

Data, we all know, is valuable especially when organizations put it to good use. However, despite the data proliferating at the edge, not all of it can be shared due to privacy or security reasons. Fiduciary models, such as the draft Digital Personal Data Protection Bill 2022 which is currently under review in India, take a central role in regulating data privacy and protection. The inability to share information has led to an industry that doesn't advance as quickly as it could. Research efforts are often duplicated, and innovation is hindered.  

Hence, in order to avoid this, swarm learning offers an opportunity for collaboration and knowledge sharing for mutual gain. This data privacy-preserving framework that uses blockchain to decentralize machine learning-based systems, also encourages increased communication and data-driven knowledge sharing for organizational growth. 

In case of swarm learning, the source data never leaves the control of the entity that owns the data. Since the data itself remains private, businesses can retain the value of their data without triggering any regulatory compliance problems. In that sense, swarm learning enables organizations to harness the power of distributed data while protecting data privacy and security. 

As an example, applying swarm learning techniques to decentralize data analysis and sharing data insights can accelerate medical research. Doctors and medical researchers use AI to diagnose diseases based on X-ray images, blood values, or genome sequences. However, individual institutes usually do not have enough patient data for AI training and find it impossible to exchange such data with other institutes because of data protection rules. Swarm learning enables sharing of learning across individual institutes without sharing raw data to build a collaborative AI model. It has been proven that it achieves as good or even better results than centralized AI training. 

Interestingly, in swarm learning, not only the learning but also the aggregation of the learning takes place in a decentralized manner. Thus, it is not only suitable for use cases in which the data may not be shared due to regulations but also when companies do not want to exchange data because of competitive relationships. The procedure also makes sense if the data sets are so large that regular transfer would be too costly or slow. 

Credit card companies use AI to detect fraud at an early stage. AI training would be much more effective if the entire industry aggregated its transaction data, however, it’s practically impossible because it would expose the companies' confidential information. Similar situations exist in other industries too, for instance with IT security or predictive maintenance. 

Similarly, the recognition of images and patterns in video recordings is one of the most common AI applications, especially in autonomous vehicles and warships, in quality assurance, or in crime fighting. Video recordings often have a file size of many gigabytes. Regular transmission to a central location is expensive, consumes a lot of energy, and is slow. Swarm learning, thus, provides an uncompromisingly positive answer to the question if the goals of data science and digital sovereignty can be reconciled. 

Krishnaprasad Shastry

---

Krishnaprasad Shastry is the Senior Distinguished Technologist at Hewlett Packard Enterprise.