Multi-modal AI takes the lead to power integrated healthcare

The pandemic unfolded many things besides a virus we knew very little about. As treating COVID-19 comorbid patients became challenging, we faced the underlying threat of comorbidities. 

According to health ministry data, 17.9% of all fatalities were among patients with pre-existing conditions, including hypertension, diabetes, and lung diseases like TB. As the condition was still evolving in 2020-21, hospitals struggled to develop risk stratification and prioritise care for high-risk patients, leading to a high mortality rate.  

However, there were many lessons learnt as we battled the virus. Healthcare providers turned to technology to improve diagnostic accuracy by integrating data from multiple sources to identify the most effective course of treatment. The aftermath of the pandemic, the increase in comorbidities and cancer cases, and the democratisation of healthcare are some of the numerous reasons exposing a great need to leverage AI technology in patients’ care. 

As precision healthcare becomes the next natural step ahead, for the first time, we see multimodal AI being leveraged to enable doctors to predict responses to treatment and prognosis for diseases like cancer and coronary heart disease. A revolutionary step.  

The role of integrating clinical data

Artificial intelligence (AI) and machine learning (ML) systems are poised to become fundamental tools in next-generation clinical practice and healthcare operations. Improving clinical efficiency and patient outcomes requires knowledge of multiple data sources and input modalities.  

Leveraging AI's power to improve clinical efficiency and patient outcomes in an interconnected world will require integrating multiple data sources and input modalities. Integrating multiple variables, including medical imaging, patient photos, verbal and written notes from physicians, lab results, patient surveys, health diagnostics, electronic health records (EHR), and electronic medical records (EMR), across different modalities helps doctors investigate and make accurate decision-making.  

For instance, in treating cancer, integrating data allows doctors to examine the probability of cancer recurrence based on disparate data, helping them decide the best time to begin chemotherapy while minimizing side effects. Deep learning algorithms extract biomarkers for prostate cancer and fuse them with a Prostate Specific Antigen to help a cardiologist predict the cardiac side effects of chemotherapy on a comorbid patient.  AI algorithms scan terabytes of clinical data, including images, lab tests and medical records, to make room for proactive decision-making.  

By combining multi-modal data, AI can help doctors put their clinical observations into context and eliminate overtreatment.

Multi-modal AI for diagnosis and treatment

In many cases, doctors have been able to personalize treatments with comparable clinical outcomes and better quality of life. In addition, as researchers develop new learning models, there's increasing hope that patients will benefit from novel, more effective therapies, including non-surgical ones. 

Soon, AI may be able to recommend a treatment based on the data at its disposal. While this will require the development of new AI models along with extensive testing and validation, it could only be a matter of time before oncologists and cardiologists have access to a ‘digital associate’ to advise them on treating some of the most complex cases. That said, multimodal AI is unlikely ever to replace the human touch.

Synchronizing clinical data

Synchronizing the time stamps of various medical devices across departments is challenging. Doctors can’t be sure they have the latest data without precise time alignment. This can make multimodal data interpretation problematic. Multimodal AI measures the differences in signal morphology and compensates for sync delays across different devices. This is proving to be a game-changer in data accuracy and integrity, improving patient outcomes and reducing hospital operating costs. 

Conclusion

The goal is to gauge which tests would lead to a favorable outcome, eventually reduce the cost burden, improve outcomes, and deeply consider the patient's quality of life as a critical factor. While still in its infancy, the potential impact of such models in patient care cannot be undermined. Once the initial bottlenecks are overcome, multi-modal will be an essential step towards personalized medicine that will enhance patient care and improve patient outcomes.

Rakesh Mullick

---

Rakesh Mullick is the Chief Scientist at GE Healthcare’s Edison AI.