Google is using retinal images to assess cardiovascular risk factors

Tech giant Google has said that it has been trying to use deep learning techniques on one of its computer vision engines to try and assess cardiovascular risk factors, with heart attacks and strokes continuing to be among the top public health issues.

Computer vision tasks include methods for acquiring, processing, analysing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information.

"Using deep learning algorithms trained on data from 284,335 patients, we were able to predict CV [cardiovascular] risk factors from retinal images with surprisingly high accuracy for patients from two independent datasets of 12,026 and 999 patients," Lily Peng, product manager at Google Brain Team, wrote in a blog post.

Doctors usually take into account various risk factors for patient assessment such as some genetic data (like age and sex) and data related to lifestyle components (such as a smoking and blood pressure).

While most of these factors can be ascertained by simply asking the patient, other factors such as cholestrol require drawing blood. Other disease factors such as diabetes significantly increases risk.

According to Peng, the computer vision algorithm can distinguish between the retinal images of a smoker from that of a non-smoker at least seven out of 10 times.

In addition, while doctors can typically distinguish between the retinal images of patients with severe high blood pressure and normal patients, the algorithm could go further to predict the systolic blood pressure within 11 mmHg on average for patients overall, including those with and without high blood pressure.  

"Recently, we’ve seen many examples of how deep learning techniques can help to increase the accuracy of diagnoses for medical imaging, especially for diabetic eye disease," Peng explained. "In addition to detecting eye disease, images of the eye can very accurately predict other indicators of CV health. This discovery is particularly exciting because it suggests we might discover even more ways to diagnose health issues from retinal images."

Google said that its algorithm used the entire image to quantify the association between the image and the risk of heart attack or stroke. 

"Given the retinal image of one patient who (up to 5 years) later experienced a major CV event (such as a heart attack) and the image of another patient who did not, our algorithm could pick out the patient who had the CV event 70% of the time. This performance approaches the accuracy of other CV risk calculators that require a blood draw to measure cholesterol," Peng said.

Peng also said that Google was able to understand how the algorithm was making its prediction. According to her, the operational methodology of the algorithm can in future allow Google to generate a heatmap that shows which pixels were the most important elements for a predicting a specific CV risk factor. 

For example, the algorithm paid more attention to blood vessels for making predictions about blood pressure, Peng said. She added that explaining how the algorithm is making its prediction gives the doctor more confidence in the algorithm itself. 

The search giant said that it is looking forward to developing and testing the algorithm on larger and more comprehensive datasets to make it more useful for patients and doctors. 

"To make this useful for patients, we will be seeking to understand the effects of interventions such as lifestyle changes or medications on our risk predictions and we will be generating new hypotheses and theories to test," Peng said.