From coughing to talking to breathing, the sounds our bodies make are packed with health information. The subtle clues hidden in these bioacoustic sounds have the potential to revolutionize how we test, diagnose, monitor, and manage a range of health conditions, including tuberculosis (TB) and chronic obstructive pulmonary disease (COPD). As Google researchers, we recognize the potential for sound to be useful health signals, and we also recognize the widespread availability of smartphone microphones. That's why we've been exploring how we can use AI to derive health insights from acoustic data.
Earlier this year, Google announced Health Acoustic Representations (HeAR), a bioacoustic-based model designed to help researchers build models that can listen to human sounds and flag early signs of disease. The Google Research team trained HeAR using 300 million pieces of audio data curated from diverse, anonymized datasets, and specifically trained the cough model using approximately 100 million cough sounds.
HeAR learns to discern patterns in health-related sounds, building a strong foundation for medical speech analysis. We found that, on average, HeAR outranked other models in generalization across a wide range of tasks and microphones, demonstrating a strong ability to capture meaningful patterns in health-related acoustic data. Models trained with HeAR achieved high performance with less training data, a critical factor in the data-scarce world of medical research.
HeAR is now available to researchers to help accelerate the development of custom bioacoustic models while reducing data, setup, and computational effort. Our goal is to enable further model exploration for specific conditions and populations, even where data is sparse or there are cost or computational barriers.
Salcit Technologies, an India-based respiratory healthcare company, has developed a product called Swaasa® that uses AI to analyze cough sounds to assess lung health. Now, the company is looking into how HeAR can help extend the capabilities of its bioacoustic AI models. First, Swaasa® is using HeAR to help research and enhance early detection of tuberculosis based on cough sounds.
TB is a treatable disease, yet millions of cases go undiagnosed each year. This is often due to people not having easy access to healthcare services. Improved diagnostics is essential to eradicate TB, and AI can play a key role in improving detection and making treatment more accessible and affordable for people around the world. Swaasa® has a proven track record of bridging the gap in accessibility, affordability, and scalability by using machine learning to aid in early detection of the disease and provide location-independent, device-free respiratory health assessments. At HeAR, we see an opportunity to build on this research and scale TB screening more widely across India.
“Missing TB is a tragedy, and a delayed diagnosis is heartbreaking,” says Sujay Kakarmath, a product manager at Google Research working on HeAR. “Acoustic biomarkers have the potential to rewrite this narrative, and we're deeply grateful for the role HeAR can play in this transformational journey.”
There has also been support for this approach from organizations such as the StopTB Partnership, a United Nations-sponsored organization that brings together TB experts and affected communities with the goal of eliminating TB by 2030.
“Solutions like HeAR, enabling AI-powered acoustic analysis, will break new ground in TB screening and detection, providing low-impact, easily accessible tools to those who need them most,” said Zhi Zhen Qin, Digital Health Specialist at the Stop TB Partnership.
HeAR is a major step forward in acoustic health research. We hope to advance the development of future diagnostic tools and monitoring solutions in tuberculosis, chest, pulmonary and other disease areas, and contribute to improving the health of communities around the world through research. Researchers interested in investigating HeAR can learn more and request access to the HeAR API.
