When Liran Ma’s mother visited Texas from China, the associate professor of computer science noticed her hearing was deteriorating.

Professor Liran Ma’s app uses Apple’s built-in functions and headphones to act as a hearing aid. Photo by Carolyn Cruz

Mother Ma, whose name is Ziyun Wang, is not alone. The Hearing Loss Association of America reports that as many as 48 million people in the United States suffer some degree of hearing loss. By 65, at least one in three people is moving in the direction of deaf.

Current hearing aid options are expensive and complex, Ma said. Devices cost an average of more than $2,000 per ear, and most private insurance companies, as well as Medicare, do not pay for hearing aids. Only one out of every five Americans with hearing loss uses a hearing aid, Ma said, and “cost is a significant barrier.”

Ma, whose primary research interest is in network security, decided to develop an iPhone app, using Apple’s built-in sound-processing tools and a pair of headphones, to work like a hearing aid at a fraction of the cost.

The professor needed to learn how hearing aids work, and by default, how humans hear. At first, he said he assumed “you just have to amplify the sound to a level that’s sufficiently loud for them so that they can hear again.”

The task would not be so simple.

Hearing loss discriminates among frequencies. The cochlea cells, which vibrate to transfer sound information to the brain, begin dying at the outside of the inner ear, and those outer cells are responsible for high-frequency sounds.

The loss of high-frequency hearing is a problem, as the sounds made by consonants register in the upper frequencies. People who cannot hear consonants have difficulty distinguishing between hard sounds, such as ‘p’ or ‘b.’ Consonants often differentiate words, enabling listeners to tell the difference between pig and big, for example.

To amplify just the high frequencies, the app needed to break sounds apart. For the mathematics behind manipulating portions of sound waves, Ma and his student research assistants sought the wisdom of Ken Richardson, professor of math at TCU. He taught them properties of the fast Fourier transform, an algorithm that can convert sound from time ranges to constituent frequencies, simplifying manipulation of specific frequencies.

Computer scientist Liran Ma (second from right) discusses coding issues with his small team of students who are developing an app that transforms a smartphone into a low-cost alternative to a hearing aid. Photo by Robert W. Hart

A phone’s processor can only handle so much sound at once, leaving a small discontinuity between processed segments. The app also needed to reduce noise. Filtering out background chatter allows the listener to focus on the desired sounds.

The app incorporated a “partition of unity,” which Richardson said “allows one to piece together things that are not smooth and create something smooth out of it.”

The total processing has to happen in a flash. If the delay between sound emission and hearing is more than 50 milliseconds, confusion and disorientation can result due to visual experiences being out of sync with audio.

Ma said he is planning future enhancements for the app, including “situational awareness,” which will rely on location-based data to estimate the average noise level around a person.

And instead of frequent trips to an audiologist to adjust a hearing device, the app will include “automatic fitting,” which instructs the user to say a sentence aloud, and then the app makes changes. “Based on [the listener’s] preference, the app can dynamically adjust the parameters,” Ma said.

The professor is still waiting to do a formal trial, but his mother is now using the app in China. She confirms it works great, Ma said.

Helping his mother and other people with age-related hearing loss is a good reason to spend so much time on his side project, Ma said. “If you cannot hear, it’s like living in the dark. … I want people to see the light again … so I will make the app easily available at a much lower cost.”

See the original infographic as it appeared in print

By Caroline Collier | Graphic by Mike Del Vechio. Copyright 2018 by Texas Christian University. Reproduction in whole or in part without permission is prohibited.