Jacoti presents our test-retest analysis of hearing thresholds measured with Jacoti Hearing Center

Fifth International Meeting on Internet & Audiology

Internet & Audiology International Meeting it’s a forum for researchers and practitioners to share their knowledge and facilitate ways to deliver high-quality care remotely and support persons with hearing problems over the internet.

This year the forum was focused on: Patient-centered care and e-Health, Social media and hearing care, Online audiological rehabilitation, and Implementation science.

Jacoti has presented its test-retest analysis of hearing thresholds measured with the Jacoti Hearing Center self-testing app

Transcript:

Hello everyone, and welcome to this presentation.

My name is Joanna Luberadzka and I have recently been working as Research Intern at Jacoti.

One of the main products of Jacoti is a hearing diagnostics smartphone application called Jacoti Hearing Center. Anyone with an iPhone and appropriate earpods can download the app for free and test their hearing at home without going to a hearing professional.

Today I will tell you about a study which we did use audiological data collected by this app. Let’s start with an introduction.

Smartphone-based audiometry, like Jacoti Hearing Center, allows performing a hearing test when there’s no direct access to a clinical audiologist.

This might be a practical solution in less privileged societies, but it’s also important in all countries to minimise the contacts between patients and healthcare providers in the time of the global pandemic.

However, there are several potential limitations of a test performed at home with a smartphone. In contrast to the clinical test, it does not ensure the acoustically treated room or dedicated equipment, like an audiometer or calibrated headphones. No specialist controls the procedure. Therefore, smartphone diagnostics have to implement a variety of solutions to compensate for these factors.

In Jacoti Hearing Center, professional assistance is substituted with detailed instructions and requirements screen. The role of the audiometer is taken by a user-friendly automated procedure called DuoTone®. DuoTone® is a patented procedure in which audiometry thresholds are measured in pairs, using a continuous tone stimuli for one frequency and a burst of short tones for the other frequency. The application also warns the user to use appropriate headphones. And, to ensure a quiet test environment, the in-built noise-monitoring system is used.

Despite all these solutions, the question if smartphone-based audiometry is a valid diagnostic tool is still openly discussed. How could we evaluate this?

Although smartphones are still relatively new, automatic audiometry has been there for a long time and there are some established ways of assessing the overall quality of an automatic test.

On one hand, we can measure the accuracy by comparing thresholds with automated audiometry with thresholds from conventional clinical audiometry, both under the same conditions. On the other hand, we can measure the reliability or the robustness of the method by checking how consistent are the results of the automatic audiometry between two different test run. And this is what we call the test-retest reliability. And this is the main objective of the study that I’m presenting here today.

There are already several studies measuring test-retest reliability of smartphone audiometry under controlled conditions. This means audiograms were typically measured in soundproof booths using calibrated headphones under the experimenter’s assistance.

The results of such studies are promising, therefore we wanted to go one step further and analyse the test-retest reliability of a smartphone app in real-life conditions. That is actually intended for this type of apps.

Thanks to the Jacoti database, which contains measurements performed by real users of this app, we were able to approach this question. However, as you might imagine, the audiological data collected by a smartphone app has its own nature. The advantage is having access to thousands of audiometry sessions from various iOS users worldwide. Such a population is rarely available for clinical studies. But there are also challenges. Even with the noise monitoring system, the app cannot fully control the acoustical environment or the attitude of the users; Patients can abort the test at any point or engage in another task simultaneously. Another important aspect is that the range of measurable thresholds values is limited, which we call the bottom and ceiling effect. At the bottom, the range of measurable values is limited because is not possible to measure precisely the very low thresholds with ambient noise. At the other end of the scale, we have the device up to output power limit, which prevents playing back sounds uncomfortable for the user.

All these factors have to be taken into account when performing the test-retest analysis, and now I will present to you our approach.

To begin with, we used the Jacoti database, which is containing audiometry sessions from almost 10,000 users. Next, we found that all consecutive test-retest session pairs originating from the same user. Next, we identified common frequencies in the test-retest session pairs. Based on these common frequencies, we analysed the test-retest threshold deviation, and we eliminated bottom and ceiling thresholds, we computed threshold differences for each available frequency and average threshold difference across frequencies. Finally, we quantified the percentage of tests written as differences within the margin of 5 dB and 10 dB.

I will demonstrate the method based on one example of the test-retest session pair, which we can see here.

This row represents 12 frequencies which can be measured with the Jacoti audiometry test.

Here you can see all thresholds that were measured in the first session, and here all thresholds measured in the second session.

In this session pair, there are only 8 common frequencies, which you can see here. And, for these 8 common frequencies, we compute differences between the first and the second measurement. Some of these differences come from threshold values that are at the bottom or at the ceiling of the measurable scale. These differences are excluded from the analysis. The remaining values contributed to the overall statistics of threshold differences in individual frequencies and to the average threshold difference between two sessions.

With this approach, we analysed the following conditions.

Two different test procedures are available in Jacoti Hearing Center: Full audiometry, with 12 frequencies, and screening test, for 4 frequencies.

We also analysed two different data sets. One consisting of all available session pairs and the second one includes only fully completed session pairs.

In the table on the right, you can see also the number of sessions pairs and the number of total users from the database we analysed in each condition.

Let’s see the results.

This table presents the most important results. We can see what percentage of threshold differences lie within the margin of 5dB and 10dB.

As I mentioned before, this is computed for thresholds differences in individual frequencies and also for average threshold differences.

We can see that between 60% and 77% of threshold differences in individual frequencies and 72% and 86% of average thresholds differences lie within the margin of 5dB. Results for complete session pairs are better than for the incomplete session pairs, which means that the test is more reliable when performed by the users who were motivated to measure all frequencies.

Now looking at the 10dB margin, we can see that between 81% and 90% of threshold differences in individual frequencies lie within that margin. Also between 86% and 89% of average threshold differences are less than 10dB. Here again, results for complete session pairs are slightly better.

Given these results, is the smartphone-based audiometry of Jacoti Hearing Center as reliable as conventional audiometry? To answer this question we can compare our results with their values for clinical audiometry from the literature. But we can only do this comparison for specific groups of participants who took part in this clinical study.

The first group is the group of normal-hearing participants. Based on several studies, in total 64 participants, we computed the weighted average for 5 different frequencies. We compared it with 275 normal-hearing participants of the Jacoti audiometry. We can see that, depending on the frequency, Jacoti full audiometry is a bit more or a bit less reliable than clinical audiometry. Still, in all frequencies, the majority of test-retest differences are in the margin of 5dB.

We also compared 217 Jacoti users with mild to moderate hearing loss with 32 participants of a clinical study who had either normal hearing or high-frequency hearing loss. Here, again, the comparison depends on the frequency. For low frequencies, Jacoti tends to be worse than clinical audiometry, but for high frequencies, it is better.

With this we almost reached the end of this presentation, so now let me summarise what I talked about:

• At the beginning of this presentation, we argued that smartphone-based audiometry provides an alternative solution for hearing diagnostics, which has to be evaluated.
• To do this, we analysed the test-retest reliability of the Jacoti Hearing Center iOS app based on real-life user data.
• We proposed methods for dealing with incomplete data and with bottom or ceiling effects.
• Our results showed that the majority of the test-retest threshold differences lie within the margin of 5dB.
• This suggests that Audiometry and Screening tests provided by Jacoti Hearing Center as reliable as clinical audiometry.
• Consequently, the hearing test can be integrated into a calibrated hearing device that also performs hearing loss compensation.

And with this, I would like to thank you very much for watching this presentation and I invite you to check more information about the study and Jacoti products in general.