The Google Pixel Smartphone for People with Hearing Loss – an Analysis



Published: September 1, 2020

 

The Google Pixel Smartphone for People with Hearing Loss – an Analysis

Apple made significant effort to make the iPhone accessible to individuals who are blind.  The result is that Apple captured hundreds of thousands (if not millions) of loyal customers with vision loss.  By contrast, Google is currently investing significant resources to offer an unparalleled level of accessibility to individuals who are deaf or hard of hearing.  It remains to be seen whether Google will be rewarded with a similar level of loyalty, but Google’s efforts have yielded impressive results.

This analysis is focused on the Pixel smartphone because, as we will see, Google has prioritized its own devices for the latest accessibility features, perhaps because some of these features require an integration of both hardware and software elements.   

Google has adopted a multi-faceted approach to making its Pixel smartphones accessible to individuals with hearing loss.  First, it has provided the devices with the ability to caption everything and anything.  If it has audio, it can be captioned.  Second, Google has focused on amplifying audio for people who are hard of hearing.  Third, the Pixel offers hearing aid support. 

Each of these categories will be examined in turn.   

A. If it has Audio, it can be Captioned

1. Live Caption for Media and Voice and Video Calls

A feature called “Live Caption” provides users the ability to caption media, as well as voice and video calls.  Media that can be captioned includes videos, podcasts and internet radio.  Live Caption provides a transcription of your internal audio stream, so that the user can access captions in virtually any app.  Mobile devices that do not have this capability, such as the iPhone, rely on the app or website itself to generate the captions.  If the app or website does not offer captions, the user is out of luck.  A simple example: podcasts are becoming a ubiquitous experience.  More than half of Americans over the age of twelve (12) have listened to a podcast.  Thirty-two percent (32%) listen to at least one podcast each month.  However, podcast apps rarely offer captions, preventing individuals with hearing loss who have an iPhone, or an alternative android device, a valuable source of entertainment and education.  On the other hand, the Pixel’s Live Caption feature will caption any podcast, regardless of the app used to play that podcast. 

Similarly, traditional radio is a domain that has never been fully accessible to individuals with hearing loss.  Internet radio is no exception.  Internet radio apps, such as TuneIn Radio do not offer captions.  The Google Pixel, however, can caption all internet radio programming (with the exception of music). 

As a result of Live Caption, people with hearing loss can now access entire categories of content that was previously inaccessible.   

Currently, Live Caption is available in English only, with additional languages available in the future.  Captions appear as a bubble that can be moved around the display.  There is an option to hide profanity and to show sound labels, such as laughter, music and applause.  Live Caption can be turned on by pressing the volume rocker and tapping on the caption symbol.  An alternative, more cumbersome way to turn on Live Caption, is to do so through the Accessibility settings.

The captions are generated on the device, not in the cloud.  This guarantees privacy and reduces latency.  On the other hand, captions generated on the device are not quite as accurate as those generated in the cloud since Google can apply a more complex algorithm in the cloud.    

As of August 2020, Live Caption supports phone calls, as well as voice and video calls placed with so-called “over-the-top” applications.  Thus, Pixel owners can now caption calls made with the native Pixel phone app, as well as voice or video calls made with Google Duo, WhatsApp, Viber, Skype and others.  This is the first time that individuals who rely on captions in order communicate can use the same communications apps as the general public.   

       

Live Caption for media is available on all Google Pixel devices going back to the Pixel 2, as well as the Samsung Galaxy S20, S20+ and S20 Ultra.  The OnePlus 7t and OnePlus 8 also offer Live Caption.  We can expect the Live Caption feature to roll-out on additional android devices in the future.  However, Live Caption for voice and video calls is available only on the Pixel.  It may come to other android devices in 2021. 

B. Live Transcribe and the Google Recorder to Caption Conversations

1. Live Transcribe

Android offers an app called Live Transcribe to individuals who wish to caption in-person conversations in near-real time.  Unlike, Live Caption, Live Transcribe is widely available to android devices through the Google Play Store. 

The hallmark of the app is its simplicity.  When the app is opened it immediately begins transcribing what it hears.  The text size is adjustable and can be made quite large.  The transcription is very accurate and adds punctuation.  It even understands some context and capitalizes words accordingly. 

Unlike Live Caption, the transcription for Live Transcribe occurs in the cloud on Google’s servers.  This means that Google has access to the conversations, which raises privacy concerns.  Google claims, however, that it does not save the transcriptions or audio and that it does not use this information to improve its algorithms. 

Because the transcriptions are generated in the cloud, Live Transcribe requires a good internet connection to work.  This limits the use of the app in locations where internet connectivity is weak or non-existent.       

The app has a few additional features that increase its usability.  A blue circle in the top right corner pulses.  The more it pulses, the greater the ambient noise level and the closer the user needs to move the microphone to the speaker for it to work effectively.  At the bottom of the display, there are a few buttons, including one to bring up a keyboard to type out replies.  This is intended to be used by people who have difficulty speaking.  The size of the window that contains the reply is adjustable.  The bottom of the display also has a button to pause the transcription so that the user can catch up to what is being spoken.

There is an option to save transcriptions for three days.  If the user wishes to save a transcription permanently, they can copy and paste the transcription into another document, such as a Word document.

Four other useful options include the ability to 1) vibrate after speech resumes following a pause in order to get the user’s attention when the speaker starts speaking again; 2) vibrate when a specific word is spoken (such as the user’s name); 3) show sound labels to identify sounds; and 4) teach Live Transcribe to properly caption difficult-to-understand words, such as names and slang.    

There are several free android apps that offer the ability to transcribe conversations.  An example is an app called TextHear.  However, these do not have the same level of features as Live Transcribe.  Also, because Live Transcribe is developed by Google, it can be integrated with other features of the device in a way that third-party apps cannot.  For example, Live Transcribe can be set up so that it can be activated with a tap of an accessibility icon that appears on the bottom right corner of the display.

iOS apps called, App MyEar and Live Caption (not to be mistaken with Google’s Live Caption service, discussed above), transcribe conversations on iPads and iPhones.  Live Caption costs $2.99/month for unlimited use and App MyEar has a one-time cost of $9.99.  Live Caption uses Google’s speech-to-text engine to transcribe conversations.  These apps do not have certain advanced features, such as the ability to teach the app to properly caption difficult-to-understand words or the option to include sound labels to identify sounds.

2. Google Recorder

The Google Recorder, which is pre-installed on the Google Pixel, records and transcribes audio.  The app is not designed for individuals with hearing loss.  As a result, it does not have a number of the features that Live Transcribe includes, such as a keyboard for a user to type responses, adjustable font size or vibrate when a specific word is spoken. 

The Recorder, however, has three advantages over Live Transcribe: first, it captions audio on the device itself.  No internet connection is required.  As a result, unlike Live Transcribe, it can be used to transcribe audio in locations where there is no internet connection, or the connection is week.  Second, transcripts can be saved indefinitely and can be shared easily with others.  Third, it includes a word search function to search the transcript for specific words.

Most importantly, the app transcribes audio with impressive accuracy.  Watch it in action.      

There is no alternative free recorder app that has a similar ability to transcribe audio.  Apple’s Voice Memos app does not have this capability. 

C. Make it Louder – Amplification of Audio with Sound Amplifier

 

Google developed the Sound Amplifier app to help people hear better.  Initially, the app was limited to improving the ability of individuals to hear sounds in the user’s environment.  Use cases include conversations in a crowded bar or watching television.  More recently, the app was updated to help people better hear media played on the Google Pixel.   

The app is available on Android devices running Android 6.0 Marshmallow and above.  You must use headphones in order for the app to work.  Initially the headphones had to be wired, but in a more recent update, Bluetooth headphones were accommodated, as well.  The user may use any brand or model of headphones they wish. 

When activated, Sound Amplifier has a prominent audio visualization feature that illustrates when sound is detected and shows the user how much the sound is boosted.  Basically, it helps the user visualize the changes made to the audio. 

The amount of amplification is controlled with a simple slider.  Amplification can be controlled for each ear individually.  The user can also control the pitch of the audio.  Thus, if the user has difficulty hearing higher range frequencies, the pitch can be adjusted accordingly.  As with amplification, pitch is controlled with a slider, and can be adjusted for each ear separately. 

To reduce background noise and white noise, there is a “noise reduction” feature that can be set at either “High,” “Medium” or “Low.”  The feature is effective, although when it is activated and set to High, it does reduce the degree of audio amplification.

The basic idea is that Sound Amplifier behaves much like a hearing aid but without the same level of sophistication.  If a user is hard of hearing and does not have hearing aids, Sound Amplifier is a convenient alternative. 

With respect to amplifying on-device audio from media, this feature is currently available exclusively on Google’s own Pixel devices.  All of the controls available for surrounding sound, are also available for on-device audio.  Accordingly, if someone is hard of hearing and wishes to listen to music on their Google Pixel, they can use Sound Amplifier to increase the volume beyond what is normally available, as well as change the pitch.

Unfortunately, Sound Amplifier does not yet have the ability to improve hearing during phone calls. 

D. Hearing Aid Compatibility and Hearing Aid Support

Hearing aid support refers to two concepts.  The first is hearing aid compatibility, and the second is direct streaming of audio from the mobile device to the hearing aid.  

The first concept is straight forward and not particularly interesting.  Pixel smartphones, like most smartphones, are hearing aid compatible.  The Google Pixel 4a has a hearing aid compatibility rating of M3,T4, the same rating as the iPhone.

The second concept of hearing aid support is direct streaming of audio from the smartphone to the hearing aids.  Mobile devices and hearing aids that do not have this capability must use an intermediary device, called a streamer.  The smartphone streams content to the streamer via Bluetooth.  The streamer converts the Bluetooth signal into a FM signal.  The streamer then sends the FM signal to the hearing aid.  For many people, using a streamer is undesirable.  The streamer is one more piece of equipment that they must remember to bring with them and to charge. 

The challenge with streaming directly to hearing aids is that the hearing aids must be Bluetooth capable in order to connect directly to the smartphone; using classic Bluetooth for this purpose consumes significant battery power, and Consumers do not want to have to charge their hearing aids every day.  Apple, however, developed a Bluetooth protocol that requires less power, and in 2013 hearing aid manufacturers started to implement this technology and introduced hearing aids known as Made for iPhone.  As a result, you could now stream phone calls and music from an Apple device directly to hearing aids without a streamer or remote.  Most hearing aid manufacturers have released hearing aids that implement Apple’s Bluetooth technology.

In the meantime, individuals with hearing aids who used android devices had to continue to use streamers.  This changed, however, in 2019 when Google introduced its own Bluetooth protocol called Audio Streaming for Hearing Aids (ASHA) that uses less battery power.   Now users of android devices can use their hearing aids like a headset to enjoy music, take calls and much more, while using very little battery power.

In order for hearing aid users who own android devices to benefit from ASHA, both the hearing aid manufacturer and the smartphone manufacturer must have implemented Google’s new protocol.  Smartphone manufacturers can implement the new protocol on devices that operate on Android 10 and that have Bluetooth 5.0 or higher.  Thus far, android smartphones that offer ASHA include the following:

  • Google Pixel 4a
  • Google Pixel 4 XL
  • Google Pixel 4
  • Google Pixel 3 XL
  • Google Pixel 3
  • Google Pixel 3a XL
  • Google Pixel 3a
  • Samsung Galaxy S20+
  • Samsung Galaxy S20
  • Samsung Galaxy S20 Ultra 5G
  • Samsung Galaxy S10+
  • Samsung Galaxy S10
  • Samsung Galaxy S10e
  • Samsung Galaxy Z Flip
  • Samsung Note 10+
  • Samsung Note 10

Accordingly, if you have a newer Pixel device or a newer Samsung device, you have access to ASHA.

Hearing aids that have implemented ASHA include:

  • Resound LiNX Quattro
  • ReSound ENZO Q
  • Beltone Imagine
  • Beltone Amaze
  • Beltone Boost Ultra
  • Audigy AGXR Q

Over time, we can expect additional hearing aids to support direct streaming from android devices.

Because Apple’s Bluetooth protocol has been available for a much longer period of time, there are many more Made for iPhone hearing aids than ASHA capable hearing aids.  However, this difference should level off over time.