Published: October 29, 2020
The Google Pixel Smartphone for People with Hearing Loss – an Analysis
Apple made significant effort to make the iPhone accessible to individuals who are blind. The result was that Apple captured hundreds of thousands (if not millions) of incredibly loyal customers with vision loss. By contrast, Google is currently investing significant resources to offer an unparalleled level of accessibility to individuals who are deaf or hard of hearing. It remains to be seen whether Google will be rewarded with a similar level of loyalty, but Google’s efforts have yielded impressive results.
This analysis is focused on the Pixel smartphone because, as we will see, Google has prioritized its own devices for the latest accessibility features, perhaps because some of these features require an integration of both hardware and software elements.
Google has adopted a multi-faceted approach to making its Pixel smartphones accessible to individuals with hearing loss. First, it has provided the devices with the ability to caption everything and anything. If it has audio, it can be captioned. Second, Google has focused on amplifying audio for people who are hard of hearing. Third, the Pixel offers hearing aid support.
Each of these categories will be examined in turn.
A. If it has Audio, it can be Captioned
1. Live Caption for Media and Voice and Video Calls
A feature called “Live Caption” provides users the ability to caption media, as well as voice and video calls. Media that can be captioned includes videos, podcasts and internet radio. Live caption provides a transcription of your internal audio stream, so that the user can get closed captions in virtually any app. Mobile devices that do not have this capability, such as the iPhone, rely on the app or website itself to generate the captions. If the app or website does not offer captions, the user is out of luck. A simple example: podcasts are becoming a ubiquitous experience. More than half of Americans over the age of twelve (12) have listened to a podcast. Thirty-two percent (32%) listen to at least one podcast a month. However, podcast apps rarely offer captions, denying individuals with hearing loss who have an iPhone a valuable source of entertainment and learning. On the other hand, the Pixel’s Live Caption feature will caption any podcast, regardless of the app used to play the podcast.
Similarly, traditional radio is a domain that has never been fully accessible to individuals with hearing loss. Internet radio is no exception. Internet radio apps, such as TuneIn radio do not offer closed captions. However, the Google Pixel can caption any internet radio played on the device since the Live Caption feature is a system-wide solution that generates captions from the internal audio stream. As a result of Live Caption, people with hearing loss can now access entire categories of content that was previously inaccessible.
Currently, Live Caption is available in English only, with additional languages available in the future. Captions appear as a bubble that can be moved around the display. There is an option to hide profanity and to show sound labels, such as laughter, music and applause. Live Caption can be turned on by pressing the volume rocker and tapping on the caption symbol. An alternative, more cumbersome way to turn on Live caption, is to do so through the Accessibility settings.
The captions are generated on the device, not in the cloud. This guarantees privacy and reduces latency. On the other hand, captions generated on the device are not quite as accurate as those generated in the cloud, since Google can apply a more sophisticated and algorithm in the cloud.
As of August 2020, Live Caption supports phone calls, as well as voice and video calls placed with so-called “over-the-top” applications. Thus, Pixel owners can now caption calls made with the native Pixel phone app, as well as voice or video calls made with Google Duo, WhatsApp, Viber, Skype and others. This is the first time that individuals with significant hearing loss can use the same communications apps as everyone else.
Live Caption for media is available on all Google Pixel devices going back to the Pixel 2, as well as the Samsung Galaxy S20, S20+ and S20 Ultra. The OnePlus 7t and OnePlus 8 also have Live Caption. We can expect the Live Caption feature to roll-out to additional android devices in the future. However, Live Caption for voice and video calls is available only on the Pixel. It may come to other android devices in 2021.
B. Live Transcribe, the Google Recorder and Sound Notifications
1. Live Transcribe
Android offers an app called Live Transcribe to individuals who wish to caption in-person conversations in near-real time. Unlike, Live Caption, Live Transcribe is widely available to android devices through the Google Play Store.
The hallmark of the app is its simplicity. When the app is opened it immediately begins transcribing what it hears. The text size is adjustable and can be made quite large. The transcription is very accurate and adds punctuation. It even understands some context and capitalizes words accordingly.
Unlike Live Caption, the transcription for Live Transcribe occurs in the cloud on Google’s servers. This means that Google has access to the conversations, which raises privacy concerns. Google claims, however, that it does not save the transcriptions or audio and that it does not use this information to improve its algorithms.
Because the transcriptions occur in the cloud, Live Transcribe requires a good internet connection to work. This limits use of the app in locations where internet connectivity is weak or non-existent.
The app has a few additional features that increase its usability. A blue circle in the top right corner pulses. The more it pulses, the greater the ambient noise level and the closer the user needs to move the microphone to the speaker for it to work. At the bottom of the display, there are few buttons, including one to bring up a keyboard to type out replies. This is intended to be used by people who cannot speak. The size of the window that contains the reply is adjustable. The bottom of the display also has a button to pause the transcription so that the user can catch up to what is being spoken.
There is an option to save transcriptions for three days. If the user wishes to save a transcription permanently, they can copy and paste the transcription into another document, such as a Word document.
Four other useful options include the ability to 1) vibrate after speech resumes following a pause in order to get the user’s attention when the speaker starts speaking again; 2) vibrate when a specific word is spoken; 3) show sound labels to identify sounds; and 4) teach Live Transcribe to properly caption difficult-to-understand words, such as names and slang.
There are a several free android apps that offer the ability to transcribe conversations. An example is an app called TextHear. However, these do not have the same level of features as Live Transcribe. Also, because Live Transcribe is developed by Google, it can be integrated with other features of the device in a way that third-party apps cannot. For example, Live Transcribe can be set up so that it can be activated with a tap of an accessibility icon that appears on the bottom right corner of the device.
iOS apps called, App MyEar and Live Caption (not to be mistaken with Google’s Live Caption service, discussed above), transcribe conversations on iPads and iPhones. Live Caption costs $2.99/month for unlimited use and App MyEar has a one-time cost of $9.99. Live Caption uses Google’s speech-to-text engine to transcribe conversations. These apps do not have certain advanced features, such as the ability to teach the app to properly caption difficult-to-understand words or the option to include sound labels to identify sounds.
Features that RAZ Mobility would like to see added to Live Transcribe include the ability to transcribe even when there is no internet connection and the capability of differentiating between multiple speakers. It would also be helpful if there was an option to vocalize typed text. Fortunately, Google is updating the app and adding new features over time. The last update occurred in May 2020.
2. Google Recorder
The Google Recorder, which is pre-installed on the Google Pixel, records and transcribes audio. The app is not designed for individuals with hearing loss. As a result, it does not have a number of the features that Live Transcribe has, such as a keyboard for a user to type responses, adjustable font size or vibrate when a specific word is spoken.
The recorder, however, has three advantages over Live Transcribe: first, it captions audio on the device itself. No internet connection is required. As a result, unlike Live Transcribe, it can be used to transcribe audio in locations where there is no internet connection, or the connection is week. Second, transcripts can be saved indefinitely and can be shared easily with others. Third, transcripts can be searched for specific words.
Most importantly, the app transcribes audio with great accuracy. Watch it in action.
There is no alternative recorder app that has a similar ability to transcribe audio. Apple’s Voice Memos app does not have this capability.
3. Sound Notifications
Sounds Notifications was introduced in October 2020 and is designed to alert individuals with hearing loss of sounds in their environment. The feature detects ten (10) sounds. These sounds include the following:
- Smoke and fire alarms
- Landline phone ringing
- Siren
- Shouting
- Baby sounds
- Doorbell ringing
- Knocking
- Dog barking
- Appliance beeping
- Water running
When the device detects one of these sounds the user is alerted with a notification, a flash from the camera or a vibration. Alerts can also be sent to a smartwatch.
Sound Notifications works completely offline, which means that it will work even if your device does not have an internet connection. Sound Notifications also tries to be as helpful as possible by providing some context. With the Timeline view, the user can see a timeline illustrating when and for how long it detected a sound. Thus, if the user has a baby Sound Notifications will inform the user when and for how long the baby was crying. Watch how Sound Notifications works.
C. Make it Louder – Amplification of Audio with Sound Amplifier
Google developed the Sound Amplifier app to help people hear better. Initially, the app was limited to improving the ability of individuals to hear sounds in the user’s environment. Use cases include conversations in a crowded bar or watching television. More recently, the app was updated to help people better hear media played on the Google Pixel.
The app is available on Android devices running Android 6.0 Marshmallow and above. You must use headphones in order for the app to work. Initially the headphones had to be wired, but in a more recent update, Bluetooth headphones were accommodated, as well. The user may use any brand or model of headphones they wish.
When activated, Sound Amplifier has a prominent audio visualization feature that illustrates when sound is detected and shows the user how much the sound is boosted. Basically, it helps the user visualize the changes made to the audio.
The amount of amplification is controlled with a simple slider. Amplification can be controlled for each ear individually. The user can also control the pitch of the audio. Thus, if the user has difficulty hearing higher range frequencies, the pitch can be adjusted accordingly. As with amplification, pitch is controlled with a slider, and can be adjusted for each ear separately.
To reduce background noise and white noise, there is a “noise reduction” feature that can be set at either “High,” “Medium” or “Low.” The feature is effective, although when it is activated and set to High, it does reduce the degree of audio amplification.
The basic idea, if that Sound Amplifier behaves much like a hearing aid but clearly without the same level of sophistication. However, if a user is hard of hearing and does not have hearing aids, Sound Amplifier is a convenient alternative.
With respect to amplifying on-device media, this feature is currently available exclusively on Google’s own Pixel devices. All of the controls available for surrounding sound, are also available for on-device media. Accordingly, if someone is hard of hearing and wishes to listen to music on their Google Pixel, they can use Sound Amplifier to increase the volume beyond what is normally available, as well as change the pitch.
Unfortunately, Sound Amplifier does not yet have the ability to improve hearing during phone calls.
D. Hearing Aid Compatibility and Hearing Aid Support
Hearing aid support refers to two concepts. The first is hearing aid compatibility, and the second is direct streaming of audio from the mobile device to the hearing aid.
The first concept is straight forward and not particularly interesting. Pixel smartphones, like most smartphones, are hearing aid compatible. The Google Pixel 4a has a hearing aid compatibility rating of M3,T4, the same rating as the iPhone.
The second concept of hearing aid support is direct streaming of audio from the smartphone to the hearing aids. Mobile devices and hearing aids that do not have this capability must use an intermediary device, called a streamer. The smartphone streams content to the streamer via Bluetooth. The streamer converts the Bluetooth signal into a FM signal. The streamer then sends the FM signal to the hearing aid. For many people, using a streamer is undesirable. The streamer is one more piece of equipment that they must remember to bring with them and to charge.
The challenge with streaming directly to hearing aids is that the hearing aids must be Bluetooth capable in order to connect directly to the smartphone; using classic Bluetooth for this purpose consumes significant battery power, and Consumers do not want to have to charge their hearing aids every day. Apple, however, developed a Bluetooth protocol that requires less power, and in 2013 hearing aid manufacturers started to implement this technology and introduced hearing aids known as Made for iPhone. As a result, you could now stream phone calls and music from an Apple device directly to hearing aids without a streamer or remote. Most hearing aid manufacturers have released hearing aids that implement Apple’s Bluetooth technology.
In the meantime, individuals with hearing aids who used android devices had to continue to use streamers. This changed, however, in 2019 when Google introduced its own Bluetooth protocol called Audio Streaming for Hearing Aids (ASHA) that uses less battery power. Now users of android devices can use their hearing aids like a headset to enjoy music, take calls and much more, while using very little battery power.
In order for hearing aid users who own android devices to benefit from ASHA, both the hearing aid manufacturer and the smartphone manufacturer must have implemented Google’s new protocol. Smartphone manufacturers can implement the new protocol on devices that operate on Android 10 and that have Bluetooth 5.0 or higher. Thus far, android smartphones that offer ASHA include the following:
- Google Pixel 4a
- Google Pixel 4 XL
- Google Pixel 4
- Google Pixel 3 XL
- Google Pixel 3
- Google Pixel 3a XL
- Google Pixel 3a
- Samsung Galaxy S20+
- Samsung Galaxy S20
- Samsung Galaxy S20 Ultra 5G
- Samsung Galaxy S10+
- Samsung Galaxy S10
- Samsung Galaxy S10e
- Samsung Galaxy Z Flip
- Samsung Note 10+
- Samsung Note 10
Accordingly, if you have a newer Pixel device or a newer Samsung device, you have access to ASHA.
Hearing aids that have implemented ASHA include:
- Resound LiNX Quattro
- ReSound ENZO Q
- Beltone Imagine
- Beltone Amaze
- Beltone Boost Ultra
- Audigy AGXR Q
Over time, we can expect additional hearing aids to support direct streaming from android devices.
Because Apple’s Bluetooth protocol has been available for a much longer period of time, there are many more Made for iPhone hearing aids than ASHA capable hearing aids. However, this difference should level off over time.
Check out other helpful articles on cell phones: