Hearables, which are computers that fit inside your ear, are going to be a big deal. Their capabilities will go way beyond what traditional wireless headphones or hearing aids can accomplish. In this post, I'll explain why we will all be augmenting our hearing within just a few years.
Let's first define the term. A hearable is a computational device that fits in your ear and is wirelessly connected to other devices, and thus ultimately to the Internet. Hearables may perform a wide range of tasks, from enhancing your hearing ability, to measuring your biometrics (such as your heart rate), to providing you with information and services. Most will be 2-way communicators, including both a speaker and microphone.
Your phone will still be with you most of the time, but will be spending a lot more of it's time tucked away in your pocket or purse.
Why focus on the ear?
Wearables are widely hailed as the next big wave of technology. There are many places on your body that it might make sense to sport wearable technology. These include your wrist (e.g. Apple Watch), your feet (e.g. Nike+), your eyes (e.g. Microsoft HoloLens), beneath your skin, and of course, in your ears.
For a few years now I've been thinking that in-ear technology that is built around an audio/voice interface will have a valuable position in our constellation of wearables. Voice interfaces haven't really taken off yet for two main reasons: 1) the accuracy just wasn't good enough (now mostly fixed) and 2) nobody wants to be in an office where everyone is talking to their computer at the same time, and where everyone can hear everyone else's device (and perhaps sensitive or private information) talking out loud.
There are two key advantages of an ear-based interface that make it appealing. Firstly, it's personal...nobody else can hear what you're hearing. Secondly, in-ear technology is ready for prime time right now, whereas augmented and mixed reality technology, which focus on a visual interface, are still several years away. Even when AR/MR hit the mainstream, audio interfaces will still be less distracting than visual ones for a lot of use cases. And you'll need audio to go along with your AR/MR experiences anyway.
Audio interfaces are discrete and natural. If implemented well they can be just like having a personal assistant who whispers just the right information in your ear at just the right time.
The major tech companies have realized that to make their platforms even more indispensable to modern life, they need to get even more personal. They must transfer their value from the phone and bring it even closer. Services like Google Assistant, Cortana, Alexa, Siri and others are most valuable when they are right there and available totally hands-free. Apple's delayed AirPods are likely the beginning of a whole new platform upon which they will create new value and through which they will deliver new services. And as I outline below, an avalanche of interesting startups are rushing into this space too.
The technological under-pinning of wireless audio devices, Bluetooth, is also getting a major revamp this year. Bluetooth 5.0 will quadruple range, double bandwidth, and dramatically reduce the power consumption for stereo audio connections, making it possible to build devices that can operate for days on a single charge.
The hearables are coming.
What new things will we be able to do with hearables?
OK, so why would you want to stick a small computer in your ear? Well, there are lots of good reasons why. Hearables won't just improve your hearing and make it even more convenient to listen to music. They will enable you to communicate in new ways, and maybe even translate language for you in real time.
Let's review the major applications of hearables. I've broken them down into major categories (that get more and more interesting as you go down the list):
- Traditional sound-related applications
- Augmented hearing
- Biometric capabilities
- Information/communications services.
Let's review these one by one. And yes, I'm saving the best for last. If you can't wait, feel free to scroll down until you see the Babel Fish picture :)
Traditional sound-related applications
This group of applications are essentially what you're able to do today with existing products spanning the headphone, hearing aid, and ear plug markets, only better.
Just like headphones, hearables will enable you to listening to music and make phone calls via a bluetooth connection to your phone. They will also help mitigate hearing loss, boosting sound levels, or intelligently boosting the signal/noise ratio for sounds you're interested to hear, like voices. They will likely not market themselves as hearing aids though, partly to avoid stigma, but mostly because you need to get FDA approval to do that. Hearables can also capture sound from one side of the head and redirect it to the other ear for people like my pal, Mark, who was born deaf in one ear. Some hearables will also offer hearing protection, digitally compressing sound that enters the ear canal to limit the volume of sound and minimize damage when in noisy environments like concerts, industrial, or construction sites. This is the focus of the new ProSounds H2P product.
Augmented hearing (better than normal hearing)
Hearables will go way beyond what traditional headphones and hearing aids can do today. New devices like the Hear One will allow you to decide how the world sounds to you, applying your own personal EQ and noise reduction. Want to hear a little more or less bass while at a concert? No problem. Where hearables start to get interesting is when you can apply increasing levels of intelligence to how you process and thus perceive sound.
Smart noise cancellation allows you to remove the sounds you don't want to hear (babies crying at the mall, the rumble of an air conditioner at your office, plane noise, the screech of a subway train) but keep the sounds you don't want to miss. For safety, cyclists need to be able to hear traffic noise while listening to music. And parents will still want to hear the sound of a child crying upstairs while watching TV.
Enhanced or "bionic" hearing is one of the promises of some hearables. These devices will intelligently boost quieter sounds and enable you to have better than perfect normal human hearing. This could be useful for first responders listening for survivors, hunters tracking animals, and law enforcement and military chasing bad guys. Enhanced hearing will also come in handy for people at the back of a room listening to under-amplified public speakers. While we're at it, why not build hearables with the ability to replay the last 10 seconds of what you just heard in case you missed something?
Real-time language translation
OK, now things start to get much sexier. How about a hearable that translates other languages in real-time? Waverly Labs is currently building a product called the Pilot which will attempt to translate languages in real-time, much like the Babel Fish in Douglas Adams' great novel, The Hitchhiker's guide to the galaxy. Waverly imagine a world without language barriers and they are targeting the Pilot at a $299 price point; $199 if you sign up to their Indigogo campaign. Let's see if they deliver something decent in May 2017, their projected launch timeframe.
Future hearables may also enable us to augment and modify the sounds that we hear, and change them to our liking. Applications may emerge that let you alter voices, perhaps letting you soften a strong regional accent that you find hard to understand, or hear everyone as if they just sucked the helium out of a balloon. You might download voice packs from your favorite movies and assign new voices to your friends and family. Make your boss sound like Darth Vader and your partner sound like C3-P0 or Princess Leia. Start thinking now about who in your life you'd give Yoda's voice to. And Jabba.
Some hearables will come with a range of biometric capabilities and will be able to use sensors in your ear to measure your heart rate, blood pressure, temperature, ECG, and even your blood oxygen level. Accelerometers will also enable these devices to measure your movement and activity. So hearables will be able to do activity monitoring, as well as health and wellness monitoring. You will be able to build a medical record of your life signs that are stored on your phone and that you can share with clinicians should you choose. Biometric hearables could also automatically sound the alarm and summon help if they detect a heart attack, or some other major medical issue.
Biometrics can also be very valuable in authentication and security. When combined together, a wide range of biometrics (heart rhythm, ear shape, gait) can form a signature that's unique to you. A hearable could authenticate you to other devices the same way an Apple Watch can sign you into your MacBook now.
NEC claims they can measure the reflection of sound waves in the ear canal to look for the unique shape of an individual ear with greater than 99% accuracy. When combined with other biometric signatures to further boost accuracy this could provide a new root of trust for security applications that require continuous authentication. Goodbye and good riddance, passwords.
Information and communication services
Perhaps the most interesting corner of the hearables market is the ability to use in-ear computing as a new platform for delivering information and communications services. This is the area of most interest to the major compute platform vendors like Google, Facebook, Amazon, Microsoft and Apple.
At the most basic level, information hearables can be used to do much of what your phone does today, just delivered in audio form. They will be able to give you notifications and reminders. Future versions will deliver context sensitive, subtle prompts and reminders such as this: "This is Jeff McHugh, you met him and his wife Shirley at a party last May".
Nobody wants intrusive notifications blurted out at inopportune moments when they are trying to concentrate, or when they are in the middle of a conversation. Hearable platforms will need to understand a user's context and use intelligence to choose the right time and place to deliver different categories of information. All information is not equal. Successful hearable platforms will need to assess the relative value of information and decide what to present to the user, and when.
As I've discussed in a previous post, digital personal assistants are going to become a primary interface for many of us in the coming years. They will allow us to control our environments ("turn on the lights"), order services ("I need an Uber"), get information ("What's the score in the Cubs game?"), get directions, and much more. This is where platforms like Microsoft Cortana, Google Assistant, Amazon Alexa, Samsung Viv, Facebook messenger, Apple Siri, and others want to go. And some are doing a better job than others. The natural place for these services to reside is in your ear, not on your phone.
Personal audio networks
Ever found yourself shouting into the ear of the person next to you at a loud concert? Or lost someone at the supermarket and then had to wander around looking for them? Personal audio networks are a great future application of hearables. You link your hearables to the hearables of others on a temporary basis to form a private audio network that only those of you on the network can access. With a personal audio network you just speak, and the other people on that network hear you. When combined with smart noise-cancellation and digital compression you'll be able to talk to your buddies at your next death metal concert without even raising your voice and they will hear you perfectly. Even if you're under assault by 150 decibels or stood 150 feet away from them at the bar getting drinks. And teamwork at the supermarket will get much easier: "Hey honey, while you're off grabbing that yoghurt please grab a 2% milk too, please. See you in the cereal aisle".
Theatro is already making a hearable product aimed at the retail market. It replaces those "walkies" you sometimes see store associates wearing which are essentially just radios that broadcast to every other employee in the store. The Theatro hearable intelligently routes communications to the right person or people you need to reach, and has all kinds of cool features including being able to ask to find subject matter experts, connect you directly to your buddy if you're a new employee, and have managers share messages to all staff. If you're in the retail world, it's worth looking at.
New services will sit on top of hearables
Look for new, perhaps unexpected services to be layered over the top of hearable platforms once they become widespread. For example, BitBite uses hearables to listen to the sounds you make while eating and analyze your eating and nutrition habits. The software then provides you audio feedback that is designed to help you improve your diet and alter your eating patterns accordingly.
Expect to see the emergence of digital coaches and other services that are based on artificial intelligence. For example, you might load a "Digital conscience" app that whispers guidance in your ear as if Jiminy Cricket were perched on your shoulder.
The companies to watch in the hearables sector
Many companies are racing towards to the hearables sector from different starting points and are now on a collision course at the center of the market. Major players eyeing this space include:
- Established hearing aid companies are busy adding features (bluetooth connectivity and microphones for music, phone calls). Key companies: Beltone, Phonak, Oticon, Resound, Sivantos, Starkey, and Widex.
- Established headphone companies are adding connectivity and hearing augmentation features, and some are adding biometric capabilities. Key companies: Bose, Jabra, Koss, LG, Motorola, Samsung, Skullcandy, SMS, and Sony.
- Established computer companies are either building new hearable devices of their own (e.g. Apple), or are looking to bring their services to the ear via partnership with other hardware vendors. Key companies: Amazon, Apple, Facebook, Google, Microsoft, and Samsung.
- Hearable startup companies - A number of interesting new startups are beavering away on new hearable products that do things from smart noise cancellation to real-time language translation. And once they have the hardware deployed, I'd expect them all to bring a steady stream of new innovation through software apps. The winners will build app stores for their devices. Key companies (and their first products) to watch: Doppler Labs' Here One product, Nuheara IQ buds, Alpha Audiotronics Skybuds, Nura, Bragi's Dash, ProSounds H2P, and Human Inc's Sound headphones, that look like designer Spock ears.
The winners in the hearables market will ultimately be the tech giants (Google et al) who will capture the lion's share of the value created here. The hardware manufacturers (Jabra, Skullcandy, Resound etc) are still in with a good chance of success though. Good quality hardware will command a premium and enable healthy margins. But branding and marketing will play a very large part, as for any fashion item. Just look at what Beats managed to achieve with decidedly average quality headphones and a big marketing budget. Hearables are fashion items, and always will be. Expect hearables to be very popular with the MTV generation. We listened to music too loud when we were younger and some of us now need help with our hearing but aren't ready to stop being cool. Hearing aids, NO! Hearables, GREAT! So long as the features, fashion, services and value are there, these things will sell like hot cakes.
New technology always affords us amazing new possibilities, but also comes with risks. We will always need to understand and minimize the risks so that we can enjoy all the benefits.
Hearables may make life easier for us, and make it easier to access services simply using our voices. They may help us to improve our health, hear better, listen better, understand people that speak other languages, be on time more often, remember people's names, and generally do a better job of looking after our hearing.
But we will need to think carefully about how we implement these devices and services. Privacy must be a primary concern. Hearables will need to be designed with privacy at their core. After all, we are talking about an always-on microphone here. What gets recorded? Where does information get stored? Who owns it? Who controls access to it?
And how will it feel if we essentially start introducing new voices into our heads? Will we start to feel schizophrenic when we have multiple software agents offering us advice throughout our days? After prolonged use, would we feel odd when we take our the hearables and the voices stop? Much ethnographic work needs to be done here to understand how personal assistant voice agents whispering in our ears will work best with the way people live their lives. There are important limits to be explored here.
What do you think?
What's your own interest level in some kind of hearable technology? What features would you want to see from the ones I've described here? And what features would you want that I didn't cover?