The potential for Voice Interfaces in assisting the blind
From Home Automation to V-Commerce, Voice Interfaces have so much potential to assist the blind in their day to day lives.
This weekend I was fortunate enough to speak at the RP Fighting Blindness’ Annual Conference attended by vision impaired people, their family and friends and medical professionals who work in the field.
I was asked to speak to discuss the Potential for Voice Assistants to aid those affected by inherited retinal dystrophies, which leads to the gradual and progressive reduction in vision. Those affected typically start to show signs of sight loss as teenagers or in their twenties.
Whether it was because I was looking out for it, or just a coincidence in timing, but I started to see an increase in the number of articles explaining how Voice Assistants, were changing the lives of people with sight loss and were a revolution for the blind. This gave me the boost I needed when planning for the presentation, I was clearly on the right lines!
Now, Voice Assistants are not just devices that you have on your mantelpiece but almost all (Alexa, Google Assistant, Cortana, Siri) are available on almost all devices you may have in your pocket (Apple is a key stumbling block here with no Siri on other platforms, and no Alexa on Apple devices…yet!) Voice Assistants have the ability to replace screens, complement them or give you an indication to look at them catering for the entire spectrum of individuals with vision impairment. They are also not just confined to voice though, there are lots of devices now which have multiple inputs (voice, touchscreens and cameras to name a few) allowing for a wider range of interaction. Now is the prime time for those not just with visual impairment but individuals with other disabilities and accessibility issues to make real use of Voice Assistants, both inside and outside the home.
Home Automation
When thinking about the transition from being able to see to having any form of vision impairment, I imagine home automation is one of the first things that pops into anyone’s head if you think about ways to make life easier.
Lights are an obvious one, with a wealth of Wi-Fi connected bulbs which are compatible with Voice Assistants now available, and within the average person’s disposable income bracket. They are simple to install and easy to setup. The main problem with these bulbs is the ‘power issue’; you need to leave the power on to get them to work by voice. Therefore, no longer using the light switch to turn them off, you should always use your voice (or compatible switch that can be bought separately). This is a major obstacle in our house, overcoming 30 plus years of switching lights off at the source is difficult for the muscle memory to forget. However, when they are working they are a great bit of kit that, for someone with sight loss, could be really useful.
The second most popular use for home automation is controlling your heating. Being able to tell your thermostat what temperature to set the house or room to, or asking what the temperature is currently set to, is something that is really helpful for someone with visual impairment. I have a modest degree of knowledge for electronics and electricity systems, I, or rather my wife, decided on this occasion it would be better to have an electrician fit our Wi-Fi connected thermostat; so although a little harder to install, the usability is really simple and the majority of common thermostats work with most Voice Assistants out there at the moment.
Smart Plugs, one of the best gadgets any user can get for a smart home, are particularly handy for someone with degenerative vision impairment, who may wake up one morning with significantly less sight than they went to bed with. Smart plugs can turn almost any device into a smart device (providing it is left switched on), so you can turn anything from your kettle to your Christmas lights on, or even your hair straighteners off from anywhere you have access to your Voice Assistant. This is particularly useful if you can no longer see the plug switch, or your home does not provide good accessibility to the switches. They combine elements of safety and practicality that will benefit all manner of individuals.
Whilst on the topic of safety, smart doorbells and smart locks are devices that are fast becoming popular among early tech adopters. Smart doorbells allow you to see who is at the door and potentially talk to them through an intercom system prior to answering. This is particularly useful for the potential it holds in the future. With the speed in which AI and Machine Learning are advancing, it is only a matter of time before smart doorbells with facial recognition or ‘category’ recognition exist for mainstream consumers (by category, I mean a camera which can identify if someone looks like a delivery person, salesman or policeman based on their clothes). While we wait for these advances, the intercom ability will no doubt ease the minds of not only those with sight loss but also the elderly and other vulnerable individuals. Smart locks allow you to unlock or lock the door with the sound of your voice (or on a phone application). You could even, theoretically, name your front door ‘Pod Bay Doors’ (if you’re a fan of 2001, A Space Odyssey) and ask your Voice Assistant to “Open the pod bay doors” and your front door would be unlocked (hopefully!).
It is not just individual devices that cater for smart home users, there are whole “ecosystems” which can be bought, Samsung Smartthings offers a range of practical and security options, the Harmony Elite allows you to control any device with infrared, Bluetooth, z-wave or on your Wi-Fi network (up to 15 devices). Both of these then integrate with Alexa and Google Home, therefore, allowing for many more devices to be voice controlled.
There is a raft of other home automation gadgets that can be controlled via voice; from blinds, to kettles, lawn mowing robots, fridges that tell you what you are running out of, to microwaves that you can turn on with your voice. The important thing to remember with all of these gadgets is they are not there to replace the need for physical interaction but to augment it and assist with difficulties that arise with the onset of conditions which take away some of the normalcy of life.
The final point to make on home automation is integration with If This Then That (IFTTT). As free online service which allows you to link “actions” to “outcomes” i.e. When your phone enters the area around your house, it automatically turns on your lights, or you leave your office and your heating comes on so your house is warm by the time you arrive home. I cannot stress enough how useful these integrations can be, and how they can make life so much simpler, and lots are compatible with Voice Assistants. IFTTT was an early inspiration for Routines on Alexa, the ability to issue one voice command to trigger a series of actions. For example, in my house I can say “Alexa, it’s movie time” and the lights in the lounge will dim to 20%, the lights in the kitchen will turn off, and the Blu-ray player will power up via a smart plug. Having the ability to give one command that result in several actions can really personalise the Voice Assistant.
Voice Applications
Whilst home automation is the obvious “go to” when it comes to making life easier for those with vision impairment, the applications within the Voice Assistants are also really useful. From the one shots that are generally the most used applications on the leading Voice Assistants such as setting a timer, making lists, asking the weather and finding simple facts, to the more complex applications, there are many use cases for those with sight loss, especially those just developing it. The one shot applications allow for the user to retain some form of normality when developing sight loss, and the more complex multi turn conversations allow for the user to then build up a new sense of confidence. The introduction of Google Duplex, where Google’s Voice Assistant can make reservations for you, for example at the hairdressers or a restaurant requires long running multi turn conversations that can deal with the unexpected. This is the first step in having a true Personal Assistant.
There are a range of long running Voice Applications that are useful for those who are suffering from a gradual loss of sight. RP Fighting Blindness hits those in their youth, at an age where their mates are playing FIFA or the latest Call of Duty game they are unable. Gaming, however, seems to be one of the most popular in routes to Voice Assistants, and there are hundreds of different voice games released each month. As these become more complex, they will integrate more into mainstream gaming and therefore retain some normalcy for those affected by degenerative sight loss.
At the time of writing there were over 450 Mindfulness & Wellbeing Voice Applications (skills) in the Amazon Alexa skill store. It is an area which lends itself to voice. It is something I go into more detail in in my ‘Potential for Voice Interfaces in Healthcare’ article. Those with degenerative sight loss, or any degenerative condition could be prone to loneliness, anxiety and depression. There are an increasing number of Voice Applications developers, designers, and psychologists who believe that Cognitive Behavioural Therapy (CBT) and Behavioural Activation, both forms of ‘talking therapy’, are primed for a Voice Application revolution. This has the potential to help those who are suffering from early onset visual impairment, especially (but not limited to) young males, who are less likely to talk about their emotions with real people to access help when they are feeling particularly low at a very difficult time in their lives. The strain on the mental health services in not only the UK, but across the globe, will have a huge impact on the speed at which this technology progresses. Linking to this, it is only a matter of time before companion applications become more mainstream, allowing a Voice Assistant to become slightly more like JARVIS from the Marvel Cinematic Universe, more of a personal friend than just a small electronic device in the corner.
Voice Applications also have great ability for information discovery, whether that be through audio websites, or through structured applications which encourage follow up questions on a particular topic, the potential for sight, or the lack thereof, being a barrier to entry for individuals who would like to learn about a topic, or find out some facts is no longer there. The main barrier to entry when it comes to these information discovery applications is remembering the name of each application.
That is not to mention the Voice Applications that are being created specifically aimed at the blind. Applications like Talking Newspapers which allow blind users to continue to ‘read’ (listen) to articles from their favourite newspapers and magazines.
V-Commerce
Voice Commerce (or V-Commerce) is on the horizon (in fact people have been stating it is on the horizon since 2016). Amazon’s Alexa allows you to buy things from Amazon’s store by default, but some Voice Applications can now have in-skill purchasing. There are now an increasing number of grocery Voice Applications that allow you to add things to shopping lists or add them to your online shopping basket ready for your next order. Voice shopping is expected to account for US$40 billion annually by 2022. In the USA, Wall Street is also using Voice Applications to transfer billions of dollars. The potential that V-Commerce has for those with visual impairment cannot be understated enough. Giving back an element of freedom, the barrier of a screen is no longer there, it will allow individuals to still order their favourite foods but without the need to enter a busy supermarket on a Saturday afternoon; and it is not just grocery shopping that is affected by this though. Takeaways can now be ordered through your Voice Assistant through independent Voice Applications like Dominos Pizza. It is also possible to order an Uber through your Voice Assistant, reducing the need for reliance on a phone which can easily be misplaced.
Amazon’s Echo Look (currently only available in the United States) is another example of how V-commerce and the improvement in visual recognition technology could aid the blind. The Echo Look (coined in my social circle as the ‘Echo — Does My Bum Look Big In This?’ device is a camera with Alexa built in. This device is designed to go in the wardrobe or bedroom and look at your outfit telling you if it matches or if it is ‘on trend’. It can also order you a new pair of shoes to go with that dress, or a new tie to go with that shirt through either Amazon’s store or other bespoke Voice Applications (similar to how Dominos or Ocado’s applications work).
Personalisation
I have already mentioned a degree of personalisation within IFTTT, however a shift is coming in how Voice Applications are written (it has arrived on the shores of the United States of America already). Recently Amazon announced Blueprints, which allow users create basic Voice Applications without any need to code. Some of the use cases include Babysitter or Pet sitter applications to relay information and important steps or a Q&A skills that allows users to do simple question and answers. This has the potential to really help those that are just learning to come to terms with sight loss. It could allow another family member to set something up to remind the young person what to do if they wake up with less sight than they went to bed with. In moments of panic and no doubt a great deal of stress, these personalised skills could be tailored to individuals to really help them in a particularly difficult time.
Not all blue skies and rainbows…
Although I have painted a very rosy picture up to this point, all is not perfect yet; there are still some fundamental flaws within proprietary voice technology which are also worth considering as they will affect those with sight loss in equal capacity. Right off the bat, it needs to be made clear that these Voice Assistants are not HAL, JARVIS or the Computer from Star Trek, we are getting closer, and they were developed in part to attempt to recreate them, but they are not as advanced as them yet.
Notifications are not yet something that voice developers are able to utlise to their full potential. With access to notifications blind users can not only get notified when something happens i.e. in the news like newsflash notifications on the phone, but also through various IFTTT notifications (for example a sensor is triggered on your window, and your Voice Assistant of choice can notify you of this so you can investigate if someone is trying to gain access to your property).
As I have mentioned previously in this article, one of the key problems with the sheer number of Voice Applications is you need to remember the ‘invocation’ names of them. So you can’t just say “OK Google, order me a pizza” you would need to say “OK Google, Ask Dominos to order me a pizza”. With pizza this might be quite an easy example, when it comes to things like insurance or gaming it might not be as easy.
Finally, whereas I have sung the praise of smart home devices at the beginning of this article, I (deliberately) failed to mention one major drawback. Most of these devices require individual hubs or bridges. Little boxes that attach to either your Wi-Fi or your wired network to allow the devices. This can cause a mess of wires and the need for multiple Ethernet bridges if you are wiring everything.
There is little joined up thinking between the big brands when thinking about this issue. But in time, it will come.
Voice Interfaces are the future; they are going to change the way that we interact with computers. However, for those suffering from sight loss, it is more than that, it allows for an easier, less scary transition into a world without sight, and a degree of normalcy that many would not have thought possible.
Veni Loqui are voice design specialists formed to create bespoke solutions for Alexa and Google Home. We are working to leverage Voice in the Health and Social Care sectors. For more information contact info@veni-loqui.com
