For most people, ordering a coffee or having a conversation with someone on a bus seems like a trivial activity. However, for those who understand the obstacles deaf or hard of hearing people face, a casual conversation becomes something a bit more significant. Engineer Chet Gnegy and research scientist Dimitri Kanevsky have collaborated to launch the Android app, Live Transcribe.
Live Transcribe converts speech to real-time captions through the use of the phone microphone. Words will appear on phone as they are spoken, and the user can reply to the conversation by typing on the phone keyboard. The app aims to make conversations virtually seamless between deaf or hard of hearing people and hearing people.
Christian Vogler, professor and researcher at Gallaudet University, says, “We can now do things that weren’t even remotely possible a few years ago, like jump into conversations at the dinner table or casually join in when the opportunity arises.”
The creators of Live Transcribe are well aware of the potential the app holds. For instance, it can lend itself to languages other than English. Future versions of the app will hopefully include more than 70 languages and dialects with accurate captions.
Kanevsky has been deaf since early childhood, and he’s worked for the last 30 years on speech recognition and communications technology. To develop the Live Transcribe technology, the team posed themselves with a question: “What if we used cloud-based automatic speech recognition to display spoken words on a screen?”
Researchers then tested prototypes of the technology at Gallaudet University, a well-known school for the deaf and hard of hearing. “On the technical side, the app relies on cloud ASR to provide greater accuracy combined with an on-device neural network-based speech detector,” SiliconANGLE, the Google Play product, says.
Users are able to reply without having to speak by typing responses. Additionally, there are optional haptic feedback signals you can choose when someone speaks. Regarding privacy, app creators say conversation transcripts aren’t stored on servers, they remain secured on your personal device.
Aatif Sumar of Android Police, says, “For an app that’s supposed to be in beta, it works impressively well. I had conversations around the phone, and the app was able to pick up most of what was said. Even when the transcription missed a word or two, you could get the gist of the conversation.”
Lauren Goode from Wired makes a thought-provoking observation saying Live Transcribe is “less about helping a user navigate the phone itself and more about people using phones to navigate the world that exists around us when we’re not buried in our phones.”
