XRAI Glass Subtitles Life in Real-Time

As I’ve said before, my favorite metaverse device (VR, AR, and XR included) to date is the Nreal Air; for various reasons you can read about here. I just learned of a new app that was developed for the platform called XRAI.

XRAI Glass is a transcription app that displays conversational captions in real-time. The software is designed specifically for those with hearing impairments, giving them the ability to follow along in conversations and read what people are saying. Currently, the app only exists for the Nreal Air augmented reality glasses.

For people who rely on lip-reading, picking up every word is impossible. Only around 40 percent of the sounds in the English language can be seen on the lips of a speaker, according to the US Centers for Disease Control and Prevention, and that’s in ideal conditions.

The prospect of having audible speech transcribed in your field of vision is exciting. It can help people with varying degrees of hearing loss, who may suffer from social isolation as a result, to pick up more of a conversation. The XRAI app also works when watching TV, which can be handy for live content, where subtitles aren’t always great (or at the cinema, where captions are absent). – Wired

When I put the glasses on, as soon as Ryan started talking to me, the text began appearing in the center of my vision. It was surprisingly accurate. Even with curse words. And there’s only a 1-2 second delay.

My reaction, of course, wasn’t as good as someone who actually needs this tech:

XRAI doesn’t just transcribe. It also saves a searchable transcript of each conversation, with speaker attribution. This feature alone expands the use cases and potential users. So instead of opening an app like Otter on my phone to record and transcribe meetings, I could just pop my Nreal’s on and open XRAI. Imagine being able to have a transcription of all those random and chance conversations you have throughout the day.

One drawback is that the software doesn’t work great in noisy environments with a lot of background noise. I don’t see this as a huge problem, though. AI models like OpenAI’s Whisper are vastly improving speech recognition by orders of magnitude. And these are the exact problems they’re working on.

The ultimate vision of Nreal Air is for them to be a “24/7” device that users have on all the time throughout the day – whether during leisure or work. As long as they continue to onboard developers creating functional apps like XRAI that provide a true additional utility, this vision actually seems possible.