Subtitles for real life? There's a pair of glasses for that, with XRAI CEO Dan Scarfe

DS: 00:00

So I'm going to pass you the glasses now. If you can put those on and I'm just going to start up the captioning software and hopefully everything that I say should start magically appearing as captions in real time for you.

AW: 00:14

This is awesome. It's super super amazing and the reason why I say that is because all my life, actually since the age of four, I've actually had to look directly at the person while they're talking to me and the fact that I'm not really looking at you when you're talking to me is like, I get a break. I can do what everybody else does. I'm sorry, I'm just amazed at how accurate this transcription is because…That's right, from my experience, I'm just... Wow. How do you do that?

DS: 00:50

Magic, it’s nothing short of magic.

TS: 00:53

Welcome to Made For Us, the show where we explore how intentional design can help build a world that works better for everyone. I'm your host, Tosin Sulaiman. Imagine walking into a room full of people talking, laughing and sharing ideas, but you struggle to follow what they're saying. You're there, but not really part of the conversation. For millions of people around the world, like Ashley Walker, one of the voices you just heard, this isn't just hypothetical, it's their everyday reality.

TS: 01:27

But what if technology could change that? What if a cleverly designed pair of glasses could turn speech into real-time subtitles, breaking down barriers not just for those who are deaf or hard of hearing, but for anyone struggling with communication? In this episode of Made For Us, we're talking to Dan Scarfee, the man who gave Ashley the magical pair of glasses. He's the founder of XRAI, a company that's using AI and augmented reality to build a more inclusive future for everyone - for the 1.5 billion people worldwide with hearing loss, but also anyone facing a language barrier. Here's my conversation with Dan. I started by asking him to share some of the highs since launching XRAI three years ago.

DS: 02:11

I'm Dan Scarfe, I'm the CEO and founder at XRAI. XRAI is all about fusing together extended reality and artificial intelligence to try and help make the world more accessible. I spent 20 years in enterprise IT before doing XRAI, kind of helping large organizations move their systems to the cloud, which was all kind of fairly interesting, but not necessarily particularly noble. So once we sold that business, wanted to do something a little bit more.

fulfilling and kind of take all of that technology and those skills that I had amassed over the previous couple of decades and kind of put them to good use. So that was the genesis of the idea for XRAI in terms of the highs. mean, there have been so many. I mean, it's a roller coaster, of course, doing a startup. There are some incredible things. When you put these glasses on users for the first time, then you see their face light up and this ability now to be included within the conversation and it continues to be at high every time we see someone do that.

TS: 03:12

So let's talk about what inspired you and your co-founders to create XRAI and what's the problem that you were trying to solve.

DS: 03:19

So the genesis of XRAI originally was I was back in the UK actually December 2021, I think with my granddad who was 96 and like many 96 year olds, he'd lost almost all of his hearing. So kind of being around him, surrounded by his family, but just unable to engage in the conversation. And obviously with my background, I knew that this technology was available. I knew how much it improved, especially over COVID for instance, and thought, why can't we just take this speech to text technology, combine it with augmented reality and create life subtitles?

So that was kind of the idea behind it. And my granddad is just one of an inordinate number of people that live with hearing loss. They're one in five people around the world. So it's a huge, huge opportunity to help an inordinate number of people. So that's kind of where it began. Then of course, as people started to experience it, the opportunities around translation started to expose themselves as well. And hearing loss is one barrier to communication. course, language is another even bigger one.

TS: 04:24

So your first product was called XRAI glass. For those who may not be familiar, can you explain how it works?

DS: 04:32

So effectively, we've taken the speech-to-text technology that people might know from their phones, apps like Google Translate, for instance, and we've now made that available through augmented reality glasses. So when we started out originally, we were working with glasses designed for watching movies, playing video games, but we built an app for them effectively that allowed them to display real-time subtitles.

But some of the feedback that we got from users was that the tint on them isn't particularly great and the fact that these need a wire and a separate device. So continue to take feedback. The technology continued to advance and improve, of course, whilst we've been on this journey. And so recently we were able to announce these glasses. So these are now all in one. So these don't require a separate compute unit. And we just put these on and all of a sudden subtitles.

TS: 05:28

And what are they called?

DS: 05:30

So these are our own glasses. So these are called the XRAI AR1s. And so these are available now.

TS: 05:39

So if someone wanted to use XRAI, they'd have to download the app and then connect it with a pair of these glasses.

DS: 05:47

Correct. And they don't necessarily have to connect it with a set of glasses. So many, many of our users just use our app. So you can just go onto the Apple store or the Google Play store, download the app and immediately get subtitles. And because these are powered by the cloud, these are subtitles which are much, much more accurate than people might be used to. And so people get a chance to try it out, see how it operates in different environments and then if they like and they want the full experience, then you can get a pair of glasses then, so it's that same software.

TS: 06:23

And your company was mentioned in a tweet that got a bit of attention. So tell us about that story.

DS: 06:30

Yes, it was a rather controversial individual called Elon Musk. And so he thought they were cool. So that's about as much as you get from Elon, but just that one word was quite impressive, of course. And then even better, we then had Andreessen Horowitz. So Marc Andreessen himself then retweeted Elon's tweet, which was quite fun with a little bullseye symbol. So we hit the bullseye according to Marc Andreessen, which was quite the accolade.

TS: 07:00

And I think Elon Musk was responding to an earlier tweet by someone else who said, this is what AI is for. Just this use case alone is worth every dollar invested in it. I wonder how widespread this view is among people working in AI that accessibility is an important use case. Is that something that you often hear being talked about in AI circles?

DS: 07:21

I think it's fantastic to have found just one genuinely brilliant good use case of AI because everybody talks about how it's going to change the world and how it's going to solve everything. But unfortunately, we're still struggling slightly to find very specific things where it can have asymmetric returns and accessibility, I think, is one of those things. Now, you can go into a debate about, well, neural networks and transcription and translation actually AI. Is AI just large language models and generative AI? And again, I don't want to go down too much of a rabbit hole, but let's just say for all intents and purposes, it is AI and AI encompasses more than just ChatGPT. In which case, yes, I think this is an absolutely incredible use case of AI. And again, this technology is not new. We’ve had transcription and translation for decades. It’s just we've now suddenly hit this threshold where the accuracy is actually at a point where it goes from being irritating to genuinely useful. So that, I think, is what is quite interesting and exciting.

TS: 08:27

Yeah. And when you saw your granddad struggling to keep up with the conversation, know, that the moment that you got that epiphany and, know, it occurred to you that there was a solution and that you could do something about it. I imagine not everyone would have made that connection. What was it about your background or upbringing that made you want to solve the problem? Have you always leaned towards doing something impact driven?

DS: 08:52

I don't know whether I've always leaned towards it. I I spent 20 years not doing that, but I think when I had the opportunity to and I fortunate to be in that position, it is an incredible thing to be able to do. And it was, I guess, that entrepreneurial technologist just constantly questioning everything around you, constantly thinking about how disparate things can come together to create something new and novel and innovative, so that epiphany and that thought about combining these two things together presented that opportunity to create this life-changing product for hundreds of millions of people.

TS: 09:33

Now let's hear the reactions of two more people trying on XRAI glasses for the first time. First up is Tasha Ghouri speaking with XRAI's Chief Marketing Officer Mitchell Feldman. Then you'll hear from Jodie Ounsley.

TG: 08:45

Oh my gosh, I'm scared.

MF: 09:48

So now in front of you, when I'm talking to you, everything that I'm saying should be coming up in subtitles.

TG: 09:57

Oh my days. It's literally just right in front of me. That's amazing. This is great. It’s really gonna change a lot of people's lives. Why am I crying?

MF: 10:14

That's exactly why we've done it. Now I'm crying as well.

JO: 10:25

Sorry, I'm a bit speechless. It's just like, like my voice has come up as well than yours. Oh my god.

MF: 10:36

So you can imagine wearing this in restaurant for the first time, where you're sitting there where normally it's been really hard to engage in the conversation and you can look around the room.

JO: 10:53

Sorry, I'm not even an emotional person but…Sorry.

MF: 11:03

You weren’t expecting that were you?

JO: 11:09

No. That’s like pretty amazing because as much like I look to embrace the deafness and like be positive about it, it's like a really isolating place at times. You miss out on a lot and it's, you know, a mental journey trying to lip read. But like just something like this could really, you know, make it so much easier, not just on the side but deep also. Sorry, that really threw me off guard, but that's pretty special. And I’m speaking fast now and it’s pretty damn accurate as well, so…

TS: 11:50

As you said, your initial focus was people who are deaf and hard of hearing. Are there any insights into this market that you discovered through your research that others might not have? I mean, you have the personal connection because of your grandfather, but what else about this market did you feel was intriguing?

DS: 12:10

I mean, some of our discoveries, of course, is that it's not just one market. There are all kinds of different individuals with all kinds of different abilities, thoughts, passions, skills. So we can't just apply this as a blanket rule across everything. For instance, sign language versus captioning. That was a sort of fairly early discovery for us. And we need to be very careful. And we are not in any way saying that captions should replace sign language. I mean, absolutely not. But on the flip side, we're providing something that is beneficial to some people and they shouldn't be denied from having that. So there are all of these nuances and intricacies that you kind of have to navigate as you enter these new markets.

TS: 12:57

And how did you go about finding pilot users to test out the product before you launched it?

DS: 13:03

So we used our launch activity to find our pilot users. So when we first announced this to the world back in July twenty twenty two, what we announced was the fact that we had just about perhaps got this to work and invited people to come and trial it. And our pitch wasn't great, to be fair. It was, do you want to go and spend four hundred pounds, or dollars, of your own money going to buy some glasses to use this bit of software that kind of sort of works? But many people did, which was quite encouraging for us. Some of our users have been on a journey for over two years with us now to kind of perfect and hone this. Some of them have bought different glasses along the way. It's very much been something we've built with our users, focusing on them.

TS: 13:54

And how is the feedback from the users helped to refine the product?

DS: 13:57

So, I mean, all kinds of things that we hadn't thought about, all kinds of usability enhancements, making sure that we can, to subtitles as quickly as possible, the translation, that whole feature came from, know, repeated requests from users. The glasses, of course, were a direct result of all of the feedback that we got early on about the fact that they didn't like the tint, they didn't like the wires. So our quest to create, you know, what's the current ultimate device, and of course there'll be

another ultimate device in another year's time. But right now we've got something that answers almost all of the feedback that we have from our users.

TS: 14:36

So I understand initially you didn't actually want to get involved with the hardware, but you have got to that point now where you've developed your own glasses.

DS: 14:44

Correct and again some of the early feedback that we got from users was, so I've been on your website, it's told me to go over to Amazon to buy some glasses that say they're designed for video games and then you want me to download this app and then you want me to install this and do that and jump through a hoop and it was just a pain and by the way these are people in their seventies, eighties that maybe don't even have a phone and so it was just a painful painful experience.

What users told us was they wanted a set of glasses that just did subtitles and I don't want to have to mess about with phones. I just want to put them on, press a button and it just starts working. So again, that was what led to this all in one design with two buttons, start stop and much, much easier for people to get their head around the patchwork. Now the patchwork is still there. People have these glasses from other manufacturers. They absolutely can still just download our software and connect them. So again, we want to be an open app. We want to support as many different glasses as possible, but we also want this signature, easy experience for people that just want to these for some time.

TS: 15:57

Right. So you have the option of either using the glasses the XRAI has developed or using a pair of glasses developed by another company with the XRAI app.

DS: 16:08

Or just use your phone. Or use your laptop or use any other device that you want. Wherever you are, we want XRAI to be available to you.

TS: 16:17

And there are other use cases as well. Beyond the deaf community, there are others who could benefit from this. At what point did you realize that it could benefit a much larger population?

DS: 16:31

For the translation, think, is where this really starts to go completely mass scale, because only eighteen percent of the world's population speak English. There are hundreds and hundreds of other languages out there, especially when we start to look through the lens of enterprises and corporate use of this technology. The number of calls that happen every day between people who don't speak the same language. Imagine you had this ability to be able to do this real-time translation.

I think that was when it dawned upon us that potentially an even bigger market. Now the accessibility market is still super important to us and where the company started and again we have the ability to change people's lives through accessibility whereas real-time translation probably helped them out a bit but it doesn't necessarily transform their lives. So we're still equally focused on both but we're very excited about our translation.

TS: 17:27

And there's also the neurodivergent community as well that could benefit. Can you talk a little bit about who else this could help?

DS: 17:34

I mean, we discover people almost every week that didn't realize that we were able to help with this. One of our very early users suffered from APD, audio processing disorder. So it's an individual who can hear fine, but they can't necessarily understand what it is that they're hearing. Whereas actually, when you're able to read that as subtitles at the same time, all of a sudden, again, you're able to engage in the conversation.

One of the features of the app, a feature just by virtue of the fact that you're using an app for something like this is you can glance back at things that might have been said just before. So you're distracted for a second or didn't quite understand something as simple as being able to glance back at what was just said and reread it and actually understand what was being said. Again, hugely helpful. There are all kinds of advances and innovations that we're working on as well in terms of real-time understanding, again, of what's being said. Imagine you had the ability to be able to compress the number of words by 50 % using a real-time summarizer. So yeah, as you start to embed AI into all of these different scenarios and use cases, new opportunities keep presenting themselves.

TS: 18:59

And so let's talk about marketing because you were initially targeting a very specific audience. How did you approach marketing and building your brand and reaching your target audience?

DS: 19:10

So we took a very PR focused approach to this, again, because it's such a good news story, because it's such a great use of AI, there was a huge amount of press interest in what we did. I think we were on national TV news again in almost every country in the world, which was quite fun. And so, yeah, a lot of our initial excitement was driven through that and continues to be driven through that. And now we've got to build on that and do the slightly duller sales and engine and actually really start to grow this at scale.

TS: 19:45

Right. And I wanted to talk about some of the biggest challenges in developing the technology. You're not a first-time entrepreneur, but there must have been some unique challenges in launching XRAI. Can you talk a little bit about that?

DS: 19:58

I spent twenty years in my previous company doing professional services and managed services, so selling time effectively and dreaming of what it might be like to have a software company. So, you know, with a software company, you write something and then it just sells itself and you put your feet up and that's the dream. But let's just say that's not all white picket fences. You know, what you don't realize, of course, when you build a software company is the amount of work you've got to do upfront to get to a point where you have a saleable piece of software.

With a professional services company, you're billing from day one. Someone's creating something, you're charging the customer for it. With a software company, you're two years into this until you've actually created something that someone will actually part with money for. Then they will only part with a very small amount of money because it's a piece of software, right? It's not some custom thing that we've been custom built for them. It's a whole different economic model and it only works at scale. So actually it's much, much, much higher risk. so, you know, that's not, I guess, been a learning. We kind of knew what we're getting ourselves into, but we make a true realization of the amount of work and investment we've to put into these things, especially when you're dealing with consumers.

DS: 21:17

Again, when you're dealing with companies, they can sometimes be a little bit forgiving if you’ve got a few rough edges and a few things that don't quite work. Whereas consumers, you get zero forgiveness with this. It's got to work. Everything's got to work perfectly. And by the way, I'm sure as heck not going to read the manual. So this thing, I'd best be able to just figure it out by just randomly clicking on stuff and so, yeah, that's been a baptism of fire of how do you create consumer grade software because it's very different from business grade software.

TS: 21:49

Can you share some stories or feedback from users about how they've been impacted by the product? Any particular moments that really stood out to you?

DS: 21:59

I mean, it's the simplest things that stand out the most. My granddad is able to listen to the wireless, as he calls it, the radio. We just take for granted that people can just go out and have lunch with their loved one and have a conversation. Again, many of us take for granted that actually is an incredibly difficult thing if you’re deaf, especially when there's lots of background noise and so forth. And so, yeah, those are the stories that have really hit home how impactful this can be.

TS: 22:30

And what was your grandfather's feedback when he first tried it on? What was his reaction?

DS: 22:35

He didn't like the tint and he didn't like the wire. So he was the chief protagonist for making sure that we got that sorted. So he uses it on his iPad. He loves that. That's nice and simple.

TS: 22:48

That’s awesome. So we've talked about how the product has evolved and one of the areas that you're moving into is live events. What's the thinking behind that and what opportunities do see there?

DS: 23:00

So one of the other consistent pieces of feedback that we got from users was, this is fantastic when I'm at home or surrounded by a small number of people, but I'd love to be able to go to the theater again. I'd love to be able to go to an event. With theaters, for instance, typically only one showing is open captioned. And typically you can only see the open captions from a small set of seats. So when you're a deaf individual going to the theater, you are very much treated as different.

Again, we thought, why should it have to be like this? Why isn't every single performance captioned? And why isn't it captioned wherever you are in the auditorium, not just in a few designated seats? What about when you go to an airport or a railway station? I frankly, I struggle to hear announcements very often in those environments. I can't even imagine what it would be like to be deaf. So put all of those together, we kind of again had this little epiphany about what if we could take this technology and rather than just showing it on one device, broadcast it to multiple different devices. So instead of it running on someone's phone at the theater, for instance, imagine it ran on a laptop backstage at the theater, and then imagine it was broadcasting out subtitles to glasses, phones, whatever people might have.

DS: 24:24

And so that was the genesis behind XRAI Stream, which is effectively the same engine, the same software, but now being used in a completely different way. Although that in itself presented some quite unique challenges. So now imagine we sat at the theatre and two actors or actresses on the stage. Who says which line is critical? The entire plot could be thrown out if we accidentally labeled the line as Romeo instead of Juliet. And actually, that's quite a difficult thing to do. So what we ended up having to do was to partner with another organization called Dante, who no one has heard of. But if you've been anywhere with any kind of live audio, you'll have actually been using Dante.

But what it means to us is that we're able to take audio from up to 64 microphones in real time. So we can now take all of the audio from all of the cast on stage and we can actually figure out who's saying what, which actually then translates into the correct annotation next to the correct line. It sounds simple, but there's hundreds of of dollars worth of engineering behind that to actually make that come to reality.

TS: 25:44

And are you trialling this with particular venues at the moment or is it something that's already been launched?

DS: 25:50

So we've had this technology being used at a number of events already. We're working with a number of theatre companies around the world to trial this technology, but it's still all relatively nascent. So again, we're just a very young startup, but yeah, this technology is actually being used out in the real world right now, which is quite exciting, terrifying, so.

TS: 26:18

And you've also had a lot of big companies reaching out to you. Are they seeing other potential uses for the technology?

DS: 26:21

I mean, we're a whole mix, as you can imagine, across this accessibility and translation kind of use cases, all kinds of different companies using it for all kinds of different things. One of the other particularly interesting use cases which came out of it, and this again started in the conference space, but we think there are broad applicability across other sectors, is actually building assistance over the top of the transcripts.

So now imagine I'm at an event and maybe there's ten tracks going on at any one time. I, of course, can't physically be in ten places at once, so I actually end up missing out on ninety percent of the content at any conference that I go to. Imagine that all of the transcripts of all of those sessions happening across all of those ten tracks were all consolidated into a central point. And then imagine I had the ability then to be able to ask questions like my own private Chat GPT, but just off the transcripts of that event. Well, that's what we created, of course. So now for each one of these events, you get a bot free with it, if you like. So attendees can then create their own personalized summaries. You know, they can say, I work in this particular sector. I'm interested in this particular thing. Can you give me my own personal summary of what I missed? And up it pops. But again, you could imagine the same technology being used for all kinds of different things. I mean, from a consumer standpoint, and ability to be able to summarize conversations, pick out pieces of information from conversations that you've had and enterprise, the same thing. So, yeah, this assistant and AI piece, again, we're super interested in how that might unfold.

TS: 28:00

And are companies interested in using it in the work base, for example, or are there certain concerns around that that you have to overcome?

DS: 28:09

So yes, they are. And again, they're using it for accessibility and they're using it for translation. And yes, there are big concerns we have to address as part of that because of course we are transcribing potentially highly sensitive conversations that are occurring inside that organization. So the first piece of good news is that the entire app is built in a decentralized form. So we XRAI don't have access to any data of any conversation that's happened anywhere. So our consumers like that and of course our enterprise customers equally like that. from a ground level, really good spot. But even then some enterprises are still not happy with us even processing any of this information, never mind having access to it. So for those customers, we've actually recently launched the ability to be able to install XRAI into their own cloud.

So these big enterprises already have relationships with companies like Microsoft, for instance, they already have their own Microsoft cloud. They don't have to use our cloud now, they can use their own cloud. So we're working with some defense organizations, for instance, about that. And so now people can access the exact same software, they have no idea of any difference, but the IT admins know that all of that data is being kept within their corporate environment.

TS: 29:37

So looking ahead, what does the future of assistive technology look like in your opinion? And how do you see advancements in AI and machine learning shaping the space over the next few years?

DS: 29:40

I mean, I think AI is going to continue to give everyone superpowers. And again, I don't believe that AI is going to destroy the world. think AI is going to equip people with additional skills. And what's ironic or fortunate perhaps is that a lot of individuals who are accessing this technology for the first time as an assistive device are actually going to have superpowers above and beyond everybody else that they're interacting with.

DS: 30:17

I often watch TV with the subtitles on that I can hear, but I gain more from having that. There will be people that use it for translation that, again, are able to understand far more insight from the conversation. They will have assistants available on tap to give them more information. Even though it's an assistive device, actually, I think it's going to provide superpowers for anybody that might want to use it.

TS: 30:43

And when it comes to the broader conversation around accessibility and inclusivity in tech, are there any lessons that you'd want to share that you think that tech industry could benefit from?

DS: 30:54

I mean, think the world in general could benefit from a little, what's the right word here, realization that this is actually a challenge that they have to try and help with. The number of companies that we speak to that say, no, no, we don't have any deaf employees that work here. Like, really? First of all, if that's true, that's abhorrent, but more likely you just don't know. well, no one's asked for any help. It's like, well, that doesn't mean that they wouldn't benefit from it. And so, again, how can we help them tell themselves? How can we make something available to people without them having to make a big deal about it? Just to be able to say, yes, that would be beneficial for me. I would just like it, please. I don't want to have to make a big fuss about it. So yeah, I think the world in general could benefit from realizing there are many, many people out there that could benefit from this technology that are afraid, unwilling, unable to ask for it. And they should be more proactive in making it available.

TS: 31:52

And for people who are thinking about starting a company one day, what's your thesis about coming up with a great idea?

DS: 32:01

Don't do it. It's hard. It really is hard. it is lots and lots and lots of blood, tears. If you can do something that's you're passionate about, that's really, really important. think as you sit there at eleven o'clock at night, still not quite finished what you need to do for the day. That is what will drive you. So make sure that passion is there. Make sure it's a big enough market opportunity. I mean, there's a thousand good ideas that would help a tiny number of people and that's just a waste of time and effort.

If you're going to pour your life into something, it's got to be something that can really benefit an inordinate number of people. But again, as AI transforms the world, there will be so many opportunities to figure out different ways to harness it, different things it can do, and incredible experiences that you can create for users using technology so there's plenty of ideas still out there.

TS: 33:02

Okay, great. Thank you and thank you so much for your time. It's been a really fun conversation.

DS: 33:07

Thank you.

TS: 33:12

Thanks for joining me on this episode of Made For Us. And thanks to Dan and the XRAI team for providing the clips you heard earlier, featuring Ashley Walker, a pharmacist and entrepreneur based in Louisiana, Tasha Ghouri a former Love Island contestant and XRAI brand ambassador, and Jodie Ounsley, the first deaf female rugby player to be selected for a senior England squad. Head to the XRAI Glass TikTok account to watch the videos in full.

I hope you enjoyed this episode and if you're still listening, why not leave us five stars or share this with someone who'd like it too. And don't forget to follow us on Instagram and LinkedIn for updates and behind the scenes content. You'll find us at Made For Us podcast. See you next time.