AI vs the feedback gap: making workplace feedback fairer, with Textio's Mykel Rangel

MR: 00:00

We found that women receive twenty two percent more personality feedback than men. Women are described as bubbly or they're described as emotional or they're described in all of these different ways that have nothing to do with their role.

TS: 00:22

Welcome to Made For Us, the show where we explore how intentional design can help build a world that works better for everyone. I'm your host, Tosin Sulaiman.

Are you a manager or someone who's ever given feedback? How much do you think about the feedback you give? And have you ever considered using AI to help improve it? Joining us today is Mykel Rangel, VP of Engineering at Textio, a company founded over a decade ago that's revolutionizing how we think about language and hiring and performance feedback. Years ago, Mykel wrote this about Textio - we're not riding the AI train, we built the tracks. In the conversation, we'll explore how Textio pioneered inclusive communication technology to ensure that language and job ads and performance reviews is high quality, actionable and free from bias.

Before we dive in, I have a favour to ask. If you're enjoying the show, please take a moment to leave a five-star rating or review wherever you're listening. In a world full of podcasts, your support helps this one get noticed. Now here's my interview with Mykel.

MR: 01:31

So my name is Mykel Wrangel, I use she her pronouns and I'm the VP of Engineering at Textio.

TS: 01:37

Tell us more about your career journey and what attracted you to Textio.

MR: 01:40

So I started my career as a tech writer doing hardware software manufacturing and doing user guides and compliance guides and instruction manuals and translating them into 15 languages to get them shipped into random countries. And what I learned as a tech writer is I was documenting decisions that were not necessarily the best for the user. And at the point that the tech writer is documenting it, the products and manufacturing, everything is like it's the last step to get the document in the box. And so I really wanted to be able to influence the decisions a little bit further upstream.

And so I left a career in tech writing and moved into software. And so my first career move there was into ad tech. And I worked there for about five, six years and I learned a ton. But when I took a step back to see what was my contribution to the world, was I using my skills to help make things better and improve the lives of people who are impacted by the technology? Ad tech didn't exactly fit that. And so when I learned about Textio, I was so drawn to the idea that there was science in language. There was things that really statistically did and did not resonate with other people.

MR: 03:10

And you could measure that and you could change the outcome. And Textio's first product was around job posts and knowing based on this language, you will get more people to apply. You will get a higher quality candidate pool. You'll close your jobs faster. And that was so cool to me. And the impact of that, like one job post that one person writes is seen by millions of people and now they're picturing themselves in this job. They understand how they could show up at this company because of the words that were put into that job post. And that was such a cool and unique and sort of mind blowing concept for me that I just, I was absolutely drawn to it. And so that was my journey to join Textio. And I've been here for seven years. I can measure it by like the height of my children.

And over that time, we started with that job post product. We expanded out to recruiting mail. So how to tailor your cold candidate outreach to increase the chances of response. And then over the last couple, we have now been building out performance feedback because we know that feedback that people experience in their careers more than maybe anything else influences how they can grow, how they can advance the opportunities that they're given. And so we saw this opportunity of recruiters and hiring managers. They worked so hard to find these great candidates. They write their job posts in Textio, they go through this entire application process and interview process and they get them in and they're like, yeah. And then the people receive low quality feedback from their managers and they leave within a year and it starts all over. And so we really saw this need that was not being addressed. And so it's a very natural expansion from helping get great talent in the door, and then helping to grow and retain them once they're there.

TS: 05:14

So, Textio is now 10 years old. So, the technology has been there for a while to address bias in recruiting and in job ads and so on. Was no one focusing on the problem of bias a decade ago?

MR: 05:30

So there was this idea that there was bias in job posts and in language, but it wasn't being addressed in a systematic scientific way. so Textio has been building equitable software. been building through AI and LLMs, bias detection and bias removal through language. There's two sorts of AI, sort of how we think about it. There's the broad general, which is what you get with ChatGPT, which is trained on the giant data set of the internet. So the results you are going to get have all of the ingrained bias of all the people that write on the internet. And then there's the vertical specific. so Textio has been building vertical specific AI to identify and remove bias for 10 years. And we are constantly evaluating the best tools for the job and the best way to bring on new tools while holding ourselves to that very high standard of ensuring safety and bias removal and mitigation in what's being brought into the data set and also what's being produced by the AI.

TS: 06:41

Right. And I noticed in one of your LinkedIn posts, you said, we're not riding the AI train, we built the tracks.

MR: 06:49

Yeah, so when we first started with Textio, one of the earliest features in it was the gender tone meter, which obviously we know that gender is fluid and there's not this binary construct, but even that was such a step forward from anything else that software was trying to do. The idea that when you are hiring, you're writing a job post and you're hiring for a VP of engineering is your default pronoun he him like there is built in bias there is the person that is reading the job post and what they're looking for and so that recognition that the bias in the language is something that impacts who is going to apply who's going to show up who is going to see themselves and so that was something that was not being built or produced in any sort of scalable way 10 years ago.

TS: 07:50

And who are your clients? What are their goals in choosing Textio?

MR: 07:58

We have a really incredible client base from smaller companies that are our primary set is the NBA is the spot of size, the VPs, the oracles, like the large companies are the ones that really understand that this is a problem that cannot be solved without software because we are showing you things in your language that you wouldn't know without Textio. And so the large companies are the ones that tend to want to buy and deploy Textio at scale across all of their managers, especially in order to help mitigate bias in how the managers are writing reviews. because managers, they're responsible for the greatest asset of the company, which is the people. And so there aren't as many tools that are there to support managers, especially as we're seeing managers needing to do more with less and needing to manage more direct reports. So it really is those larger companies that are trying to solve problems through software.

TS: 09:12

As you mentioned, you have products that help recruiters write inclusive job ads, but also help managers notice the bias in their performance reviews. Can you describe how each of those products works?

MR: 09:26

So they both have a similar UI convention where you're writing in an editor and we highlight phrases for you to pay attention to and address. In performance reviews, those phrases might be about including non-actionable feedback because we think about high quality feedback as nice, good feedback is nice feedback, and that's not true. Like the feedback that is great is the feedback that is high quality and actionable, and people know what to do with it to grow and advance in their careers. And so as you're writing in the performance feedback, we give information about this is a blanket statement, or this is a cliche, or this is, worst case, it's a personality phrase. You shouldn't be commenting about their personality. That's not relevant to their job.

MR: 10:19

And in job posts, it's things like you should include more information about the company's benefits. You should include the equal opportunity statement. You should remove this fixed mindset because that is communicating that it's innate ability, not something that someone can grow into. And so that's sort of the convention across both products is that we give you in the moment, real time guidance to inform the writing as you're doing it. We also have our generative features that we have built and have our built-in bias protections in them so that you can generate a job post just with a few clicks. You put in some metadata information and we know which company you work for because you're in your Textio account and we generate a job post for you very quickly.

In the performance feedback, we know that there are four categories of feedback that you tend to give in a performance review. There's problematic behavior, there's coaching behavior, so it's not a problem, but it's a growth area. And then there's celebratory behavior. And then there's just raw notes. Like, I just need something out of my head and then help. And so we have UI there where we guide managers through how to create high quality, actionable feedback for each of those four categories very, very quickly. The amount of time and efficiency saved for performance reviews is astronomical. Managers spend eight to 10 hours per review per person and with Textio, that gets cut in half. so it really there's the equity, but there's also the time saving that comes in both the products with the UI we've built.

TS: 12:00

Right, so what you're describing sounds a little bit like Grammarly, but instead of correcting your grammar, it sort of corrects your bias.

MR: 12:07

It corrects your bias and it also generates content that you might not know. So when you sit down to write a performance review, sometimes you have an idea of what you want to say, but sometimes you're sitting in front of the blank screen with the four prompts that your HR team gave you and you're like, all right, what do I do here? And so Textio can take just a little bit of information from you and create a performance review. Obviously you need to read it, you need to add content that is relevant to your person and where they are in their career and all of that, but taking you from nothing to something that is high quality very quickly.

TS: 12:46

And in terms of the data sets you use, this is data from your customers that they've opted to share with you.

MR: 12:53

Yeah, so there's a few different ways for both job posts and performance feedback. We did the pretty hard intentional work of collecting datasets. And if you think about performance reviews, not many people want to share their performance reviews, not many companies want to share their performance reviews. It's incredibly personal. And so we were able to build out datasets for both of those products. And then also pulled different surveys together so we could grab it from people that were not necessarily being represented within the company because the culture and dynamic within a company is unique to that company. And so we wanted to make sure the data set went across industries and companies and teams and people. And so we pulled from a few different places.

TS: 13:35

I'm interested in the company patterns that you've noticed and what it reveals about the culture of a company. I think you did some research into companies like Uber and Amazon and the types of words that they use in their job posts.

MR: 13:48

That one was so interesting because it was X-raying the company. It was looking at the language and that the language is showing, it's holding a mirror up to what's happening in the company. And when we did that analysis and looked at the results, no one was surprised. Like the people in the companies felt very validated. They're like, yeah, that is how I experienced it. And so there was some really cool actions that came out of that where companies were part of that or companies that weren't, but they wanted a similar thing about their company.

And so we were able to pull those reports and like give them that, that insight. And then we could ask them, is that something that you want to be known for? Is that a part of your culture that you embrace or a part that you want to move away from? And in Textio, we could program in, let's say the word was maniacal and they didn't want that to be part of their culture anymore we could program that into Textio. So when people were writing it and that word came up, we would give them a different replacement suggestion. And we can do that for each company for the words that were unique to them. And so it really was helping to bring the data, bring the visualization to something that people kind of felt when they worked there, but they couldn't necessarily articulate.

TS: 15:05

That's so interesting because I think one of the examples was that Uber uses the word maniacal 11 times more than the rest of the software industry and Amazon uses the phrase whatever it takes much more than other companies.

MR: 15:19

And some of the responses were like, yes, that is what we want to be known for. We're glad that that comes out and you can opt in or out of that. And it is actually helpful for people who are applying to jobs to see that phrase and know that that is what they're coming into as a culture.

TS: 15:37

And have you worked with companies across the world? Have you noticed any international differences in terms of language?

MR: 15:45

We have customers all around the world and there is nuance in language. And I think some of the biggest changes we've seen is we get the feedback that our guidance can be very Americanized. And so when we are building out different pieces of, of language guidance, we try to get a more representative sample. That's not just us in Textio who work in the United States. And so when we're getting feedback from customers about this is too Americanized. We try to understand more what is appropriate. Like in some of our customers' cultures, the directness of the performance feedback product is not appropriate. And so we work with them to understand what makes the most sense for the environments that they are working in, that cultural differences that we might not have. So it's about how to get that from the people that do.

TS: 16:43

And I wanted to turn to the performance feedback, because as you said earlier, even after people are hired, there's still a lot that we can learn from the language that's used in the workplace. So you've been looking at performance reviews for many years and you recently released a report on language bias and performance feedback, which we'll spend some time talking about. But first, could you explain why you decided to look at feedback in the first place?

MR: 17:07

Yeah, it's one of those areas where we've all felt it. Like we all have some experience of where feedback has impacted our careers. And so often it's outside of necessarily the formal review, but the formal review is the thing that is consistent across companies, it's the thing that from a practical perspective, our customers were asking us to They're like, please go build this. So like we had that obviously, but when you look at if there is a place where we could make the most impact in a person's experience at work in the most consistent way, the formal performance reviews are that moment in time.

And we know that not all companies do them, that some companies are trying to switch to more quarterly or anytime feedback, but there's so many companies that the HR team's like, we just want our managers to do them. We want them to write anything. And we're like, well, maybe instead of writing anything, we can make it so they can write it very fast and bias-free and high quality. And so it really was filling a need that we were hearing and that many of us had felt personally in our careers around feedback.

TS: 18:29

And I guess we should define high quality versus low quality feedback. You talked a little bit about this earlier, actionable, giving examples, but yeah, could you expand on that a bit more?

MR: 18:40

So high quality feedback, it's clear, it's relevant, it's actionable, and it includes examples. It's not related to their personality or their likability or including hedging statements like I think. And so that difference of it might make you feel warm and fuzzy, that's not high quality. That doesn't tell you if you're meeting the expectations of your role, that doesn't tell you what you need to do to advance in your career. And so that high quality feedback versus low quality feedback, it really is, it's very clear cut. And we've seen that the higher performers on a team, they get one and a half times more feedback than anyone else. And they also get significantly more problematic feedback, significantly more low quality feedback, even though they have more words.

TS: 19:40

That's so interesting. And also you found that there's a correlation between the quality of feedback and employee retention and pay.

MR: 19:49

Yeah, that people that get low quality feedback, they are 63 % more likely to quit when they get that low quality feedback. And so you have these high performers, the people you most want to retain, the people that the managers are writing the most words for. And if it's low quality, they're 63 % more likely to quit. And so now you have your highest performing people leaving your organization because of the feedback they're getting from their managers.

TS: 20:20

So you've identified feedback trends in your reports going back for the past 10 years. What are the trends that have stayed consistent?

MR: 20:28

Yeah, so over the years, we've continued to look at the data through the lens of race and ethnicity. some of the things that we found is from previous report, we found that women received twenty two percent more personality feedback than men. Women are described as bubbly or they're described as emotional or they're described in all of these different ways that have nothing to do with their role. And then that continued into the report that we saw this year. So we saw this year that employees are likely to remember and internalize the feedback that reinforces the naked of stereotypes. So in previous years, we identified what are some of those stereotypes for different race, genders, ethnicities. And in this report, we looked at what are the impacts of that? How do the people who receive that internalize it?

And so some of the discrepancies we saw is men are two to four times as likely to internalize the positive stereotypes that they get about themselves. Whereas women are seven times more likely to internalize the negative stereotypes. They recall being described as unlikeable, fifty six percent of women recall that. In a performance review, fifty six percent of women were called unlikeable and remember that. Whereas men it was sixteen percent. And so it's real, it's personal. And people remember that. And it impacts how they show up at work. If you're described as unlikable, how easy is it going to be for you to stick your neck out in a hard conversation and bring a point of view that you know is not going to be popular. And now you're described as unlikable again.

TS: 22:18

I was curious, as you were doing this research, how much of it resonated with you and your experience throughout your career in terms of the type of feedback that you've got.

MR: 22:27

It was very interesting because I've had a few different moments of feedback that stick with you in your, in your career. I remember one, I had just transitioned from being a tech writer where technical communications, like it's structured. There are rules, there is order, there is no gray area. And I had written my first spec for a software product and I had gone back and forth with the engineers. I had answered every single question. I was so proud of it. Like I'd never done this before. I didn't know what I was doing. And so I was so excited and my boss calls me into his office and he's like, we don't get in pissing contests with the devs. They know more than you. wow. Okay. I'm going to get fired right now. So I went back and I edited the spec to add contractions to soften it to add questions on things that were not questions. I knew they were answers. And that was the first moment where I realized I need to change how I show up. And that was my first moment that was so recognizable in the research of performance feedback because I was doing my job really well. But the way he took it and then how I had to change how I show up to fit what was an appropriate view, it took a decade to unlearn that. And it took living and working in a safe space to unlearn it, because I was very successful in that model, because that's what the feedback told me I had to do to succeed.

TS: 24:12

It's a great example of non-actionable feedback. At least you ended up in a good place. So I think it'd be good to turn to the implications of your research. I guess a bigger one is that in addition to trying to fix the gender pay gap, organizations should also be tackling the feedback gap.

MR: 24:33

Absolutely. Yeah, absolutely. And it's one of those places where a lot of companies are just starting from, desperately want managers to give any amount of feedback. And that's why it can be so helpful to have Textio because it's not just, please write anything, because that's not enough. Because if they just write anything, then this is the sort of problematic feedback that they are most likely going to write and it might not be ill intended, but that's the outcomes of what they're doing. And so by having in the moment guidance of what they're writing, it closes the feedback gap because now it's not, please just write anything. It's you are going to write something and you can be confident that what's written is high quality, is equitable, and will start to remove some of these statistics.

TS: 25:27

And are there any other specific recommendations that you have for leaders?

MR: 25:35

So AI is something that is, it's been around for decades, but it's something that people are either very excited about or very scared of. And regardless of which you are, if you're starting to evaluate what sort of products, what sort of AI you want to bring into your company, start with the people building it. Like ask if you can meet the people that are building the AI. And if you can't, that's your first flag because it means it's probably open AI wrapped in a pretty wrapper. And then once you meet the people, understand how they're mitigating bias in what they are building. Do they have a representative group of people building the software for the outcomes that the AI is intended? And if not, what steps are they taking to fill those gaps? Are they working with consultants?

MR: 26:31

Are they working with contractors that have that specialized skill set, how often are they rotating out those contractors so that they have unique perspectives coming into what they're building? Where are they mitigating for bias going into the AI? And then when AI produces results, how are they being evaluated and updated to ensure that they're both serving their intended purpose and remaining free of bias? And so I'd say AI is, it's not new, but it has had fantastic advancements over the last few years. so approaching it with curiosity, optimism, and also a healthy dose of making sure you understand what you're buying so that you're not unintentionally doing more harm than good in bringing it into your company.

TS: 27:30

Yeah, that's great advice. And should people be approaching it from the point of view that it is sort of biased by default unless it's been built intentionally to mitigate bias?

MR: 27:34

Exactly, yes. And they should be able to speak to how they do that. So the data sets, for example, the data sets that go into building the product, if it's curated from out of copyright literature, corpora, that contains a heavy text that's authored by educated Western men, not uncommon. Well, that means the descriptions of women and underrepresented ethnic groups may reflect unfair stereotypes of those times.

Is that an important thing for the product you are building? Or we have in our product where we look for potentially harmful phrases, we look for different slang. Well, maybe we want to curate some Reddit comments because those tend to have a demographic that skews younger and leans left in American politics and the text, they have some misogynistic or racist or ableist perspectives and some newer slang.

MR: 28:29

Well, if we're trying to build a model to capture those things, then we want that because then we can capture it. We have a better representative set. And so it really is understanding all the way through the system, how the team is paying attention to where there could be built in bias, by what views are and are not represented and making sure they're intentional about how they mitigate those.

TS: 28:55

Mykel, it's been a pleasure having you on the show. Thank you so much for sharing your insights and sharing your story.

MR: 29:00

Absolutely. Thank you so much. I really appreciate it.

TS: 29:08

Thanks to Mykel Rangel, VP of Engineering at Textio for joining me on the show. By exploring the role of feedback in the workplace and the ways language can carry bias, we hope we've given you some food for thought and we'd love to hear from you. Have you ever come across a job ad that seemed tailored to a specific gender or felt that your personality was the focus during a performance review? Share your experiences with us on social media. You'll find us on LinkedIn and Instagram @MadeForUsPodcast. And if you're listening on Spotify, you can also leave comments below the show notes.

TS: 29:42

Coming up next week on Made For Us.

TB: 29:44

So I started thinking more about like the craft of design and our responsibility and power and things like that. And I just thought about like, product development is actually broken, it’s actually done in such a way that we aren't meeting the needs of humans who use our products and we see it and feel it every day.

TS: 30:03

I'll be speaking to TB Bardlavens, Director of Product Equity at Adobe, on the intersection between technology, design and social impact. We'll also have our first ever audience Q&A with TB answering questions from listeners. If you'd like to get a sneak peek of the episode or you want the chance to put your questions to future guests, sign up for the Made For Us newsletter at madeforuspodcast.beehiiv.com. I've also included the link in the show notes. See you next time.