Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Business Report Deep Dive: Reading between the lines of the written and unwritten
39 Plays8 months ago
In this episode, we breakdown a white paper written about feedback and how it pertains to Asian professionals. What did the paper reveal about the type of feedback Asians receive from their managers, and how should we interpret these findings? Should we be skeptical? We discuss these questions and share how to be a savvy consumer of these types of business reports!
Recommended
Transcript
Introduction to Language Bias in Feedback
00:00:10
Speaker
Okay, welcome back to another episode of Hidden in Plain Sight. We are your hosts. I'm Jenny. I'm Echo. And I'm Doug.
00:00:19
Speaker
All right, so in today's episode, we're going to be going over or kind of dissecting a language bias and performance feedback report that we found online just for some context. I was looking for, you know, has there been a study or some sort of report looking at the different types of feedback that people get at work, not just by gender, but also by race. And I came across this report and we thought it would be interesting to kind of dissect what they did.
00:00:48
Speaker
what they did well and what they could have improved on. So I guess we can jump right in. We need to give a quick overview on what they mentioned. I think they have a couple of bullet points on the main findings.
00:01:03
Speaker
So at least the audience are familiar with the content.
NLP's Role in Identifying Language Bias
00:01:07
Speaker
Okay, so the report was written by a company called Text.io. And Echo could probably speak more to this because she actually got a demo from this company. But it's a company that specializes in managing performance feedback. But Echo, could you tell us more about what you know about Text.io? Yeah, it's a third-party
00:01:30
Speaker
small tech firm that specialize using like NLP, like national language processing, looking to the talent practices in including like hiring. And I think in the past where I got a demo from them trying to see how they use their technique to identify some bias in the employer's job posting. So make sure the job posting is
00:01:54
Speaker
getting unbiased texts so you can attract more diverse and be more inclusive to your candidate. And I think in recent years, they also apply a very similar technique and in this round focused more on like a performance feedback that we all see and we all do in a company. And this report is actually about that. Companies that you will see, usually they publish research report
00:02:22
Speaker
in order for them to get more clients. So some of the content that they publish, it's very interesting, but definitely like a disclaimer that we do not get any promotion from them and nor do them ask us to do a promotion
Impact of Feedback Quality on Retention
00:02:37
Speaker
on them. Thanks for providing that context. I think it's very helpful to set the stage for our audience. Another thing that I think we should do that might be helpful is to just talk about performance management in general and why we're talking about this feedback report.
00:02:51
Speaker
Yeah, I think one thing interesting that in the report, they already also mentioned like why they started to dive deeper into this field. I think one of the key connection they draw is about employee retention. One thing they did trying to get to is when they were analyzing the performance feedback data, they realized the poor quality of the performance feedback actually showing people away. So they see people have no intention at work.
00:03:19
Speaker
and more likely to leave their job. So that's like a big statement that you're trying to put to draw people attention on why you need to spend to look at your performance feedback. Darko can see you're nothing.
00:03:35
Speaker
I agree with the premise that they have. If you have good quality feedback and you know what your goals are, you're more likely to stay. I'll go ahead and go on a limb and agree with that general statement that everyone else probably would too. I think it's an interesting premise that they're going off of showcasing the information that they have access to could potentially lead to that.
Verbal vs Written Feedback Gap
00:04:00
Speaker
And what would you say your experience with feedback has been in the workplace? Most feedback I've gotten is kind of mediocre, or things that I've already known, or it wasn't surprising to me. So for me, I guess there's that situational awareness where I knew what maybe my weaknesses were already. I didn't need someone else to put that out.
00:04:26
Speaker
to me. In the workplace, I would say maybe I got my mediocre feedback, but I knew what I was getting into with that. What about you, Echo? I actually do think I got a lot of good feedback along my way through my career, but oftentimes it's not in the reach and format.
00:04:50
Speaker
meaning I actually heard a bit more constructive and quality feedback through my conversation with the other person and this person did not necessarily to be my boss. Whether or not the written feedback is a reflection of the same quality I receive, probably not. I see some people put more effort into the written format, some people care less about it. I do think
00:05:17
Speaker
the missing link they probably haven't been able to uncover is the linkage between the career development and the quality of the feedback. And then how does that going to lead to this person going to stay or leave? Because I almost feel like they're trying to see that linkage, but because the data they received is not
00:05:42
Speaker
designing the way that capture there's more system that way. But I can clearly see the feedback loop on the one's career paths and then on the person's intention to stay.
Challenges of Feedback with Gender and Race
00:05:55
Speaker
Okay, so you're just jumping right into the article. So I guess we could just jump in and- Wait a second, Jenny, what about you? Yeah, but what about you? I thought I was gonna be sneaky about this and just- Nice try. Go into the article.
00:06:11
Speaker
So I've been pretty fortunate with feedback. I think I've gotten pretty good feedback from my managers, both present and in the past. And I think a part of it is I'm very proactive about asking for feedback and opening up the conversation with what I'm good at and what I need to improve. And I think that kind of makes it an easier conversation to be had for the manager. Yeah.
00:06:40
Speaker
I love that like I love that you're in constant communication and you know going in and you're right like good actually feedback that proxy really should be that like you should know throughout all this time if you're surprised by anything on your performance appraisal like it's it potentially feels like retaliation at that point if you don't know about it because it's this buildup of all the stuff you've done throughout the year right so if you don't
00:07:06
Speaker
Know about this or don't remember that that shouldn't yeah
00:07:11
Speaker
Yeah. Hopefully people remember what they do or didn't do within the year. Or you get feedback that is just not even surprising, just underwhelming. So there's new learn, no new insights. And you walk away from that meeting being like, huh, what was that about? Yeah. Yeah. Or the fact that, you know, some people are trained managers and they're giving feedback just to give feedback. And that is just a whole nothing sandwich.
00:07:37
Speaker
So I think that's a really good segue into the report. The report's premise is basically feedback is difficult to do, not a lot of managers are good at it, and we'll get into this later, but it's even harder when there are things like gender and race at play.
Stereotypes and Unexpected Findings
00:07:53
Speaker
So where do we want to start?
00:07:55
Speaker
Yeah, can I just say this is like, like to describe this to our viewers and of course, or sorry viewers, our listeners say they can't view us. This is more like a business report where they're giving us executive summary and key findings. It's quite different from the other readings that we've actually provided to our listeners in like academic articles and HBR articles.
00:08:16
Speaker
So it's kind of to the point and has graphs here and there that will kind of bring people along. So it's a much easier read, I feel like. But then it probably has less information than what I'm more comfortable with. So we'll dive into that later though. But more so like a business report.
00:08:39
Speaker
That's a good point. The problem is a lot of managers and leaders are going to be looking at a report like this. They're not going to spend time reading dense empirical studies combing through statistical analysis. They're going to look at something like this. So being able to parse out what's insightful versus what you should take with a grain of salt, that's going to be a really important skill to have as a manager.
00:09:05
Speaker
So the first word we can start with, or the first area that we can look at are the personality words that people use to describe Asians. And just for reference, they looked at four different groups, white, black, Latino, and Asian professionals. So Asians were described as the, or were the most likely group to be described as ambitious compared to, for example, black employees. So I think we could start there.
00:09:29
Speaker
I got a bit of a confused on that one and probably also surprised is the word and we all know like Asians and if you all caught up our previous episode, you will know that Asians are usually described as
00:09:45
Speaker
less ambitious and non-dominant. I wouldn't necessarily associate Asian with ambition. Jenny mentioned ambitious is the word that they find on Asians, but I think what surprised me the most is they also find Asians are
00:10:01
Speaker
more likely to be described as aggressive, three times more than whites. And it was like, why is that? I got so confused. I can't really argue this is the fact because that's what they find, but I'm just thinking what is actually getting into this.
Critique of Report's Organization and Clarity
00:10:19
Speaker
Well, let's talk about it. So ambitious and aggressive. I think those two could, could those two things be related? Yeah.
00:10:26
Speaker
I would say they did give us some, if they gave us more information on maybe what sector this was in. Is this private equity? Is it finance? Is it startups? I think maybe context like that would be helpful in this situation. I think they just mentioned that they got
00:10:46
Speaker
25,000 people from 253 different organizations, but didn't really say what types or what sectors they're in. Because I was kind of thrown off too as well, just like you echo, because I'm like, the research that we've done and the research that we've seen that have come in the last few decades, and even before that, doesn't show that it's kind of a contra to what we've found.
00:11:15
Speaker
as Asians are viewed as submissive and subservient and kind of opposite of ambitious. Not that I'm against it. It's good that we're breaking the norms and stereotypes. That's wonderful. I'd love to see where this is happening and to see why are Asians so aggressive in this industry.
00:11:38
Speaker
You know, that's a really good point. It doesn't say, I don't think it says whether this was US specific. So it says 253 different organizations, 25,000 people. I do know that like a lot of the companies who work with Taxio are in the tech industry and are rather than like a small to medium size. If you're looking at like 200 something companies and with
00:12:03
Speaker
a total of 25K employees that's not a big tech firm or anything like that. So I think it's safe to say it's a small to medium firms and a lot of them are in tech.
Cultural Challenges in Feedback Delivery
00:12:18
Speaker
The interesting part there is I
00:12:20
Speaker
And it's like how I rationalize that fighting is like people don't like people being aggressive. And but they also see this is more likely happen to certain group versus the other. For example, usually
00:12:37
Speaker
women are less likely to be expected to act aggressive than men. The issues here that I'm seeing there, wouldn't it be possible that because people are expecting Asians to not be aggressive, so when you act even the normal level of aggressiveness, you are getting called out or called out on like, hey, you're actually being aggressive and or ambitious.
00:13:05
Speaker
And that's where I think could be at play here. Otherwise, I couldn't explain why. I'm very confused by how Asians are being called out, being aggressive. So let me go through the rest of this table. So Asians are, like I said, the most likely group to be described as ambitious. And then they are the least likely group to be described as passionate, less likely to be described as easy to work with.
00:13:32
Speaker
You know, it's kind of interesting because, you know, as psychologists, when we say personality, we probably think of like the big five. I think the way that this report kind of showcases it by, you know, indicating that these things are personality. So ambition, easy to work with, professionalism, like passionate.
00:13:51
Speaker
Which is kind of kind of different from I would say what I'm used to So it did take me like a little while to wrap my head around it and try to make sense of it and not just brush it off I think it is quite interesting and going back to echoes for like yeah, I wonder what like why You know why Asians are perceived as ambitious And but if you're ambitious, how are you not passionate?
00:14:17
Speaker
I feel like it could be like hand in hand or are you just like ambitious but just like cold and mean? I don't know. How they understand impassionate is somewhere like different from how we understand passionate because I think Hispanic employees in their report were rated the
Disorganized Presentation of Studies
00:14:40
Speaker
group that is most passionate. One of the most, yes.
00:14:44
Speaker
Yeah. The terminology is subject to interpretation. Yeah. I mean, the other thing I was going to ask was like, I just wish they did like some sort of like even simple thing, like a regression or T test or something like that, just to just compare the percentage to see if they're actually meaningful difference. And there probably is.
00:15:06
Speaker
and to just really, really highlight that. Because I think their data is actually really interesting if they were to actually combine it with some traditional stats. And I think what we're probably used to is probably, when we say passionate, we have to operationalize that in.
00:15:21
Speaker
the psychological realm we have to say what it is and what the construct is and what it the sub construct may be and all that here they just have the word and we don't we're kind of filling the blank ourselves with what passionate means and what ambitious means and that's why i'm kind of
00:15:38
Speaker
Yeah, I think based on my understanding on their methodology, this goes way back because we were more probing into the technique they were using. Most likely they were using a library where they have words say like
00:15:54
Speaker
they categorize this group of words that falls under passionate or they categorize group of words. Of course, based on the NLP technique, the library is one way to go. There's like a word embedding, which is like joining similar words that give you a cluster of words in that meaning in essence. I don't know where things are at with them today,
00:16:23
Speaker
my educated guess is they probably have some sort of behind the scenes categories that set already, which almost made me wanted to do an audit on how they classify aggressive, how they classify, collaborate, how they classify passionate, just because the fighting that we find is kind of counterintuitive.
00:16:58
Speaker
Another area that I wanted to bring up was just how disorganized the report felt at times. You know, they were drawing from several studies, but it was kind of hard to tell like which study they were referring to. What do you think? I think they had like what, three studies right, Jenny?
00:17:14
Speaker
Like they had the one in the previous year that they alluded to, the one that they're talking about right now. And then there's another one where they actually surveyed 500 folks. I'm totally with you. Like I was reading through that. I'm like, what data are you presenting? Is it the current study? Is it this other study? Like I couldn't follow them. Like I had to wrap my head around that. And I was trying to figure out like, where are you going with this? Like where are you taking me?
00:17:42
Speaker
Yeah, I'm with you on that. I wish they would have shared that in the methods. No, they kind of went back and forth between different studies. I think one of the study was based on results from an employee survey. So they asked people, you know, how often are you described this way? And then the other.
00:17:59
Speaker
study was based on actual performance data, like performance management data that they gathered from organizations. But they didn't really make that distinction clear. So I had to go back and forth and I found myself confused. So I'm glad I'm not the only one who felt that way.
Unclear Promotion Paths for Asians
00:18:15
Speaker
Yeah. And Jenny, you know what, this is so interesting is
00:18:19
Speaker
I think what strikes me the most is Asians reported. 54% of Asians say they understand how to earn their next promotion, meaning there are 46% of them they don't, even though they're getting a lot of positive feedback being described as competent.
00:18:40
Speaker
And seasoned listeners, you know we have a problem with positive stereotypes. But I'm really questioning, like, what is actionable feedback? I know this is their, like, a proprietary, like, algorithm to tell whether this feedback is actionable. So in that sense, Asians getting the most feedback on things they need to improve. And Asians still don't know
00:19:08
Speaker
what needed to be done in order to get that promotion. Gosh, I'm trying to wrap my head around this, too, because in this case, I feel like it's either in actionable or actionable, right? But it doesn't mean that it's quality. Or should I be assuming that it's quality? Should we just call it? I'm going to clarify. We didn't really come here with your report.
00:19:32
Speaker
Just make sure that we are understanding what you're trying to strike the point here. I
Direct vs Indirect Feedback
00:19:39
Speaker
want a second. Can I second that?
00:19:44
Speaker
One thing I also wanted to call out this concept of like providing hedging feedback is kind of interesting. Yeah. I think what they define like hedging is like the manager gave indirect and kind of give you the instructions, but not say that exact expectations. That concept is very interesting for two reasons. The first reason as a people manager, I know sometimes I had a tendency.
00:20:11
Speaker
to dilute the ask of my employees or my team. This is so in contrast to all the manager trainings that I have received so far that even though we need to give direct feedback,
00:20:26
Speaker
But the tone and the feedback that we give, the first one is the one that we actually opposed to, because that actually triggered people's defense right away. You're almost giving orders to someone. I know even from where I have been brought up, and I know this is true for a lot of Asian people, giving direct feedback is not something that we're good at, because we
00:20:53
Speaker
or respect this harmonious culture, like we are modest, we give back, but we are not really trying to deepen the relationship with each other.
00:21:07
Speaker
Even when I see the words they use to, as an example, not as like a hedging feedback, I'm like, there's no way I'm going to say this to my team. So I'm curious, how would you ask someone or how would you give someone feedback echo? I would probably just say, hey, I noticed that recently you have this tendency that we have seen like you kept missing those deadlines.
00:21:29
Speaker
I wanted us to come up with the action that how to avoid this happening in the future. You might want to consider this and this and that. And so this is all in this sort of like a conversational format instead of like, hey, so on and so forth. You need to finish this. You need to do this. You have to do this. You must do this. I don't know.
00:21:55
Speaker
That actually challenged my entire philosophy of being a manager to someone. So it sounds more like problem solving together, like partnering with the person as opposed to directing them. Yeah.
00:22:10
Speaker
we should definitely consider giving more like a definite or more clear expectations. So like cutting some words on like a bit or not, or I think some of the example they gave there is the words they consider as hedging words. But I kind of disagree on their example that they were given. Like some of the example they were given is too arbitrary that I doubt anyone would think this is a good practice for managers.
00:22:38
Speaker
We're just like, you know, as a, as a, you know, a manager and maybe even like a leader in the team, like, I mean, I think there are questions you can ask, like, you know, Hey, Brian, can you do you think you can finish this report by this coming Thursday? And you get a yes, I didn't know out of that. And I don't think it's a, you must do it. But it kind of sets a, this is the Brown degree I'm looking for, if there's pushback, or there are other things that are coming into
00:23:18
Speaker
I agree with that, and I think if you hedge your
00:23:25
Speaker
feedback or your request, it can actually come across as a suggestion, not an ask or directive. So I agree with you, Doug. I think that we need to be more direct about how we make declarations at work. I think they made all this stuff up. Oh wow, the truth comes out! So we're really losing faith in this report, huh?
00:23:47
Speaker
They have quotes in it, and I'm like, oh, did they actually pull this from actual data they had? And I'm like, it's so poorly written. I actually, oh gosh. I don't know. It makes you wonder what their background is, the people who were responsible for making this report. Yeah, I do think the founders of that team is a PhD in linguistics or something like that. That's why I'm not too worried about using the statistic technique they mentioned in the report.
00:24:17
Speaker
I think it's the angle how you analyze data and interpret the data is usually where I see their firm can use IOs or social psychologists to help them to interpret the data. Because those are absolutely interesting. I just don't know if we would interpret the data in the same way. And oftentimes, our value added is that interpretation piece.
00:24:42
Speaker
If three PhDs are confused about this report, imagine what it's like reading this report as someone who, you know, maybe you're really smart or like you have an MBA or whatever, but you don't have that technical lens to kind of question some of their findings. Like what's going to happen?
00:24:59
Speaker
For their credit, they do cite actual studies that were done. So for example, they cite like this 2022 study that was done out of Cornell. And that was actually, I download the article and it was published in like JPSB, I think. So I think that that lends credibility.
00:25:14
Speaker
This is actually really interesting to take a look at what the potential difference makers would be. And for them to dive in, I thought that was actually quite innovative, I think, for them to come back at this and be a little bit clearer on their own methodology and how they broke certain things down and be clearer on their constructs. Because I think there could be something there. I don't want to discount it completely. But I think, like Jenny, you said, I think things could be a lot cleaner if they do it maybe potentially a third time around.
00:25:43
Speaker
And I know that this could be a white paper, this could be a business report. It also could be a case study that they use to present to clients as well too, to get them on another call to take a look at their platform and all that. I think it's a good, it's a decent start. I think they can try to dive, like if they have this data, it's an opportunity for, if they have data scientists and social scientists to come together and try to make something of this, improve it on the things that we mentioned throughout
00:26:12
Speaker
this podcast. And to add to your point, Doug, you know, a lot of vendors or solution providers don't even have like an appendix or where they kind of explain their methodology. And these guys, they actually do. So they have a survey methodology. They kind of describe like what statistical analysis that they use. And they did say that they used logistic regression with propensity score and covariate. So they did control for a lot of things. So they don't say what they control for, but
00:26:42
Speaker
They did make an effort to at least have somewhat of a methodology section, which I commend them. But we all agree, they could have
Effective Feedback Strategies
00:26:51
Speaker
done more. So I guess this brings us to the so what. If I'm a manager, I see this report, and I'm like, OK, well, Asians are described as intelligent and brilliant. Great. There's not much I need to do for them, other than, I guess, be more specific about how to get promoted.
00:27:09
Speaker
I mean, what do you think are the main takeaways that someone who maybe does not have our background might take away from this? I would be like more mindful about describing their work and their behavior and not the person.
00:27:24
Speaker
So don't talk about this person being ambitious, aggressive, just talk about the work that they do and try and do that for whether you're managing a white employee, a black employee, an Asian employee. And I think potentially, and this includes myself, like maybe that will start to uncover biases and words that you tend to use for certain groups.
00:27:46
Speaker
I would say I'm with you in that I don't think terms like brilliant and genius that show up on my performance review will actually help me out that much. For me, what I've done in the past is I actually write logs of what this person who was reporting it to me did right and how I either reinforced it or if something this person did wrong and I had to fix it and how this person continued to do it right and then write about that.
00:28:15
Speaker
It became like from every other week to like a month and then I kind of had those stories that I had ready when the time for the performance appraisal come around. I actually had tangible examples of things that I wanted to continue to happen, things that I didn't want to happen and where I see this person's growth going should they continue and whether they want to have
00:28:41
Speaker
conversations in that, or if that's not happening in their intuition at all, we stopped the conversation right there. But for me, it became like biweekly logs and monthly logs of people's behaviors in the workplace.
00:28:56
Speaker
you know, and potentially things that they can do that I know of, and it may not happen, granted as a start-up, and always a lot of things were up in the air. I also let that person know that as well, too, because it's very real that, you know, things can just all hit the fan and it all goes to shambles, I think.
00:29:17
Speaker
folks who are in management, I would say take good notes on your subordinates and give them good feedback. I think that's a really good actionable item that anybody can start doing. Start taking notes. It doesn't have to be a dissertation. It doesn't have to be long. But just write down specific examples so it's easier to give feedback to your direct reports when you have examples on things that they did well and then things that could be improved.
00:29:47
Speaker
Like actually when we do interviews too, when we take notes on interviews, it's like, you know, we try to take it on behaviors that people exhibit and not necessarily how they did it, whether they were like frustrated or whatnot, because sometimes that does come through, but it's one way for us to even
00:30:05
Speaker
you know anchor ourselves because we know the biases are there but that's that loser interviews is totally different from performance appraisals but what not but it's it's good to be able to write it down and then come back later and say oh this is what this person did this week and it was great because it made all these other things happen and i think people here like oh you remember yes
00:30:31
Speaker
Okay, thank you, Echo, for putting in the prompts for our chat GPT haiku. Opinions, color. Feedback, echoes, failed biases. Truth, hue, often lost.
Conclusion with Haiku on Episode Themes
00:30:51
Speaker
Aww. Yeah, simple and elegant. I like it.
00:30:56
Speaker
I love it. I love it. Well, listeners, thank you for joining us on another episode of Hidden in Plain Sight. We'll see you next time. Bye. Bye.