Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
MBTI, is it any good?
60 Plays11 months ago
In this episode, we talk about the ins and outs of assessments, including the MBTI and how they are used in the workplace. Tune in to hear about the research, our thoughts, and the haiku!
Articles for Nerds!
How The Myers-Briggs Personality Test Began In A Mother's Living Room Lab
Persona Documentary Trailer
https://www.youtube.com/watch?v=XWBXniurrA0
'If They Have It, It's Undeniable': Lorne Michaels On Casting 'SNL'
Leadership perceptions as a function of race–occupation fit: The case of Asian Americans. (Sy et al., 2010)
https://psycnet.apa.org/record/2010-16991-001
An interview Professor Thomas Sy
https://www.npr.org/templates/story/story.php?storyId=135513478
How Women End Up on the “Glass Cliff”
https://hbr.org/2011/01/how-women-end-up-on-the-glass-cliff
Differential Item Functioning by Sex and Race in the Hogan Personality Inventory. (Sheppard et al., 2006)
Recommended
Transcript
Introduction to MBTI in Workplaces
00:00:11
Speaker
Hi, listeners. Welcome to another episode of Healing in Plain Side. I'm Echo. I'm Joke. And I'm Jenny. So in this episode, we're going to be talking about assessment and how we can use our assessment at work. Awesome. Great assessments. I have a question,
Personal Experiences with MBTI
00:00:30
Speaker
guys. What's your MBTI?
00:00:32
Speaker
I can go first. So I've taken it for fun a couple of times and I get ENFP sometimes. ENFP. Yeah, so the E is extrovert. Gosh, what is the N? Intuition. Intuition. F is for feeling.
00:00:54
Speaker
P is perceiver. And for me, sometimes I actually switch between a P and a J. And I think a J is, I was gonna say judgmental judgment. Yes, thank you, Echo. I am judgmental too. What about you, Echo?
00:01:16
Speaker
Um, I actually had the biggest switch over I and E, which is quite interesting because, um, I took it multiple times like you guys. And, um, I also took it in the English settings where I'm here, like in the States. And I took it in the Chinese version where I, uh, that was like my life before coming to the States. And I think in the Chinese version that I took, I was a E.
00:01:44
Speaker
which is extrovert that I'm getting the energy being hang out with people. And I know, I wish I'm an East American, you can see everyone's facial expression. I don't believe it either in those days, especially after pandemic. Nope. Don't want to see anyone. But it has been I after this crazy
00:02:12
Speaker
years. I would say like ever since like I taking the I think I took it when I was in year one in the master program and I take it after I graduated and in the work settings, it has always been I ever since. And then I think it's either E-N-T-J or I-N-T-J, but it has always been N-T-J that has been very stable for me.
00:02:39
Speaker
All right, so it's reliable for you. And we'll talk about what reliable means. I've always been an ISTJ. I've taken the free versions. I've taken the actual paid versions, always been an ISTJ.
00:02:52
Speaker
Huh, interesting. So
Critique of MBTI in Recruitment
00:02:54
Speaker
that's a good segue into today's topic, which is about assessments, right? Yes, exactly. Just naturally as psychologists, we all wanted to categorize people in certain buckets and trying to predict their behavior, patterns, things like that. But I know it can be really irritating when someone talking about a bad assessment and thinking about this documentary that
00:03:22
Speaker
went out back in like 2021, the title for that documentary, I think it's called Persona. It's on HBO Max in that episode where they talked about MBTI as assessment. The important part in that documentary was they have been talking about how some of the companies started to using MBTI, which is unvalidated
00:03:46
Speaker
assessment tool and has gotten some popularity among companies using for their selection process. And what come out is some of the candidates got reject because some of the traits are not considered being fed into the job that required for. When this documentary just came out, there were some heated debate on like, how should we using the assessments? What type of assessment that we should be looking at?
00:04:16
Speaker
how the assessment should be properly validated before we can start to make for an employee decision. And I thought one thing that came out in that documentary was super interesting. And this goes back to that validity topic. When we are doing selections for those employment posts,
00:04:43
Speaker
we wanted to see what type of person, what type of people that come into recruiting process are best candidates that might demonstrate future performance. So we want
MBTI and Job Performance
00:04:56
Speaker
this, so one of the criteria that we will set for validate the validity of the measures or the assessment is the job performance. So in that case, we wanted to see the assessment that they gave it to the candidates
00:05:13
Speaker
have a strong correlation to the job performance. But I think in the documentary, what he says is it's actually demonstrated very weak correlation between the MBTI assessments and job performance. And therefore it has sparked some of the question on like, why do they even start using the personality tests for those things? Because also at the same time, like personalities, regardless
00:05:42
Speaker
more of a stable, that's something like not easily too maneuverable and people just feel like this is more of a fixed versus something that it can develop or over time.
00:05:54
Speaker
So I think that's some of the controversy that I have heard or seen from that documentary. I would say to get a sense of people's personality when you're working with them to know what their style is like. I think that allows you to kind of feel each other out and whatnot in the workplace.
00:06:13
Speaker
But in terms of the MBTI being used for, I think, a form of assessment to predict job performance, I'm not sure how well it does that.
00:06:32
Speaker
I do work in, I do hire folks in my previous role, in my current role right now. And what we try to do is we have assessments, we make sure that's related to the job, very much so in terms of content so that there isn't a possibility or there's limited possibilities that people would take us to court and sue us. I mean, that's one of the things that we worry about.
00:07:01
Speaker
probably why we would probably never use the MBTI in selecting folks. So have you ever been on a job where they've used the MBTI for selection? I haven't. Not me neither. Yeah, I haven't seen any company that personally that I interview for or that I'm aware of who was using it.
00:07:29
Speaker
But even in that documentary, I recall, I think a lot of the companies or the manufacturer were using it with blue corner type of roles. So I think one example they used was like a Kroger supermarket. They were trying to recruit for cashier or some of the warehouse workers. And I found that quite interesting because I know some companies also spend
00:07:56
Speaker
hundreds of thousands of dollars every year to those assessment companies trying to buy their assessment center and ask them to design and develop a certain assessment for their specific jobs. But MPTI is right there. There are so many versions of different tools that are already out there in the market and some people can
00:08:24
Speaker
just use it. So in this documentary, were they using the MBTI as part of the assessment process? Yeah, so I think what is, that's why the, I think the documentary was describing this as like a dark side of the personality test. So they were actually using MBTI to recruit and to select some of the victims that are coming to the screen and talking about their experiences, how they get discriminated because
00:08:53
Speaker
they were assessed for a certain type of personality that was not considered as suitable for the job that they are trying for.
Ethical Concerns in Hiring Practices
00:09:03
Speaker
So there's a whole set of this like ethical questions that has arise and even like all the criticisms about like how they should be using assessment, how they should be using personality assessment in particular, because in some of the settings that personality test or the personality itself is considered as
00:09:23
Speaker
more of a stable, unchangeable, unmanipable trades versus when you're in the work settings, there's always room for grow and develop. So that has been like called out as one big thing. The other thing they have been called out in that documentary is MPTI assessment itself has always been showing very poor
00:09:49
Speaker
correlation to job performance. So that actually casts the same question is why do those companies will use this for selecting the candidates because clearly this assessment doesn't indicate any of the future performance for their employees. So they started to question like why they started using those. And the third one is I personally haven't dug into this like too much.
00:10:16
Speaker
But apparently they were mentioning that BTI was kind of derived from Carl Jung's theory, which some part of it has been outdated and has shown some of the racial discriminations. I'm not personally very familiar with that part of the history, but apparently it just gets on the nerves for some of the psychiatrists and some people who knows this matter.
00:10:42
Speaker
So that's another thing that they called out is how inherently that this is racial discrimination for some people. Echo, does that follow the Jungian archetypes? I'm actually not quite sure which they're referring to on that part of the story, but I think in the documentary itself that I've been talking about is
00:11:09
Speaker
Korean theory itself. I asked that because like there's also like his work is kind of based on stereotypes as well too and I think there's in a way like I think even more Michaels as I think the
00:11:23
Speaker
something with Saturday Night Live. He's an executive director of Saturday Night Live. He hires people. He hires them based on these like union types of like, who's funny, who's not. And I think there's an interview of him like saying that he
00:11:40
Speaker
uses archetypes in a way like stereotypes. It's code for stereotypes, just another way of saying stereotypes. When I heard that, it was so frustrating to me because I'm like, Lauren Michaels, you're saying you're using stereotypes of people.
00:11:59
Speaker
to, in a way, assess their sense of humor. If their sense of humor matches with yours or other folks that you deem to be funny, then you bring them in. But you may not bring them in because you don't see this other forms of humor.
00:12:16
Speaker
For me, if I tie that to, if we tie that from like, from like senses of humor to like even leadership, we know based on other studies, previous studies, that there are different perceptions of leadership that white candidates have, black candidates have, versus Asian candidates. And I guess that's the thing that,
00:12:46
Speaker
I mean, I think that's a great example because like the lone Michael's example, I think he just vocalized or said what a lot of people do in their job and when they are looking for candidates, right? Yeah. Yeah. And you mentioned there just totally remind me of the article that we read.
00:13:09
Speaker
Um, years ago, um, it's done by Sayadaw. Um, but then, I mean, like I was trying to recall a few other authors in that. Yeah. Ning Shore, I think. I think we should just go back to that article. So the article was about archetypes that people have in their head about who's a leader and who's not.
00:13:29
Speaker
correct. And in that case, that people have been seeing that and in their study, they look at like a for Asian Americans and for white Americans, and they look at them in both engineering and sales occupations. Okay, so engineering and sales, assuming that engineering role would be more archetypically matched to Asians versus
00:13:56
Speaker
sales would be more matched to what we consider white leaders. Is that right? Exactly. Okay. And what they found was Asians don't match with sales, but they match with engineering, but white people. Yes. Yes. And, and also in that case that they also seen like
00:14:15
Speaker
when people are actually being viewed as leaders in those roles, they see white people are demonstrating like dynamic or tyrannic attributes, meaning that there are dynamic and they see them sometimes can be intimidating to other people, whereas like they see this like very differently for Asians, which they can also see like people just having a different
00:14:45
Speaker
archetype in mind where like Asians should be looked like when they were actually seen as like leaders. And when you say tyrannical that means like agentic like go-getter like sort of like a tyrant but in a leader form? Uh yeah and some of them um I think the the the word exact word they use is like a tyranny and so you can also imagine like agentic sometimes even
00:15:12
Speaker
It depends on the context. Yeah, I just want a white tyrant as my leader. Why is that fair? That's not fair. Why can't Asians be tyrannical too? Oh, and Asians can't be tyrannical. But it sounds like from what Echo was saying is that Asians have to come off as something else, right, to become viewed as leader-like.
00:15:43
Speaker
Yeah, in that case that they only see Asians as leaders when they are competent and they are dedicated to certain things. In this case, I mean, versus like why they don't, they don't have to be competent in order to be a leader.
00:16:01
Speaker
And I remember from that study, they said that this is bad for Asians because when you are a leader, it's not just about having the competence like an engineer. You also need the skills of a salesman. You also need that interpersonal, that, I don't know, je ne sais quoi that people have that goes beyond your, can you code? Can you get this done? You have to have that extra thing and people just don't think that Asians have that
00:16:30
Speaker
extra thing that white leaders have. And so this prevents them from getting promoted. Is that right? Yes. And in certain points is like, because also people already have that pre-assumption that you're not very good at those social skills. So they will automatically just put you in that lower rank on like, oh, those people can work. They are excellent workers, but they never be able to set up for the leadership role.
00:16:54
Speaker
because of those pre-assumed thinking they already have the mind for Asians. I could definitely see the MBTI. Well, Dr. I was going to throw some fuel on the fire, Franco.
00:17:15
Speaker
Well, what's the problem? I mean, you get like Asians got like, they're like, you know, they, they're in these good jobs. And they're, they're out there. Like, why, why worried about leadership? Like, they have secured
00:17:35
Speaker
Oh, you're plenty devil's advocate. So they're, they're overrepresented in STEM. There are, there have these like high paying, you know, engineer jobs. They're good, right? Who cares, right? I mean, I think that's a very good point.
00:17:46
Speaker
a very good point and then I think a lot of people think that way but why does it matter like why do we care? Well I think in the same way as we care if women can also have a space in the leadership roles there's exactly the reason why we also care why black and Latino population also have a space in the leadership role because when you have a system that is
00:18:10
Speaker
unevenly distributed in your workplace for a certain race, that itself is a workforce issue that you're facing. Why Asians can only be overly reprinted at the bottom, I think there is a systematic issue that they were facing when they are not being able to reach through that leadership roles. So our audience can't see this, but I had to clean my tongue a little bit because I felt a little dirty just even saying that. Why worry about this?
00:18:40
Speaker
No, but I think that's a very good point because a lot of people think that way. Like, why are you complaining? Like, shut up. Like, let other people have the mic. But I think for me, it goes back to representation. And if I see an Asian leader at, you know, a company where I work at, then I think, oh, this is a company that I could go with. Like, I could see myself in that person. But when you don't see that, you're sort of always wondering, like, you know, self doubting yourself and wondering, are we even fit for this job? Yeah.
00:19:09
Speaker
And slightly like a slightly different
Stereotypes and Asian Leaders
00:19:13
Speaker
topic. And when someone, some reason like they, they were having bad performance as a leader for Asians, if they were in that role, that role didn't so bad.
00:19:28
Speaker
for this entire population, where for someone, like if they were white and they were in the leadership role, people were just saying, hey, this person just don't go that their job. But for Asian, it's very easy to sing as like, hey, that's because they're Asians. And they're just not very good at doing like leadership roles. And we've seen that happens over and over for women. And this could happen to Asians too.
00:19:55
Speaker
Oh, yeah. So are you talking about the glass cliff ceiling that they tend to put like woman or a person of color in charge knowing that it's going to fail? And when it does fail, they they blame the person rather than blaming other circumstances. That's one. And the second piece is like if where people not very good at their job as a leader.
00:20:18
Speaker
I think it comes to the fundamental attribution, where they will very easily attribute this to individual ability not be able to do that job well to some white people. But they can very easily attribute this to an entire Asian as a group, as they are not very good at doing their job as a leader. Yeah, so the stress to prove, that's a lot of weight to carry for one person. Yeah.
00:20:49
Speaker
Actually, it kind of reminds me of when Barack Obama became president. You all remember when he became president in 2008, when we had that downturn in the economy and the economy slash the stock market was going to crash. And I think some of the things I heard in the background was like, yep.
00:21:14
Speaker
He's getting this job the worst job available to this black man to take this on. And I think he did a great job of getting us out of that. But it could have been, the other outcome could have been well. See, he couldn't have done the job because of XYZ or whatever. And it's kind of frustrating when
00:21:43
Speaker
those attributions or those stereotypes can be put upon other people. That negative stereotype of not being social enough, not being leader-like is put on that person before that you can get a chance to showcase what they can do. Yeah.
00:22:01
Speaker
It's kind of frustrating to see that sometimes people don't get a chance. And guess what? You're going to have white leaders that are going to be incompetent and not always hit the mark or perform right away. And that's going to happen with other folks as well, too. So why aren't we giving them a chance?
00:22:21
Speaker
Yeah. And I was going to say this earlier on, but at the MBTI, I feel like where we would potentially get penalized, like Asians in general is the E and the I part, the extra versus the introverted, because the assumption is Asians are introverted. They don't have the interpersonal skills to be a leader.
00:22:43
Speaker
So I could see how something like that could potentially be used in the wrong hands. Could negatively affect, you know. We all started to advertise ourselves as like an E.
00:22:55
Speaker
So I will no longer switch back to the Chinese version of me. It's just me. Yeah, I am the extrovert in any occasions. And I can also, perhaps, like, also argue in the same case for the J, where they can see, like, Asian's lack of emotions and, like,
00:23:18
Speaker
lack of social ability, where you're being a J versus P, that could allude to, hey, this person is just using the logic every single time and they're not able to relate to people. So that could also potentially have some of the drawbacks when Asians are being categorized as J versus P.
00:23:44
Speaker
You know, what's really surprising is the fact that prior to a call, I did a quick search to look at like, for example, the Hogan, the MBTI, just looking at racial differences. And surprisingly, I wasn't able to find much. There was one article that looked at white and black participants who took the Hogan. The Hogan is another leadership assessment and it's used widely in the corporate world, but they did find racial differences between how
00:24:10
Speaker
white and black respondents answered a question, but this was in 2006, I want to say. Again, we'll put this in the description.
00:24:19
Speaker
But it's just kind of interesting that there hasn't been a lot of studies kind of validating these assessments. Well, all those assessments too will say they have been validated by so many times and which I think part of it is like a commercial reasons because they do sell assessments for profit.
00:24:41
Speaker
And even like our dearest colleague in our programs graduated like years ago, Naomi, she actually also got the data set from the Hogan assessments and she also find there are some gender difference. How women take the assessments are different from men.
00:25:04
Speaker
And so like how do we also thinking about from that regard in relation to their leadership potentials in the organization? Yeah, I think one question on the Hogan is I like to drive fast cars and that's such a masculine stereotypes question. Actually, I will probably say yes, I really like to drive racing cars, even though I don't have a car. But I can totally say like how some item itself
00:25:32
Speaker
can be gender biased in a way that is hidden to some people. So I realize we talked about validity and reliability. I think
Reliability and Validity of Assessments
00:25:44
Speaker
we should define that, right? Dirk, this is your area of expertise. You want to define what reliability and validity mean for
00:25:51
Speaker
Yeah, the non IO. Oh my gosh. Oh, they have to be non IO. All right, let me try to simplify it. So reliability is the consistency of your measure. And when we talk about validity, it's really about we want to know whether you're
00:26:10
Speaker
if you're trying to measure leadership, if that measure is actually tapping into that. So there are special forms of that and forms of content in the items that you put out there. And I think Echo and Jenny, you gave a great example of poor content in that I'd like to drive fast cars. And that could be something that could be more men would like to do than women, but does it really actually differentiate leadership?
00:26:39
Speaker
We don't know. Maybe not. And we're leaning on the no on that. And I would say, I don't want to drive fast cars because I'd rather have someone else drive me.
00:26:51
Speaker
So we want to make sure that those items are valid. And usually with a content validation, you have experts come in and actually judge those items on clarity and relevance. And in terms of the measure itself, when we go through an assessment process, what we want to make sure that what it does is that it's related to other measures. If you have a measure of leadership that is related to or highly correlated with
00:27:19
Speaker
another talent development or leadership scale as well too. One that's been previously validated, so we know that it's highly related to that. It's tapping into what we would say in the psychological world, a similar construct. Of course, we wanted also to not measure certain things that we don't want it to measure. We want it to be dissimilar. Maybe we want it to be different from
00:27:47
Speaker
a impression management measure or even an eighth grade math exam or whatever it may be, something that's totally different from it. So in a way, those are reliability and validity in a nutshell. Thank you, Doug. And I think we actually had that example on some of that.
00:28:11
Speaker
reliability look like like in my case that it actually flipped from I from E to I but I guess like we could argue because the environment has changed or the context and change but still like it really depends on like where this person take the test and how they
00:28:31
Speaker
look at the environment around them, all of that will also affect the results itself. So using that for employment decisions, that's a very big decision that could affect someone's life being. So I think that that's just something that we should always be very cautious about.
00:28:52
Speaker
Right. Add something real quick. And this is like something that was just like really like pounded into me in my master's program in northern Iowa is that validity is like a it's a process and not an end goal. Right. You can't just validate it once and that's it. Like you have to continuously add to it and more data to it.
00:29:15
Speaker
in a way like we become psychometricians when we do that. We want to add more data to it, but it's actually in a way a good thing to add more folks who are in leadership who have not been previously in leadership roles to showcase like, hey, there are other forms of leadership. You don't have to just drive fast cars to be recognized as a leader.
00:29:39
Speaker
And I would say in the work that I do right now, you know, the assessments that we have, we make sure we revise that with our subject matter experts each time you run a test. So we advise a, you know, the written exam they have, the interview that they have.
00:30:03
Speaker
And sometimes even a practical test that they have to do to showcase that they have the skill set in order to do that to make sure that it's up to date with the job that they're actually doing versus what it was five, 10 years ago, because your job actually can change. So if your organization is using the same scale or something that's really old and outdated from decades ago, you may want to revisit that or ask your local,
00:30:30
Speaker
organizational psychologists within your organization when the last time they revisited or revamped it because some things could be out of out of date or just not or not used properly and if that's the case
00:30:46
Speaker
your organization now is prone for or has a potential for there's potential for litigation against your organization for using poor practices that are laid out by both the government and SIOP that we have and we'll link those in the description to have your selection process updated.
00:31:12
Speaker
who's listening, who has people running studies and assessments have industrial organizational psychologists working for them. If you got folks, gosh, I know this came up last time we spoke ECHO, and I think Genevieve may be a part of it as well too, is that I think we said like sometimes in organizations we have like these employee experience surveys that are not
00:31:43
Speaker
that are ran or created by non-psychologists and they wonder why certain things don't work or certain things don't pan out or why data don't come out and they come to us last minute and ask us to fix it and we're like, nope, we can't because there's a whole process to this and we can't fix what you just, you know, garbage in, garbage out. You're not going to get anything out of us for this.
Appropriate Use of MBTI
00:32:07
Speaker
You can't bypass the process. So I know we've really thrown the MBTI and the concepts of assessments under the bus, but I think we want to leave on a positive note and say,
00:32:22
Speaker
You know, the MBTI is not complete bullshit. It's, it can be used for developmental purposes. It can be used to understand how your team and you think, but what we're saying is it should be used with caution. Never, it should never be used for selection, never be used to predict performance. That's how we're going to close out the. I think Echo has a haiku for us.
00:32:46
Speaker
Oh, nice. As you know, we have the haiku from chat GPT. And for this episode, it right as three nine. Those are hidden judgments cast on validated tests deceive bias clouds the choice. I do think they actually those three lines capture some of the essence
00:33:11
Speaker
from this episode on how we should be using assessments and how we should be using validated assessments to eliminate some of the potential from discrimination or bias.
00:33:25
Speaker
that we see in those decisions. And what was the prompt that you used for this? Yes, so the prompt that they used is, write a high-cool for implicit bias in unvalidated assessment in selection. I have to say, I did make some twist because my first version was just having chargeability to write a high-cool for running assessment. And it was like a very plain one. So I started to
00:33:52
Speaker
intentionally call out some of the pitfalls that people might have using assessments. That's why the final version of it does seem like addressing some of the elements that we cared about. So that's the trick. Yeah, we're making chat GPT smarter with each and every episode. Whenever we ask them, we ask it to give us a prompt, right? Yeah. To give us a haiku.
00:34:17
Speaker
All right, so this is just a tip of the iceberg. We could probably spend an entire podcast, an entire podcast just dedicated to talking about assessments, but this was just like an introduction. We'll probably revisit this topic down the line again, but hopefully you guys liked listening to this. So tune in for our next episode. We'll catch you later. Bye. Bye, everyone.
00:34:50
Speaker
Bye!