Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Episode 3: Good interviewing
62 Plays16 days ago
Sam and Steve do a deep dive into interviewing. What makes a good interviewer even better? What's the difference between a journalist and a researcher? Two expert interviewers interview each other about interviewing!
Recommended
Transcript
Introduction to 'Off the Path' Podcast
00:00:14
Speaker
Welcome to Off the Path, a show where we talk about research careers, especially our own. I'm Sam Ladner. And I'm Steve Portigal. Sam and I have been having a bunch of conversations about research careers, and with Off the Path, we're going to share those conversations with you.
Career Paths and Interview Challenges
00:00:30
Speaker
When I was thinking about this conversation, i was thinking about how often i have been in an interview where it goes come completely off where it was supposed to go.
00:00:42
Speaker
And i was thinking it's kind of a good metaphor for a career. When you think about interviewing, which I'm sure you've thought about it more than 98.999% of people in the world, what comes to mind when I say that interview went off the rails?
00:00:59
Speaker
What does that make you think? This is the point, I think, in all these conversations where we, and I mean me, get slightly pedantic about the language. And then we come up with, what I think, what we're trying to get at. This is off the path. And we've talked about off the path as a metaphor and what that means. Whether we've talked about that on the record or just talked about it and we're kind of planning this.
00:01:19
Speaker
Off the path and off the rails are really different. Indeed. And off the rails is sort of alarmist language that says, ah everything went to shit. Right. Right. I wrote a whole book of other researchers' stories about when things go to shit. I mean, that's not all the stories that people
Insights from Unusual Interviews
00:01:36
Speaker
had to share. um So that is very dramatic when things go off the rails that suggests...
00:01:42
Speaker
this person is insane or this person is racist or ah violently misogynist. Or girl, you're a danger. Yes. Yeah. Literally danger. Yes. Yes.
00:01:55
Speaker
Or your client is or your stakeholder is behaving in a way that's really messing things up. So we can talk about that, but we can also talk about off the path.
00:02:06
Speaker
Well, then what's an off the path interview then instead of an off the rails alarmist crazy interview? You're talking about off the path. What's that? And I think you sort of said at some point, right, ah different than how we thought it was going to go, different than what we expected.
00:02:19
Speaker
And that to me is really interesting because it gets at, well, what did you expect and why did you expect that? And maybe that expectation was wrong. Like there's a whole gray zone, I guess, around here's what I thought was going to happen.
00:02:35
Speaker
here's what did happen. Cool. Cool. Right. Or, oh my God, oh my God, oh my God. But that's a panicked, oh, my God. I mean, yeah I think you can panic because if we're not getting the information that we thought we were going to get because we've hypothesized, we want to get people to explain something to us. And it turns out they can't explain it because they don't care about it or have never thought about it before.
00:03:00
Speaker
Sort of how wide is your aperture or how comfortable are you with the unknowns? Oh, yeah. It was a tiny little project, so I didn't get to go as far into the areas I wanted to, but I interviewed people about... Hard hats? Yeah.
00:03:15
Speaker
Like you wear at a construction site? Oh, okay. Hard hats. All right. The interviews were interesting, of course, as they always are, but this was an interesting interview that kind of failed because the client had a bunch of design and feature and performance questions.
00:03:32
Speaker
Okay. Why did the interview fail, exactly? I mean, and let's put fail in big air quotes, right? Sure. The interview was very challenging because the person didn't want to talk about hard hats.
00:03:44
Speaker
You know, I say that, but I think we can all recognize that moment where we're putting someone into a category by associating them with a set of behaviors or references that don't match their identity.
00:03:58
Speaker
And so. That's a big, like, that's a meta statement right there, but keep going. And we've probably heard those responses. Like, it feels like the interview is, it's uncomfortable. And the person says, I don't know why you're asking that.
00:04:10
Speaker
Or they laugh or like, I don't know if I'm the person to kind of tell you about that. Like, it's hard to get into a, I have a question, give me your answer, tell me a story. Like, it's hard to get to that because they're bobbing and weaving about the premise of the interview.
00:04:25
Speaker
Yes, exactly. And this is a person who's qualified user with screen. Like we do all the right sort of operational things to get there. when we get there, even though they've agreed, they're still like, wait, why are we talking about?
00:04:37
Speaker
Hard hats. Right. and It could be any topic at all. i think you sometimes find this gap between. what you assume sort of the customer user's feelings about themselves in relation to the product or the behavior or the category, whatever that is. And by pointing to it and pointing to them, now they're like, well, wait, I'm not a this. I'm not a hard hat wearer.
Emotions and Challenges in Interviewing
00:05:01
Speaker
So anyway, I think the gap here was, and then the insight, which was very indirect, was that PPE, personal yeah protective equipment, uh, In fact, if you look at some of the design trends, this is already old hat, but they were starting to make protective eyewear that looked like, I think Oakley was like a sort of yeah fashion brand, sort of swoopy looking. yeah
00:05:24
Speaker
And when you see foot protection sort of become mainstream fashion, yeah there's these sort of moves between what you should be wearing to be a normal person and what you wear in this sort of job site. and so I got stories about how they felt, you know, they're on the job site and they like go to a restaurant to get lunch.
00:05:43
Speaker
So you leave the site and you've got all this stuff on and you look just weird, right? Not normal in the sort of sociological sense of of norms. yeah um And so all the gear had to work in safety context, but it also had to work off stage or I guess on stage, right? You go out in public and yeah and it had to make these transitions.
00:06:06
Speaker
You know, I did get some of that eventually, but the discomfort in even talking about it was related to that because I'm a civilian talking to someone about their PPE and i kind of highlighted the discomfort that I think these folks had in moving between those two zones. And I was on the outside talking to him about this inside behavior and it was it was uncomfortable.
00:06:30
Speaker
you know, I'm grossly oversimplifying a lot of nuance here, but the interviews were hard to have because the person didn't want to talk about it and I didn't get that. But the interviews were super revelatory because, you know, as happens in research, like the opportunity team is seeking is different than maybe how it was framed. i mean, that's why we're doing the research.
00:06:54
Speaker
A hundred percent. Well, the opportunity is different than what we thought we were going to get in the actual established interview. And that is an off-the-path instance, not an off-the-rails instance. yeah I think it takes a lot of, like, emotional work and training and perseverance to know the difference between that in the live interviews, right?
00:07:17
Speaker
I've had instances... where I've had interviews go off the path, in the moment I'm thinking, this is off the rails, right? Like, this is completely off the rails. Like, a good example is I'm interviewing a person who's supposed to be reading an English language newspaper at least three times a week.
00:07:35
Speaker
They did all the screening questions, whatever, and it is for a newspaper. The study is for a newspaper. And it's clear to me 20 minutes in that this person doesn't even really speak English, much less read a newspaper.
00:07:47
Speaker
And they have very little opinions about newspapers. There's no newspapers in their house. There's nothing happening. Like, they don't know anything about the newspaper that I'm interviewing for, much less any other newspaper. And I'm thinking, okay, this is bad.
00:08:01
Speaker
Now, that I thought was a bit, frankly, off the rails. But in that same study, I had a different interviewee. And they had, like, the physical newspaper, the newspaper I was studying, the client's. And they had it open and it was on the table right in front of them. And they're like, yeah, like, look at this. And it was a two-page spread of all politicians in a certain election that was happening. And they had their photos and they had like little blurbs about them and like some quotes and stuff.
00:08:27
Speaker
And it was supposed to be like a voter guide or something like that. And she's like, like, look at this. And I was like, Oh, no, this is like she doesn't read this newspaper. She doesn't like news. Oh, God. And she starts talking about how alienated she felt from this, how she felt like this didn't represent her. None of these people on this paper look like me. They don't talk like me. They don't have the same concerns as me, blah, blah, blah. And I was like, oh, wait a minute.
00:08:50
Speaker
This is off the path because it's not what I was thinking we were going to get. But this is very good stuff. Right. You have to know the difference between those two things. Maybe you make a really good point that off the path can feel like off off the rails. Yeah. And so it becomes a lot of what do you do about it?
00:09:07
Speaker
What do you do as an interviewer tactically? But what also do you do for yourself as you're. Well, what do you do? What do you do when you feel like it's off the rails? You're like, OK, what do you do about it?
00:09:18
Speaker
At one level, I'm doing the same things that I always do, which are reflecting back, coming up with useful follow-up questions to try to get at what that person's trying to tell you.
00:09:29
Speaker
Just practicing the listening activities of a good interviewer and... and thinking ahead to what I might get to to talk about, but also trying to stay, if not in the exact moment of present, like not seven steps ahead, but one step ahead.
00:09:44
Speaker
But then part of the brain is like seven steps ahead. So that's all sort of cognitively demanding and a little bit emotionally demanding. A little bit. I mean little bit. But I think every interview has some of that. You know, the person that won't talk very much.
00:09:58
Speaker
Yeah. It takes longer to warm up. Yeah. And then i think, my God, I'm going to get through this whole interview in 20 minutes and my client's going to wonder why did I do such a short interview and not get anywhere like that? How many interviews do you think you've done, like conservatively? Hundreds, I guess.
00:10:16
Speaker
Hundreds? Yeah. I mean, for sure, nowhere near as many as many researchers that we know. Just given the nature of what my job is. mean, you see these like in-house teams where they just put up these stats of like, oh, we did.
00:10:29
Speaker
oh fair enough. Yeah, fair enough. I mean, sometimes the demand is there and they have people set up and they're just running studies and running studies and running studies. Well, the reason I ask that is because when you said, i mean, it's cognitively intense being able to think in the moment and then two steps ahead and then maybe like subconsciously seven steps ahead.
00:10:50
Speaker
That's cognitively difficult. But you said, and it's a little emotionally difficult, just a little. And I'm thinking to myself, what do you mean a little? I mean, it's always a little like the floor is emotionally difficult.
00:11:05
Speaker
there's no There's no interview. That's why I'm saying it's what I always do, which is partly worry, right? Like, how is this going to go? ah people going to be happy with this? Am I getting it? Am I doing something wrong? Like, there is a lot of that voice in my head that, yeah, I just have to give it a little place to sit in and chatter. And I think, yes, I've done a lot of interviews. I've also done interviews over a lot of years.
00:11:29
Speaker
Like, you learn through repetition, but you also learn through duration or tenure. Yes. Like, yes, it's probably going to be OK. It's probably. Chances are. And I know sort of how much of a leash to give myself.
00:11:41
Speaker
That's a weird mixed metaphor. But I mean, it's back to one of our topics here.
Embracing Ambiguity in Research
00:11:45
Speaker
I think I'm very comfortable, in fact, prefer off the path interviews. Really?
00:11:52
Speaker
if you want to hire someone to like hand them a guide and have them run that interview, don't hire me. i am i'm a little on the fence about that one myself, which is weird because like if somebody were to give me that interview guide and say, do exactly what we tell you, I'd be like, yeah, OK, thanks, whatever. No, thanks.
00:12:10
Speaker
At the same time, i mean, I'm with you. There is no interview that I have ever done. that I have not worried about in the moment beforehand, even after.
00:12:22
Speaker
Well, not so much after, but like definitely in the moment. The moment I'm in there and I'm like, okay, especially if there's a stakeholder with me, I'm asking the questions. It's patently obvious that they don't want to talk about hard hats or they don't want to do this or they don't want to do that.
00:12:35
Speaker
And yet I have that tenure you're talking about. You know, I've done hundreds of interviews. I might've even done thousands. Honestly, might've even done that many. And so I know I know we're going to get back onto the path that I originally thought in some form or fashion, but it never ceases to stop. I always am worried still.
00:12:54
Speaker
And I kind of wish I didn't have to be worried. And I kind of wish that I never had this norm, the path that I'm supposed to be on. i wish that didn't even exist, right? Like I wish I didn't even know about it so that I could just go in and have like a a conversation. With somebody and have them tell me stuff about themselves and give me a piece of their experience. I wish that's all I ever had to do.
00:13:18
Speaker
And yet that's not the job. That's not how it works. This might be surprising. It's surprising to hear me say it for myself. I kind of like that worry. Why?
00:13:29
Speaker
It's kind of energizing. Oh, really? Right. I remember learning in an undergraduate the stress performance curve, right? There's some, I think it's just a straight parabola, right? And so, yes you know, a certain amount of stress increases performance. Too much decreases performance. Right. like What do they call that? Eustress versus distress?
00:13:47
Speaker
Oh, wow. i think Am I right about that? I don't know I don't know. That's what I get for bringing up something I learned in school to you. don't you tell me who it's named after? and and oh Oh, I don't know that. Good question. Now I have to go look it up.
00:14:00
Speaker
We'll come back to that. Seems like a thing that you would know. It does feel like a thing that I would know, but I don't actually know. It's not named after anybody. I do know that. There is sort of an energizing feeling of the worry and there's a satisfaction for me in overcoming it.
00:14:15
Speaker
I'm not a booyah kind of person, but there is a feeling at the end of certainly first interview in a study or in a challenging interview. It was like yeah, that was a performance. And I guess the thing is, I have very, very, very few like being booed off stage, like things that you would call a failure. We're like, well, that was a waste of time. Well, that was awful. like Well, who's been booed off stage? Very few people.
00:14:37
Speaker
but Yes. Well, the thing that you're worried about happening doesn't often happen. And then and I think i think in most cases you don't get everything. It doesn't end up being perfect. But in most cases, it feels good. I mean, there are things you can't overcome like a bad recruit. right The person doesn't speak English. The person shouldn't be interviewing.
00:14:55
Speaker
And even there, there's always this gray area. Well, I didn't expect this person in my sample, but they ended up in my sample. And then this other thing happened and we're not going to build for them. But now we understand something different. now we Yes, we understand that this guy is not the target and he's still a human, though. And he's telling us something about who the target could be or should be.
00:15:17
Speaker
I understand what you're saying about the charge of bringing it home despite the challenge. Because if you look back on like choices that you might have made, like gambles, maybe you've made not just in interviews, but in your career, you've made general gambles and you're thinking, I don't know if I can pull this off. I really don't know if this is going to work. Oh, my God. And then you do somehow.
00:15:36
Speaker
You bring it home. Yes. That's the satisfaction. Yeah. And I hear you on the amount of worry and wishing that wasn't the case. I think that's very true for me and lots of gambles.
00:15:48
Speaker
and Not that I do lots of gambles, but if you sort of look at all of them, I have spent much more time worrying in an emotional way about the failure outcome and then having it go well and being good and not being able to say, oh, I worried and therefore it was good.
00:16:05
Speaker
I think in an interview, again, there's that baseline that I've gotten it down to where I do have this worry. First interview of a study or anytime I feel rusty, i definitely get, if not trembling, then just sort of complete physical unease in my body. Like just a lot of nervousness.
00:16:22
Speaker
Yeah, i I do the same. I do the same. I don't want that. Mm-hmm. But there is there is a uneasiness, edginess, sort of worry that's also attending to the impossible mechanics and outcomes and uncontrollable aspects of an interview that is enlivening.
00:16:40
Speaker
But in general, you know, those things in life, I think, can become too much. But in interviews, again, because I've been doing them for a long time, I know how much of that I kind of like and can kind of use. Yeah.
00:16:52
Speaker
Yeah, yeah. I have learned, i i guess maybe I should be a a little bit more circumspect about the whole thing. Like, ah I take a look at what I've learned through being, you know, a primarily qualitative researcher.
00:17:06
Speaker
I have found that there's so many things that you can't control. And there's so many places where it can go completely awry in live interviewing. And yet over and over and over again, i have shown to myself and to others that I can bring it home.
00:17:22
Speaker
I can rise to the occasion and I can get us back onto the path that they expected. Actually, sorry, not that they expected. A good path, but maybe not what they expected. Maybe not what I expected.
00:17:33
Speaker
So constantly putting yourself in that position where you're going to get those jitters. You're going to feel it in your body. You're going to go, wow, this is uncomfortable. I don't like this. I think there's a gift in there.
00:17:44
Speaker
There was a chance that I could have been a purely quantitative researcher and back to just a single choice in grad school where I was looking for a data set to do my dissertation. And I was sure i had a pretty good question. I was going to look at technology adoption. i was going to look at it in the nonprofit sector. I was going to look at how it changed, like this and that.
00:18:02
Speaker
And the reason I chose that is because it had all these theoretical things associated with it. And thought, these are really meaty questions. And The not-for-profit is kind of an interesting, like, context.
00:18:13
Speaker
And I also thought that there was good data that I could access, right? So, like, from a large data set from a statistical agency that I didn't have to collect myself. So, turns out, no, first of all, there isn't really good data. There was some data. And secondly, getting access to the crappy-ish, I couldn't even tell how good data was going to be hard.
00:18:34
Speaker
And I was like, I'd spent all this time, like, Learning, like I used SPSS at the time, but I learned SAS syntax in order to do this. And I was ready to write queries and do all this stuff. And I couldn't get freaking access to the quantitative data. And, you know, it was the certainty that I was so looking for.
00:18:53
Speaker
I was looking for this like crisp hypothesis and it was a yes or no. And it was going to be so easy, not easy, but like straightforward. And I was compelled, given that there was only so much time in my life, I was compelled to make a decision. Like, can you go forward now?
00:19:09
Speaker
Can you go forward with technology adoption of something with something else? Right? Right. So I chose a qualitative study and a design and the rest is history, is I guess, as they say. And I kind of like kicked myself for like years for a long period, to be perfectly honest, because I was like, why are you subjecting yourself to this ambiguity all the time? Every single study you do, you're not sure where it's going.
00:19:35
Speaker
You're dependent on these people talking about hard hats or not hard hats. Why did you do this to yourself? And yet, looking back, of course, there's a gift there. Definitely. Ambiguity.
00:19:46
Speaker
You got to learn how to deal with it. Well, you're making me think about sort of the layers of ambiguity because there's the ambiguity of the whole enterprise, the project. Yes. Like, I'm to go talk to tech owners inside nonprofits and try to understand their motivations and see what patterns I can draw from that and see if there's conclusions and implications. Well, that feels shaky hu because there is no path yet for that. You're trying to create that path. So we're not on or off. We're just pushing a lawnmower in front of us or whatever. Oh, God.
00:20:21
Speaker
and yeah I'm thinking of a machete. Yes, we're hacking through the underbrush. Yes. And so the whole enterprise is ambiguous. And then the method, we're talking about the ambiguity, the uncertainty and the worry of the method. But yeah, I got to go talk to somebody about hard hats and they don't really want to talk to me about that.
00:20:39
Speaker
I am also in an uncertain position because I've just told somebody who thinks that the hard hat problem is and what they thought it was, was just a little cuckoo. I have to help them and I'm uncertain about that.
00:20:51
Speaker
And now I'm talking to someone that doesn't even want to talk to me about anything about hard hats, let alone the thing that I was told that i had to ask them about. And so i think the in the interview uncertainty is sort of exacerbated by the uncertainty that we're carrying around that has brought us into having that conversation.
00:21:10
Speaker
Back to your point about couldn't you just go hang out with people and talk to them like, yeah, in ah in a coffee shop or in line to board the plane or something. But the stakes for bringing you there are low.
00:21:21
Speaker
it is hard when the thing that you're worried about trying to solve, you know, the rubber meets the road. You've got to have that face time and get that. Got to get it. Get that answer. Got to get the answer. So it's but there's a big ambiguity that's always there.
00:21:35
Speaker
And then there's the the ambiguity of the task of doing the interview. The big ambiguity is like, you know, in research, right? Like the big ambiguity of the project, like I don't even know if I can find anything here, like whatever, much less a specific task ambiguity, right?
00:21:49
Speaker
I don't know if there are many careers that in good faith acknowledge that that's actually kind of the case all the time. If you're an engineer and you're coding up something,
00:22:00
Speaker
And you're making a new whatever, an app or something. There's a big ambiguity there. You don't know if this is going to go anywhere. You don't know if this is going to work. You have no idea, right? You're kind of betting.
00:22:11
Speaker
But I don't think that's actually acknowledged most of the time. Whereas I'm like constantly having to acknowledge it. I don't know if this person even knows what a hard hat is. so how am I going to manage this all the time?
00:22:24
Speaker
Yeah, the thing that about research being so much about other people, if you're an engineer, you have a code base to work with and you have you don't know the validity of it, the reliability of it, or or can concern whatever sort of attributes of it, there's things that you don't know. But when your base is another person that you have to intersect with in some way or data, you know, that that's generated by people. Yes.
00:22:48
Speaker
What is it they say about other people? I believe I know. Hell is other people. I mean, Sartre said that when he was talking about like how difficult it is to be in the world, to be yourself, to be in the world. You can't be a Robinson Crusoe. You can't be on an island somewhere. You have to be with other people. And that is hell.
00:23:08
Speaker
Relying on other people for like the outputs of your career. Like your job is contingent upon... And not just like project managers or it's contingent upon other people doing what your project management plan says. Like as researchers, we have to like pull stuff out of people that may or may not be there.
00:23:27
Speaker
And then we are held accountable for not having pulled out something that didn't exist in the first place. Yes. God, that's hard. Yes. Why didn't you find anything out about this and this? Here's an example. This is kind of a nice example, though, because I did a study for a pharma client once upon a time, and it was an all-day ethnographic visit with a set of specific type of physicians.
00:23:51
Speaker
And we, of course, could not observe patient interactions unless the patient explicitly consented, which did happen once or twice specifically. So it was me and a colleague and we were there with this doctor for some time and we saw some stuff and we were in the office with him, sometimes with patients, mostly without ah just observing the office and that talking with the secretaries. And at the end of the day, he's sitting there behind his like doctor desk with his little lab coat on and his stethoscope around his neck. And he had really changed over the course of the day and his interactions with us. And he's like, this is not what I expected. this day with you. And we were like, well, what were you expecting? He goes, oh. And he told us about how he had done all sorts of studies before, which is actually kind of why we selected him, because he passed these recruitment criteria. And they were basically focus groups.
00:24:46
Speaker
um More or less. And the way he described it was, yeah, so I go to these groups and I start talking to them and they want to tell me, like, how do you like this
Listening to Participants' Needs
00:24:54
Speaker
birthday hat? And he's like, I don't like the birthday hat. And they're like, yeah, but tell me more about the birthday hat. And he's like, well, I don't need a birthday hat. I need an elephant.
00:25:01
Speaker
And they these the words he's using. Right. And they're like an elephant. Well, anyway, let's go back to the birthday hat. tell me more about the birthday hat. And he's like, i don't think you heard me. I want an elephant. I don't want a birthday hat. I don't care for the birthday hat, not because it's not a nice birthday hat.
00:25:17
Speaker
It's because I need an elephant. And we were sitting there and we were listening to this and this guy seemed to be like joyfully telling us that he felt listened to for the first time in any of these studies. He's like, I feel like I can tell you I need an elephant.
00:25:32
Speaker
And we were like, yes, you can tell us that you need an elephant. That's totally fine. And we went back to our client with part of what he had told us and we translated into their terms, right?
00:25:43
Speaker
I guess this is a good story because it totally went off the path, but the client figured out that they were making a birthday hat. On their own, they figured this out. And they needed to not make a birthday hat. They needed to make an elephant or is something like an elephant or at least understand why did they need an elephant?
00:25:59
Speaker
ah We didn't even know that whereas elephants existed. What is this about? So if you're going to sell somebody a birthday hat, then you you better understand that their expectation is is an elephant. So, you know, your messaging or your positioning. Exactly.
00:26:12
Speaker
yeah Exactly. Exactly. And if you don't have that information, you're going to sound really tone deaf, like all of those other focus groups that he was in. Being able to have a reconciliation, I guess, with reality, you kind of feel something. I'm even feeling it now, just as we're talking about, I kind of feel my chest loosen a little bit. And I think about like,
00:26:32
Speaker
oh, it doesn't have to be this one way that everybody thinks it's supposed to be. It doesn't have to be a birthday hat. It can be an elephant. And it's okay. Just like making peace with the fact that it's not a birthday hat and it can't be and no one wants a birthday hat.
00:26:50
Speaker
It just makes you relax, you know, and feel a little bit more accepted. Maybe. Why do I tell you this story? I think I tell you the story because that lesson of being able to feel a participant relax and tell you that they're relaxed, relaxed enough to tell you what they really like, what they really don't like, what they really want, what they really need, and then being able to tell your your stakeholders that.
00:27:12
Speaker
That is just such a wonderful, like, let us dispense with the fiction of the birthday hat because it doesn't work. But a bunch of pieces need to be in place for that to happen. So you, the researcher, need to not do what, at least in this participant's experience, every researcher he'd ever encountered had done.
00:27:30
Speaker
so you need to sort of be there and create the conditions for him to tell you that because lots of people would just simply say, All right. The fourth time they get asked about the birthday hat, they're like, fine, I wish it was red. And so you need to create the conditions for that.
00:27:43
Speaker
You need to have stakeholders that are able to hear that. And you need to do that dance where you figure out, well, how do I bring back the yes and so it feels like a yes and and not a no buts?
00:27:58
Speaker
I know you wanted us to look at birthday hats, but we're here today with this report to tell you about elephants. I mean, that's a bad meeting. So how do you you know work with that team to help them have learning ready moments and and see that the greater possibilities and the risks if they stay with the mental model that they have?
00:28:16
Speaker
I think you have to do the work yourself. This is the gift I'm trying to get to, right? So you have this realization when you're in the field or when you're doing interviews or whatever, where, okay, my stakeholders want to sell a birthday hat and nobody wants it.
00:28:31
Speaker
And they want elephants instead. So I have to create this yes and opportunity to open the the door. And if I don't do that, if I just keep asking about birthday hats, I may get some like short term positive validation from my stakeholders, but I don't solve anything.
00:28:52
Speaker
And You know, it might take you a few tries to figure out how to get to yes and. And with some stakeholder groups, I mean, I think we can all say it's impossible, like you're never going to get there. But it's like the practice and the trying of opening and like letting go.
00:29:09
Speaker
There's a concept in the Neo-Freudian literature called neurotic claim. And, you know, hell is other people. Well, expecting other people to do something that they will not do is a neurotic thing to do.
00:29:22
Speaker
You expect them to kind of come to your point of view and you do everything you can to make them come to your point of view. And then they just don't do it. OK, what are you going to do in that case? And I think this is good personhood training to confront that regularly.
00:29:38
Speaker
They are not going to do this. Okay, what am I going to do about it? I mean, that's a micro lesson in that particular study. It's a micro lesson for that particular product line. It's a micro lesson getting a little bit bigger, at the like a meso lesson for like the whole project.
00:29:54
Speaker
But then it's a macro lesson. Wow, I can't make people do anything ever. And so how do I grapple with that reality? Because it is a reality. You know, I think you can consider a variety of outcomes.
00:30:08
Speaker
It's not like being listened to or not being listened to is a binary. huh So like in the hard hat example, that guy was basically telling me like, I don't want to, well, it was a hat, right?
00:30:19
Speaker
I don't want a birthday hat. I want an elephant. And we were able to define elephant for the client. Client didn't want to make elephants. But not they did not make birthday hats. And i think I think that's a business success,
Reflecting on Research Experiences
00:30:33
Speaker
right? So the research prevented them from making a costly mistake.
00:30:38
Speaker
And it's not like i don't think they said, oh, you know, Steve's research really revealed the risks to us. I suspect they just ran out of steam. They're kind of like, eh. Like no one got excited from the research or the design exercise that came from that.
00:30:51
Speaker
Because we didn't have anything to give them that would get them excited on their path. We had other things. We had other things. But if no one bites on that, then no one starts coming up with interesting designs and future visions of, you know, what it is to provide elephants to people.
00:31:07
Speaker
So i I didn't feel listened to. ah definitely got some, huh, moments when I was sort of sharing examples of how other PPE and even like there's a brand of consumer bike helmets that looks like. They look like hats. Yeah.
00:31:22
Speaker
Not that. I think I think I know these. Yes, because they they have a very specific niche. They're looking. It's the same as the PPE thing. Yes. When you get off your bike, you look like a dork with your helmet on. Right. yeah So when you go into shared public spaces like a cafe or going to go get lunch or whatever, if you still have your PPE or your helmet, you look like a dork.
00:31:42
Speaker
Can we ease that transition a little bit? I don't want a birthday hat. I don't want a hard hat. I want something that looks like a normal hat. Yeah, that's the problem statement or the opportunity area.
00:31:53
Speaker
And there was precedent out there in other categories and in sort of workplace PPE. And that wasn't exciting to them for whatever reason. I just got sort of a skeptical reaction. But I'll take it as a win that they didn't try to do the thing that was not going to go anywhere. that was i think that...
00:32:10
Speaker
That's a lesson right there. That is a win. Okay, so this is what I'm talking about, about the continual learning, like the personhood training that you get as a researcher. Ideally, what you want is for them to listen to absolutely everything you say and say, oh, my God, I can't believe we thought about birthday hats and to begin with. This is crazy. Of course, we're making elephants.
00:32:29
Speaker
Because that's what we're taught, right? I mean, if you look at sort of the literature from promotional innovation consultancies, it's always that. We're the smartest. We told them they did this thing and it made a billion dollars. And that and it was amazing. Yeah.
00:32:43
Speaker
And likewise, you get this impact, like this whole internal narrative from your own organization talking about impact, like researchers have to make impact, impact. And how do you make impact? Well, my product team wanted to make birthday hats and they didn't. They made elephants because I found out we needed elephants.
00:33:00
Speaker
And, you know, big promotion. ya Congratulations. But the reality is often not going to be like that. Now, the fact that they didn't make a birthday hat, maybe they made a hat, but they and they didn't make an elephant, but they didn't make the thing that nobody wanted.
00:33:16
Speaker
They didn't, right? That's a big win. That's a personhood training moment where you were like, I'm going to take this win. It's a small one, but I'm going to take it. And, you know, we could also, we're sort of presenting these scenarios as if it's a binary choice between this product, birthday hats and elephants. But there are those other scenarios where, well, if you're going to do that, here's the barriers you have to overcome. Here's the positioning. Here's the way. There's a different narrative. And those maybe are harder to observe because you're, and we've talked about this in other conversations, we're
00:33:47
Speaker
Maybe changing the way someone thinks maybe the way that they write, the name they give something that right they're going to describe a process. They might start with the thing the person already has and not the thing that they're making. So those are more subtle wins that you maybe don't get evidence of as easily that you still have to feel as a person.
00:34:09
Speaker
have to work at. Some hopefulness for that. Yeah, you have to work at getting that feeling. I think that that's the thing that researchers do a good job at generally for their stakeholders, but I don't think they do it a good enough job for themselves, which is like, OK, I'm going to have to like reflect on that experience that I personally had. How did I feel about that experience? What did I get out of this? Where's that internal sense making? Like, I got external validation because I gave them all the things.
00:34:37
Speaker
I read a post this morning. You and I were were chatting about it online where it's UX researchers are not OK because they're going out there and they're being asked to ask participants, you know, how much do you like AI? And tell me how much.
00:34:51
Speaker
Right. And people are like, I don't. And they're like, please use AI, please use AI. and that And the users are like, I don't want to, you know. And then they have to come back to their stakeholders and they have to tell their stakeholders, yeah, we talked to the users and nobody likes this.
00:35:07
Speaker
Nobody wants this. Nobody knows why to use it. And they're worried about losing their jobs as a result. And that is a real problem. Maybe you shouldn't be focusing so much on stopping them from building a birthday hat.
00:35:20
Speaker
altogether. Instead, you need to at least give them maybe some different mental models about what is AI. Is it an oracle? Is it an assistant? Is it, you know, a machine? Is it a Cuisinart food processor? Like, what is it?
00:35:35
Speaker
Can you stop the AI train? No, not as an individual. I don't think so. I mean, i don't think we're going to have to worry about the AI train is going to stop itself in a few minutes because it's garbage. Right. And this is same as it ever was. i mean, I i think 15 years ago, someone said to me, oh the problem with ethnographic research is that it's always bad news.
00:35:54
Speaker
And I thought, wow, that's, I mean, that's a really distressing way to kind of hear it framed. It is indeed. So is it? I don't think it is. yeah Is it? Why did they say that?
00:36:05
Speaker
I mean, I think it's the, you know, our job is to get feedback and then go try to tell people their baby is ugly. I mean, you wouldn't be doing the research if you didn't want to find the gaps, the misfires and the misalignments.
00:36:16
Speaker
But it's like saying like, oh, a corona coronavirus is spiky on top. What bad news? I'd rather not know that. Right. Your virus is ugly. Your virus is so ugly. Oh, here I was thinking it was so smooth and easily combated, you know.
00:36:34
Speaker
I really hate that research that they told me, were how spiky it is. What bad news. but if you come back and say, guess what? Nobody wants birthday hats. Maybe you're better than all the researchers that your participant had met.
00:36:47
Speaker
But that still, that doesn't really pass the so what test. Yes, that's true. and We shouldn't assume that our stakeholders... And not even the emotional work of that, but just sort of the creative work of that can then turn around and say, well, OK, we're going kind of pivot to this.
00:37:04
Speaker
So if we want to get from as a chatbot to AI as a Cuisinart. hmm. We have to find some hooks for that. Some how might we question some clues from the research that say, you know, the mental model was this. We have to use all the richness that we're gathering in all these difficult conversations to bring, if not opportunity, to bring the trailheads that people can explore.
00:37:27
Speaker
and Closer into view. Closer into view. This is the trailhead right here, friends. Yeah, we have a story where this person said this and said that, and they use language that was more like appliances and less like relationships. And so that that's a question for us to explore in the design exercises. like I think what it's easy to lose sight of that in the short term. You know, you're probably getting...
00:37:50
Speaker
a lot of feedback these days as a researcher that maybe people are saying, you only bring us bad news. Maybe they're saying that. Maybe they're saying you're too slow. Maybe they're saying lots of things. But, you know, history is long.
00:38:03
Speaker
And the everything right now feels big and overwhelming and stupefying with its intention and its, like, force. There's an intense force of events these days. It feels that way.
00:38:17
Speaker
And if you just kind of, like, chill out a little bit and like drop for yourself, drop the intensity for yourself, get a little healthy disinterest and recognize, yeah, I hear the bad news. Yep. I hear you're thinking it's bad news. Yep. I hear it. Yep. I hear it. Do what you can to get your little wins in those moments. so But then...
00:38:35
Speaker
it comes around. It comes around. Like, this is why an interview is such a good personhood training, because the interview comes around, right? You get back onto a path. Maybe it's not the one you thought, but there's definitely a path there, and it starts to emerge, and you follow it. The trailhead's over here. And that's the personhood, right? That's the life skill, is to figure out that different path is a good one.
00:38:58
Speaker
Your discussion guide that you revised and got feedback on is just a hypothesis, and bears some overlap with the person that you're going to talk to and then the next person that you're going to talk to the next person you're going to talk to so figuring out how do you bring sort of questions and intentions and a script or whatever that is and then how do you set that aside and have it guide you not script you yes guide you that's that's all that's a lot of uncertainty in that
00:39:29
Speaker
and And because you're really just picking the next question or the next utterance, which could be nodding or saying or just saying, oh, or nothing.
00:39:41
Speaker
Jinx. ah and we Do we get a prize for that because we said the same thing, which is about nothing, at the same time? So no, we don't get a prize. Right. there If there was an irony award, we might might get that. We can award it. I mean, who's we're in charge.
00:39:55
Speaker
Congratulations, Steve. Yes. Yes. so i i like I'm happy to share this award with you. Thank you. i appreciate that. Yes. ah Talk about the losing your train of thought. Yeah, so this is just all good personhood because you can't control the whole thing and you also can't decide the next thing. So the more you are in the moment means you are just making short-term decisions about what to talk about because the next thing they tell you might be something that makes you think, oh, well, the exciting part of this conversation is over here.
00:40:25
Speaker
oh, now the exciting part of the conversation is over here. And maybe it does come around and you're like, all right, let's go way back to where we started. But it might not. Yes, it might not. and that's neither inherently good nor bad. You just don't know. You're just following it where it goes, which is a surrender, right? You have all this agency and yet you are surrendering it. And again, that's worrisome, but also energizing if You kind of feel that force. I'm sure there's some martial arts metaphor that someone that knows martial arts would kind of bring in here. There is a just a a different way of being.
00:40:57
Speaker
And yeah, personally, I wish I could be that way in my life. I wish I could. Well, you know, there's a good chance that you're more that way than you think. And this is my whole point with this topic of interviewing.
00:41:08
Speaker
You being an expert interviewer, you know, me being an expert interviewer, we actually gained some skills about navigation. So you probably have more of that than you think. Research is a gift for that.
00:41:22
Speaker
And I think the in the moment and in the project piece are really important too. And I i like how you've, well, we both have drawn the the parallel between them. There's a task that you're involved in in life or in in research, the interview.
00:41:34
Speaker
And then there's this set of goals, this risk and uncertainty and concern about the expectations other people have from you and are you going to measure up in that. The more I am able to go with the flow of those things, I think the better off, the happier. I mean, I think there's sort of success, like we shipped birthday hats.
00:41:53
Speaker
And there's also success like, oh, I didn't lose my shit over some failure, which was out of my control. That's a success. Yeah. And that's a win. I think we can confidently say we got somewhere today.
00:42:05
Speaker
What do you think? We did. Can we throw in an appendix? Of course. yeah that's your i love appendices. Oh, your your description of the guy behind his doctor desk and his lab coat made me think about an off-the-path, potentially off-the-rails moment that I think goes back to where he says with a doctor, i just wanted to share ah a little story about that, was interviewing a neurosurgeon or like a head of neurosurgery at like a big cancer hospital.
Adapting to Participant Dynamics
00:42:32
Speaker
it's a Kind of a lightweight, obviously. Kind of a lightweight. yeah mean I mean, you're meeting these people that are extremely talented and smart and successful. They're sort of smart at everything. And they were quite interesting. And they were challenging. And I was definitely given warnings by everybody. Oh, doctors. Oh, they're going tough. Oh, surgeons. Oh, neurosurgeons. And so we we go in and meet this guy. He's the head of whatever. And, you know, we had, I mean, he was a great recruit. He was a good get because he had all these qualifications. Yeah. And for business reasons, the study was blind. So our client knew who he was and I think maybe identified him, but he didn't know who we were.
00:43:08
Speaker
We weren't using that relationship. And we go in and just we're sitting in his office and he's giving us sort of the background and he's got his book. you know, it's one of those offices where his book is like spine forward and there's like two shells of it and there's a framed photo of like maybe him with either like, I can't remember what it was now, but it was like a president throwing out the opening pitch or him with this president or his son who played, you know, in in the National League. There was some fairly rarefied accomplishment that was there that I don't think we had to say, oh, what's that picture?
00:43:44
Speaker
i think he said, and here's this. And so he was pretty proud of himself. And i I think justifiably so, but he was pretty proud of himself. And at one point he sees that the, basically I had this set of scenarios and he, think he snatches it from my hand.
00:44:02
Speaker
I've had that happen a couple of times or or demanded it and then sort of took over the mode of, of querying. Like he was just kind of shuffling through these pages and being like, yes, yes, yes, no, no, no, I and i don't know. And then proceeding to giving me lectures about how you guys should approach this. And just the whole, if you picture that kind of person, how they would how they might treat you.
00:44:23
Speaker
And, you know, anyone that's listened to these conversations has has heard me talk about I don't know, my just general social vulnerability and not like being, you know, marginalized and the fear of being sort of marginalized by people. And this guy did everything to me. I mean, he was kind of a dick.
00:44:39
Speaker
It was a great interview. And when we left, the person I was with who was in the industry and was a researcher, but was not sort of as far outside as me, was sort of looked at me worriedly and apologized.
00:44:51
Speaker
and And I would i said, in all honesty, like, I'm fine. That was great. And so for all my insecurity, again, like I'm a white man. So being talked down to maybe is less freighted than somebody in a different situation.
00:45:06
Speaker
But he gave us great information. I mean, he sort of took over the interview. But the stuff that he gave us, for the most part, I mean, he definitely had other things he wanted to tell us. was extremely specific, sometimes too specific. And I was able to play to his ego, not in kind of a fawning way, but it was clear how to talk to this guy and what pose he wanted me to take and how, i mean, we always want our participants to be the expert.
00:45:33
Speaker
This guy put the crown on before we walked in. And so he was rude if this was you know a dinner party I would still be upset about it three weeks later you know sure but in this case you know it went off the rails for the other person they felt like that this was harmful or dangerous to me um and I guess I was surprised by myself like I was not upset by this And, you know, in any circumstance, like if a waiter talked to this me this way or the neighbor or like, I mean, I there's no end of things that that are very small slights that I will fume about at length.
00:46:10
Speaker
And here this was a lot of slighting, I think, because this context was so different. And, you know, you're playing a character whose ego can be set aside.
00:46:21
Speaker
As an introvert, I'm more able to play a character, I think. And, you know, have it be over when was over. So I think about this interview and I can get some of the emotion of how this guy was talking to me. But it's almost like bemused surprise that I can kind of recall. And also just when someone's giving you the good stuff and you're like, this is gold. This is gold.
00:46:42
Speaker
Right. That feeling is so good. And it was great. It's like getting a hole in one. It feels amazing. It feels amazing. Yeah. And so I feel like there are, there's this emotional work that has to be done.
00:46:55
Speaker
And there is this uncertainty about what are we going to learn and how is this going to go? and there is sort of who you are in that moment. And, you know, I think you just get interesting mixes where like, like the emotional work, there could have been more emotional work, but a sense that like, oh, I'm going to be good with this project because I'm going to be able to come back and say like,
00:47:15
Speaker
Oh, he was just so clear about the mistaken assumptions in this prototype. Yeah, yeah is it's funny how your stakeholder didn't see that in the same way. It is interesting, isn't it?
00:47:26
Speaker
I think it says a little something to the gift of being the researcher is that you can disappear, you know, like you can ego less. You're just like transparent. You're completely nothing sticks to you. It just goes completely through you. You're a conduit, you know.
00:47:40
Speaker
At least within a certain range, right? and mean Yeah. You brought up danger early on. I think there is a whole set of things that are that are really awful and and hurtful to people.
00:47:50
Speaker
And I guess it's cool to think about one that looks like it could be, but was not. But was not. And to think about why that was. yeah Yeah. And my my colleague didn't sort of understand that.
00:48:03
Speaker
And, you know, kudos to them for doing caretaking to come back to me and saying, like, are you OK, Steve? Like, you know, I'm sorry that happened. And you were like, yep, I'm good, man.
00:48:13
Speaker
Yeah, that was great. But that's I remember also being very pleased that they took care of me that way. Well, there's a lot to be happy about in that situation.
00:48:24
Speaker
And you get to tell this story to me, and which I can fully imagine him snatching the sheaf of papers from your hand. No, no, no, this is dumb. ill Let me tell you this. You know, I don't know if you guys know anything about this, but let me tell you. i just I love that.
00:48:39
Speaker
I can picture it. Yeah. It was one where I had a big discussion with the client about the fidelity of the prototypes because it was really, it was a different concept for how this work would take place and what they did, whatever sort of three d rendering tool does you might use in 3D modeling, like ah yeah like a very high fidelity visual thing.
00:48:59
Speaker
And they had it over seven or eight scenes and I really pushed them for low fidelity. And they're like, nope, we're designing this. It has to be as great as we can possibly make it But they did things like the color of a hose changed between frame three and frame four, or there was like a chair and then the chair was like a different design, like they were using sort of standard elements.
00:49:23
Speaker
Yeah. So they were not consistent. and And think about it as an animation, right? As you kind of show the narrative unfolding. There were some sloppy things that I didn't see. And this guy caught them. And he's like, well, why does the hose change when you do this to that? And and so like that's the stuff that's what you don't want to spend time in the interview talking about. ah hundred Because like it's a mistake. We're not really designing the chair. though That's not but we this is not a chair project. We're not working on chairs. And then you got to then you got to bring him back.
00:49:51
Speaker
But this guy was not going to let anything go by without saying it. So he gave a complete inventory of everything that was sort of wrong, including like prototype errors. But I don't know if my dark heart was sort of enlivened by that because I felt like, see, told you guys not to do this and see what a waste of time it is. Having this guy kind of go off on this other stuff. So. So like Dr. Brain Surgeon noticed and you guys didn't listen to me. So maybe now Dr. Brain Surgeon's comment will allow you to see that I was right.
00:50:23
Speaker
and You know, probably I brought none of that back, but I definitely had that like, hey, hey, hey moment of like, yes. That feels good. you know i can i can I can get a sense of that.
00:50:33
Speaker
I can feel how that would feel. So there is, and this is back to ego, right? If we do it this way, it's going to go wrong in some small or large amount. And it did go wrong.
00:50:44
Speaker
It did go off the rails very, very slightly because we're wasting time talking about hoses and chairs. very precious time with the head of neurosurgery at a big cancer hospital. I did feel sort of, i didn't get my way. and We're talking about sort of wins, right? Yeah, sure. I didn't get the client to listen to me, but I affirmed for myself that my guidance was right.
00:51:05
Speaker
ah Even though I had to clean up that mess, i had to be like, let's not talk about the hoses. you know, or or just nod, smile and nod, whatever it was, it sort of came down to me, but I did feel some validation of like, yes, you know what I'm talking about. That is good advice on how to approach this study.
00:51:20
Speaker
And that is also a win, even though you didn't get your way. And the fact that you got to feel validated and your stakeholder asked you if you were OK, that's also a win. And you get the story. I
Final Thoughts on Diverse Research Paths
00:51:35
Speaker
get to feel Dr. Brain Surgeon's presence in this conversation, which I love.
00:51:39
Speaker
I love this story. I can almost imagine it as if I had done the interview myself. Excellent. I am sure that the lesson here is you can't own a path. You think you can, but their trailheads move and you gotta find the ends of it wherever it happens to be with that.
00:51:58
Speaker
Until next time. You have found the end of this. i have found it. Indeed. We have found it together. Until next time. Great. Thanks for listening to the Off the Path with Sam and Steve, the show that takes you off the beaten path of research careers and onto your own chosen path.
00:52:14
Speaker
Hire Steve Portigal to lead a research study with your team or to help build user research skills in your organization. or to deliver a talk or workshop for your event. Learn more portugal.com slash services.
00:52:28
Speaker
Steve's classic book, Interviewing Users, is in its second edition with an audiobook. Check out portugal.com slash books for more. And Steve has his own podcast, Dollars to Donuts, where he talks with people who lead user research.
00:52:40
Speaker
That's at portugal.com slash podcast. You can also connect with him on LinkedIn. Hire Sam for research projects or research coaching. Take one of her classes or sign up for exclusive video content at samladner.com.
00:52:55
Speaker
Sam's books, Practical Ethnography and Mixed Methods are both available on Amazon and her new book on strategic foresight is coming out in 2026. Sign up for her newsletter to find out more at samladner.com slash newsletter.
00:53:43
Speaker
Bye.