Transcript
Speaker: companies would rather invest millions of pounds getting those people in rather than taking care of the people who've already sold into whatever it is they're selling, if it's a service or a product or a brand story, whatever it might be. So, you know, i think there's a general dislike of trying to understand our fellow humans because If I'm being honest, we're all massively complicated, right? It's a complicated job. We're not easy. We can't be put into numbers and dashboards and black and white. There's so much greyness and ambiguity and context that needs to be taken into consideration. And I think in today's fast-paced world, that is too much of a mountain for people to try and overcome, even though they should. it's hard selling.
Speaker: Welcome to Unite Voices, hosted by Katie Green. Real stories from the people behind today's most innovative experimentation programs. No fluff, just wins, failures, and the lessons in between.
Speaker: Thank you so much for being on Unite Voices and making time pretty last minute. So shout out to you for being able to do that. We appreciate you so much. So I'm Katie Green. I'm serving as principal advocate at Chameleon. I'm responsible for managing the lovely community of experimenters and that...
Speaker: Join episodes like this, people like you. So the other thing I got to do was host our experimentation thought leadership awards, which Abby, you won the UXpert.
Speaker: I love a good pun. It was brilliant. Shirley, note that, note that for our bosses, okay? No, I'm so happy you won the UXpert category. You share really helpful stuff online. So tell us a little bit about yourself. Oh, how far back you want me to go? How long have we got?
Speaker: No, I don't know.
Speaker: As far back as you want, Queen. right. Okay. Well, I'll take you way back in time then because why not? So... I went to university, as you do, and um I started off studying microbiology and zoology, right?
Speaker: That was my thing. That's what I wanted to do. Thought it was massively enjoyable. And then my course got completely canned, right? They just cancelled it for some reason. And I was left with a dilemma as to what what should I swap to? What should my course be? And it came down to two things. The first one was geography.
Speaker: because actually I really like geography and I found it really easy to do, so that'd be easy. And the other choice was computer science, right? And there are reasons why that was on the shortlist, which I'm not going to go into.
Speaker: Yeah, I noticed you so you started your your career in, um but you know, to like web development. Yes. So the computer science course ran and very this is like early, no, late 1999, think it was.
Speaker: ninety nine i think it was And at that time, usability wasn't a thing, right? Nobody knew what usability was. And in fact, my course was nearly all programming databases, databases, hardware, all of that traditional computer stuff. And there was one course on usability at the time, and it was very niche. And the lecturer thought he was amazing because he'd come up with this new thing. And for the three, four years I was there, it was the only one module in it, it was the only module that I actually enjoyed for the entire course.
Speaker: Anyway, left uni, went to do the London thing like everybody does, got a job and I was front-end developer, very exciting. And was working on a website called Love Film at the time, which I suppose, don't know if you remember it, do you? You heard of it?
Speaker: Nah. But I'm in the US. I haven't heard of anything. Ignore me. Modern day version of Blockbuster, basically. You went onto a website, chose films, you got posted to DVD, if anybody remembers what they are, in the post. You watched your film, you sent it back.
Speaker: um And at that point, we actually did some user testing on the website because I was busy building it, as you do thinking I was amazing because I was building it and designing it. and But we did some user testing and I watched the videos of these people using what I created. And I was horrified, to be honest with you, because nobody knew what they were doing.
Speaker: Yeah, that was going to be my question is like in that era. I mean, this is really like. Was it in your, on your, I was looking at your LinkedIn, was it like 2005, 2004 that you were there?
Speaker: Yeah. So like in that era, how were, how were you making decisions on design and functionality? Was it just kind of like, Ooh, this looks cool. We're going to build it.
Speaker: Absolutely that it was whoever had the biggest opinion in the room. Nobody else thought about users or, you know can people actually do this it was whatever got floated people thought was a good idea we built it released it end of story job done I mean I don't think you'd be surprised to hear that that is happening today yeah I'd like to say things have moved on but apparently not It's just got fancier names and people are poking around the edges still. They're not really understanding what their users need when they're on a website and a lot of stuff does still get pushed through which is of poor quality and doesn't address those particular needs that really do need to be addressing. So this is why I'm still in the industry and still banging on about the same thing later.
Speaker: I was going to say, you're getting far. You're getting far. i think the, i so I've been an experimenter for like 10 years. I kind of, my favorite line to say is that I ran my first test in 2016 remember it well,
Speaker: but You know, I think that obviously the baseline expectation is higher now, right? It is that you are coming to the table with data, with this, with that. But something that I've noticed throughout my career is that UX, conversion rate optimization, when people are cutting corners on budget. Those are kind of some of the first things to go. i know. God, a budget cut? No, I've never heard of her. um But like, you know what I mean? it's it's It's very puzzling to me because it is what separates people
Speaker: a fine program and an excellent program. it It is a mature digital experience. So have you found throughout your career that usability, UX, I know you were director of optimization at one point in your career. Have you found it being more center to the business objectives of customers you work for or places you work at? Are you finding it more going more in that direction or kind of plateauing?
Speaker: ah No, from my view, it's still plateauing. I think um good user research, those good insights, they're still not being... They're being done by a minority of companies, but to the level that I would expect them to be done, absolutely. People are still nowhere near that. And we have even more of an issue now, and I'm going to say it, if anybody's got bingo card, because AI, right...
Speaker: right
Speaker: AI is simply fanning the fire towards even less understanding. um In my view, yes, it can assimilate huge amounts of data, got no doubt about that, does a far better job at doing that but in terms of everything else it's still being used as a shortcut so people will feed it all into ai go oh look there's our top 10 themes let's run an ab test to try and address that but they don't understand the underlying reason why that has come out as a theme and they don't bother to investigate it so they end up wasting time um
Speaker: on tests that will never do anything because they're not actually addressing a problem or a need from a user's point of view. In terms of business goals, I wish more businesses would recognize the importance of aligning their goals with their users more rather than the usual thing that we see where it's just profit profit, profit, profit, more users, more users, more users, how many more can we convert?
Speaker: Oftentimes businesses don't even consider things like retention, which is another one of my top soapbox moments. People don't understand that if they can, well, businesses don't understand that if they can retain customers, it is cheaper than trying to get more in through the door who, you know, if you're lucky, they'll convert.
Speaker: maybe or they'll convert once and then they're gone but companies would rather invest millions of pounds getting those people in rather than taking care of the people who've already sold into whatever it is they're selling if it's a service or a product or a brand story whatever it might be so you know i think there's a general dislike of trying to understand our fellow humans because If being honest, we're all massively complicated, right? It's a complicated job. We're not easy. We can't be put into numbers and dashboards and black and white. There's so much grayness and ambiguity and context that needs to be taken into consideration. And I think in today's fast-paced world, that that is too much of a ah mountain for people to try and overcome.
Speaker: Even though they should. So it's hard selling, in I think. And that's, I kind of want to take that, the especially the first part of your answer and this whole answer and post that directly on LinkedIn. Because one of my favorite questions to ask, and Johnny Longden actually posted me asking him this question on his LinkedIn, is, ah you know, everyone's using AI. It's kind of like, and it is...
Speaker: or a requirement of the job these days is AI literacy. And, um you know, the thing I asked Johnny is, are you using AI to just move faster or ask better, deeper questions? Because i know synthetic audiences are,
Speaker: It's kind of the most surprising thing about this job that I'm in now because I'm an experimenter by practice. I have run tests. I've run experiments for some of the biggest, specifically e-com ah brands in the world. And I'm very fortunate for that experience.
Speaker: but it never even crossed my mind to use AI for synthetic users. And that's, as I'm here, I'm realizing that is such a trend in experimentation, but I'm like, is that just helping you fail faster? Are you failing more?
Speaker: And even by failing, usually you learn something from a fail, but I don't know if synthetic users are providing a valuable fail. So I wanted to ask you, AI, is there a way in this world as we see it today to use it in a way that asks better under question better questions, creates better understanding of users? Where do you see its role in usability and optimization? That's a big question. You said they were going to be easy.
Speaker: i lied to you to get you on the podcast.
Speaker: What is AI's role in the world? I can only give you AI's role in terms of, I suppose the easiest place to start would be my context of how I use it, right? Just as a starting point.
Speaker: For me, would I use synthetic users?
Speaker: No, I wouldn't actually, not at this point, ah because... That's the general consensus I'm hearing. You're not alone. Yeah, no, I wouldn't because ah no matter how much you dress it up and whatever fancy terminology you use or how many people you try and map it against, it's never going to be that one person sat at website trying to figure out what to do, right? it It hasn't got the real world stressors. And yes, you could prompt and say, oh, imagine you're making dinner and your toddler's hanging off your ankle or some shit like that, right?
Speaker: It still doesn't experience... Adrenaline going up or, you know, feeling like you want to punch somebody or so it it hasn't got that real biological response to human stresses that people might be using when they're trying to work on earth.
Speaker: buy something on a website, whatever it might be. So I'm not a fan of the synthetic user at this point. I'd rather speak to humans. um There's a lot you can, and I'm talking face-to-face communication here because there's a lot you can pick up in body language and the way people talk and, you know, just human stuff, right? There's so much you can pick up from that that actually feeds into an analysis if you're doing those proper one-to-ones.
Speaker: to So no one the synthetic users. For me, AI's place at the moment is trying to One, organize all of the research that I have got, right? And most research programs, or maybe I'm just a bad example, I get everything and then I try to organize it, right? That's just the way that I work. So AI for me is a mechanism to be able to organize that chain of, that mess into some kind of chain of thought and to try and understand that.
Speaker: But, you know, even that has its downsides because I remember in the early days I was fed in a load of research into AI and said, oh, pull me some quotes out from those transcripts.
Speaker: And bearing in mind, i had done the interviews in the morning and I was asking this in the afternoon. and I discovered it was just making random shit up right? So it was making quotes up out the transcripts. I don't remember somebody saying that and you get the typical response, you're absolutely right. Yes, I completely made that up. It's like, well, please don't ever do that again.
Speaker: yeah So, you know, that's where the benefit is for AI for me. It's interesting in terms of, I think the more interesting question is, is how normal folk are using AI, right? Because on LinkedIn, in this industry, we're all in our own bubble. but How are going to use AI to make our jobs more efficient?
Speaker: Or whatever. and The more interesting point to me is how Joe Bloggs Public is using AI. That is ah far bigger question and a far more...
Speaker: impactful question that as optimizers we should be asking ourselves yes we can do whatever we like right optimize process make it faster assimilate in 5000 transcripts or 3000 surveys whatever it might be but how are our consumers and the people we're trying to influence or target or optimize for how are they using ai and how does that affect the work that we're doing I think that's a much more interesting question.
Speaker: And that's what I'm trying to answer at the minute. And I still, I'm not sure I know. um But i what I do know from real world is people in our industry are who assume that normal people are not using AI are misguided.
Speaker: and short sighted. only have to go to the supermarket or look at people who are waiting at the bus stop or speak to even my dad who is 80. Right.
Speaker: right And they're all using AI in various different ways. And it is primarily research that they're using it for right they're fed up of having to go to Google to search for something that's way too much effort. I'll just chat into this thing.
Speaker: and it tells me all the answers. And nine times out of 10, they don't even question it, right? This is the downside. They just take it verbatim. Oh, that's what I will do. So then the interesting interesting bit for businesses, optimizers is, okay, so how do we influence what the AI is citing, right, to give people those responses? So that's more interesting because I think three, four, five years' time, maybe UI won't even exist in the current format that we know it as, right?
Speaker: So we'll all be doing stuff for no apparent reason whatsoever. um So it's that kind of upstream element that I i would be focusing on.
Speaker: You know, I'm lucky. You know, I'm planning on retiring in five years, so it won't be my problem. But if I was younger in my career, that's the kind of stuff.
Speaker: i'd be looking at because that's that's where the money pot will be in my view and I might be completely wrong but it's more interesting right now to look at that than look at another website and say you know what your form is terrible you should really improve that there's only so many times you can do it before you're like yeah been in this industry for too long so for me personally i'm using the AI thing as a pivot point uh
Speaker: to more and still related to users massively related to that but it's taken it away from the actual website ui way further upstream to figure out what's going on up there and it's an interesting place to be yeah i think people are so the the thing i talk to people all the day right that's literally my job at chameleon which is incredible for a gapper like me also my dad is also 80 and and I've never thought about asking him how he uses AI. So shout out.
Speaker: Shout out, Johnny. I'm probably going call you today. He'd be like, Dad, how are you using AI? I fear the answer, but honestly, it's probably because he takes care of like a thousand outside cats. So I think he probably i know he's like such a cat person. It's so funny.
Speaker: um It's very sweet. That's different to my dad. My dad is an outside garden rail enthusiast, so he uses it to help plan his track layouts, get planting suggestions, find weird and wonderful stock to put on said track.
Speaker: And yeah, and it's like all the people are optimizing or like the the smart ones are optimizing for how to be a part of those answers. um it is It is super interesting. I realize we're coming up on time, so I want to be mindful of that because there is one question I like to ask all of my guests at the very end, which is what I call the Monday morning advice. i't it's not It's assuming that people are listening to this on Friday and after the weekend on Monday – They're going back to work and they're like, okay, how how am I using ai How am I taking stock of this? Is it answering real questions about my user problems? Is it helping me understand people better?
Speaker: Where do they start, in your opinion? Step away from the computer. Seriously, step away from the computer. Go and do some old school analysis and see if the two tally.
Speaker: Do your own analysis manually. It doesn't matter how long it takes. Just do a benchmark between what it is telling you and what you as a human consider to be true and see where you're at between the two.
Speaker: hi You probably guessed, I'm big advocate in actually speaking to people and not getting so technologically isolated that you lose that connection with the work that you're doing and why you're doing it so that would be my monday advice go away and speak to somebody do some proper user sessions You know, it's a morning and afternoon's worth of work. And it doesn't have to be fancy. Go do some guerrilla testing and go get a coffee.
Speaker: Ask somebody's opinion on what it is you're building, right, while you're having a latte and a slab of cake. I don't know. um But don't don't ever lose sight of what the work is that you're doing. Don't assume AI has all of the answers because it doesn't.
Speaker: nine times out of ten, or it certainly can't necessarily interpret them correctly. So that would be my one piece of advice. If you are in user research in any way whatsoever, don't lose the reason why you started doing this work in the first place. Don't lose sight of it and Always keep it there to your side like a little, you know, devil and angel on your shoulder. You've got the angel going, go speak to the humans, and you've got the devil going, ah, I'll just do it all via AI because, you know, i want to know you lunch whatever don don't
Speaker: don't don't do that to yourself. Keep true to it and keep going, I think. Maybe adjust where the ship is going. So maybe not so much, but on the website, maybe a bit further elsewhere. um That would be my one piece of advice to people.
Speaker: I think that's great advice. It's yeah, totally. Because like, it's, I feel like the existence in between the devil and the angel on your shoulders is, is balanced. Right. And I think if I, if I were to answer that for somebody more, like I don't have a user testing user experience background at all. Like I've done user testing,
Speaker: In the sense that I've asked users to test things, but I've never like held my own, you know, group like blind study about something. Right. And I feel like this kind of underlines the importance of qualitative and quantitative.
Speaker: Oh, absolutely. You can't rely um either in isolation. It has to be a mix of the both. But our industry is very guilty of relying on numbers over everything else, right?
Speaker: It's been the case ever since I've been working in this industry. And my view has always been you can't you can't weigh them differently, right? They have to have an equal amount on both sides, which is why I get very...
Speaker: concerned about, you know, automated A-B testing and things just going out in the wild and regulated, uh, based off numbers alone. um that concerns me, but that's an entirely different podcast. That's a, that's episode two.
Speaker: Yes, episode two. Well, I was saying it'd be fun. i was thinking it would be fun to have you and a data scientist because I just interviewed a data scientist who I really, really love chatting to. And I know you would, too. um You know, she is so smart, has an advanced degree in statistics. And it's like, you know, it's funny. You guys are the devil and the angel right on my shoulders. And it depends on which to do, who you know, it's like, it's perfect. But I think that to round it all out, i I do want to encourage anybody listening who's made it this far to follow you on LinkedIn because you do post a lot of really helpful stuff that, you know, has opened my eyes to parts of the business that, you know, I'm an experimentation generalist. I've worked more on the marketing side of things, not
Speaker: nearly as technical as some of my colleagues, but you know you've really opened my eyes to parts of the experimentation process that are actually really key. I previously worked at a product that is an analytics tool. So, you know, I got really into the numbers for a second, but you're totally right. It's it's about finding that balance. So everybody listening should totally follow Abby on LinkedIn. um But thank you again so much for making the time, especially so last minute to be on Unite Voices.
Speaker: Oh, no, more than a pleasure. And Yeah. um Read the LinkedIn stuff, man. I don't know. I just don't know why I post on there sometimes. It's like, what is this? people what People want to know. People want to know.
Speaker: Myself included. If you're doing it for anybody, keep doing it for me. all right then. I should bear that in mind. I like posting on there because usually should I'm saying something that everybody else is. I'm just causing trouble. Cause trouble. Cause trouble.
Speaker: All right. Thank you again, Abby. Perfect. Thank you. And as that episode comes to an end, i did just want to say a quick goodbye to Unite Voices. This is the last episode. You'll hear me as your host, but please continue subscribing, following along. There will be more guests to come in the future. and more really interesting stuff. So thank you so much for allowing me to take you on this journey and launch this podcast with you all. We've had such incredible brains that I've been able to pick in real time with you. But this is my last episode and as I've continued on to another opportunity. But have a wonderful day and tune in next time.
