Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
The biggest predictor of behavior tomorrow is behavior today, and that should change how you experiment | Kristen Berman
1 Plays3 seconds ago
What if your users already know exactly what they want, but your product is still too hard to use? Behavioral science has a word for that problem, and it's not "friction."
Kristen Berman is a behavioral scientist and CEO of Irrational Labs, the firm she built after running Google's behavioral economics group. She studies how people actually make decisions, and she spends a lot of time showing product teams why their assumptions about users are off.
In this episode, Kristen walks through the gap between what users tell you in research and what actually drives their behavior. She and host Katie Green dig into how behavioral psychology can sharpen experimentation hypotheses, de-risk big product bets, and help teams build a genuine culture of learning.
Key takeaways
- When building an experiment hypothesis, start from a psychological theory about why behavior isn't happening, not just from what users say in qualitative interviews.
- Out-of-product A/B testing, such as running four versions of a pricing page with recruited participants before touching the live product, lets teams take bigger swings with far less business risk.
- The teams with the strongest experimentation cultures tend to share one trait: humility. They assume the first version of anything is probably not the best version.
Recommended
Transcript
Impact of AI on Workflows and Productivity
00:00:00
Speaker
behavior change is everywhere. Like if I want to just get you to do anything, the theory is I have to change your environment of decision making. I don't need to bang you over the head and send you five emails. I need to change the interface of the app.
00:00:12
Speaker
I need to get you to do something different. and so I think fundamentally AI has changed how people work. How I get work done has changed. My whole workflows are changing. oh at some level, like that is the intervention with AI is that every single person is going to a different app than they did two years ago, five years ago to get work done.
Kristen's Insights on Decision-Making and Behavioral Science
00:00:38
Speaker
Welcome to Unite Voices, hosted by Katie Green. Real stories from the people behind today's most innovative experimentation programs. No fluff, just wins, failures, and the lessons in between.
00:00:53
Speaker
Kristen, thank you so much for being on Unite Voices. We're stoked to have you. I'm obviously personally a fan of your work. You and I have talked about that previously. But before we dig into the nitty gritty details, I want to give you a chance to introduce yourself to the masses listening. Great. Well, thanks for having me. Excited to to be here and chat. So name's Kristen, behavioral scientist, CEO of Irrational Labs. And basically, we think about decision making. So we study the psychology of decision making, and we apply that to all sorts of things in the product and tech world and sometimes outside of it. um so Started Google's behavioral economics group, helped run that for three years.
00:01:31
Speaker
um And then now we work with all these different types of companies from finance, health, kind of random other companies with behavioral economics type problems. ah And we help change behavior for good. The change behavior for good good is what I want to attach to really quick.
00:01:49
Speaker
I watch all your stuff on LinkedIn and you do really good content there. So plug for your LinkedIn for anybody who's not following. Great place to follow. You have such good
Effective Product Integration into User Lives
00:01:58
Speaker
content. One of the things that I want to ask you about is obviously psychology and product, huge relationship there. What are some of the biggest differences in your view between a good product and one that actually changes behavior, comma, for good, right? Can you tell us a little bit more about the differences between what is technically good and what is actually moving the needle?
00:02:19
Speaker
So as a behavioral scientist, I become more and more pessimistic about human behavior change. It is really, really, really, really hard to get someone to do something different. The most likely thing that I will do is the same thing I did yesterday. So the biggest predictor of if I will exercise today is if I exercised yesterday, i didn't exercise today. i Yes, I did not exercise yesterday. I will likely not exercise today. ah and so sadly, the status quo kind of runs our life. And so for a product to be really good or great, it has to acknowledge that it's hard to change behavior. And the way to do that is not to kind of go light. yeah It's not about telling me how cool your product is. It's not about kind of assuming that I will discover the new feature that you just launched. It's much more assuming that actually that the thing lowest on the user's priority list is discovering your feature. And so you have to build the feature so that it really intersects with my life.
Case Study: Granola App's Seamless Integration
00:03:14
Speaker
you know
00:03:15
Speaker
Many folks use Granola. I think the big unlock in this was that it has an alert when my Zoom meeting pops up for me to start a Granola. I don't have to go to Granola and say, i want to start that meeting. So when you start thinking like your job is to help the user be successful and really intervene within their life versus them come to you, discover the thing that they want to do, find the feature, know how to use the feature and be successful with the feature, that's kind of where products have good intentions but fail is that they assume the user will just do the thing that you want them to do. And and great products go to the user and make it easy. Love that. I use granola. Yeah.
00:03:57
Speaker
Plug for granola, everybody. i think granola is like the best thing since sliced bread. By the way, actually it's ah just granola as a transcript app is pretty bad as because it doesn't actually tell me who's talking.
00:04:10
Speaker
So if you go to most of these other transcript apps, it will tell me different. It says them, which is crazy, right? It's like, i actually want to know who is saying the thing. So despite it being actually kind of lower quality in those weird ways, like it's great quality on summary and transcripts, but kind of lower quality in the speaker thing, which is so critical. We love it because it's so easy. And so Granola can like have a kind of technically worse product, but a way better product because they fit into my life and made it easy for me to get the transcript, get the summary and share it with my team in a way that is like more automated than other apps. I think that's such a good point.
Behavioral Theories in Experimentation
00:04:48
Speaker
It's it's the ease. And so in experimentation, right, that that's what we do. I feel we often assume that our users are rational thinkers. That's not to say everybody on the internet isn't a rational thinker. My gosh, I don't want to assume here. But with experimentation, right it's...
00:05:07
Speaker
we learn some of the wildest stuff. Like you don't really care about the technical elements because you are getting the ease. Is there anything in your work that i know you kind of disprove the rational thinker a lot, rational user, whatever we want to call them here. Is there anything in your work that really shifts how you approach what might be a hypothesis, right? Because that's what we deal in is we're assuming we're we're taking data, creating hypotheses and trying to create data backed assumptions. And in your work, is there any shift with how experimentation comes into the user behavior? So when we think about experimentation, we're thinking about what is the theory that we're testing. So we try to think about the model of human behavior and say, what would motivate somebody to do anything? And you know you can kind of have different ideas there. But the ideas, especially as a behavioral scientist, we lack those ideas in a theory about what would change. So you can say, people aren't doing this because
00:06:09
Speaker
the value isn't high enough. And so your theory there would say we need to either make the value higher make the benefit and the value more immediate.
00:06:19
Speaker
So can say those are our two theories that we have. Either the value is not high enough or it's just not close enough to the action we're asking the user to do. It's a very, like we'd say the word, present bias. So we we think about what are the psychologies motivating people, and then we build our experiments around them.
00:06:36
Speaker
And this is different because if you go to a user and ask them why aren't they doing something, they will respond in a very rational way. They may say, like um it's too expensive, which by the way, it could be.
00:06:47
Speaker
um That may not be the core reason why they're not taking action right now. They could say things like, um the feature doesn't work for me and my team. That may be true, but it also could be true that the value isn't there fast enough, good enough for them to try it. So they're not going to give you all the information so that they'll actually get the benefit that your team can provide.
00:07:11
Speaker
So you as an experimental person would say, what is the theory by which we're working with? And somebody giving me qualitative answers is one part of the data collection to inform that theory.
00:07:23
Speaker
As behavioral scientists, we think psychology's understanding that kind of core underlying motivators are the other part to that informs the theory that drives the experiment. Well, that's the piece that I think experimenters struggle with the most are the invisible forces, right? The elements that we can't measure. I think as an experimenter myself, I love black and white data. I'm like, but numbers say this is better. I'm curious if there are any invisible forces that you kind of you You gravitate towards is the word I'm looking for. Like what invent what invisible forces do you gravitate towards when you're looking at the behavior of users? And then as a follow-up to that, which I want to dig into afterwards, is the AI of it all, right? Everyone's talking about AI. the pieces that you're talking about –
00:08:13
Speaker
ah AI will will never have EQ. These are the elements that are hard for experimenters dealing with black and white data, and it's also hard for AI. So I want to make sure we're covering both of those elements when we're talking about those invisible
AI and Behavioral Science Intersection
00:08:27
Speaker
forces. So when we kind of say invisible forces, sometimes i want to clarify for folks, it's like the things you can observe. The things I can observe is how many clicks it took me, or if I am answered a question a certain way, you know that I'm an enterprise buyer versus not, or if I'm a repeat user. So these are the things you can observe, and then you can kind of make some guesses based on that. When we think about the non-observables, we can think about confidence. So am I confident using this product? If I'm not confident, social norms will help me make a decision. If I am confident, like I've already bought this thing multiple times, telling me other people buy it is just not going to work.
00:09:04
Speaker
um It's really hard to understand if someone's confident or not confident in the decision-making process just with observables. um Uncertainty is another thing. it's If you are uncertain, you delay a decision.
00:09:18
Speaker
So if you have uncertainty in the purchase process or the usage process, people just may not take action. Well, how do you measure uncertainty? Well, it's very hard to do. And so you kind of have to make some level of understanding is do people have uncertainty? How much? And then how is i as a product team can I eliminate or mitigate this type of uncertainty?
00:09:37
Speaker
So that, and by the way, another unobservable is perceived friction. So I understand that if I add a click, or or like a button step, right less people will continue. That's not always true. Sometimes friction helps increase conversion, but many times more steps decreases conversion. Now, what if somebody thinks there's going to be more steps?
00:10:01
Speaker
So that's really hard to understand. Like if I think this is a hard sign up flow process, I'm just going to give up. No, may not be. But if I think that, that decreases my likelihood to continue. Again, hard to measure. So when you're kind of a behavioral scientist, you understand that all of these forces are acting on somebody as they come across your product, either on the top of the funnel, mid or at the end, that could be driving them to behave in a certain way. And your job is not just to understand the observables, but also the unobservables. and With AI coming into the scene full force, right, for many years now, but it feels like every day there's a new AI product like granola, like whatever it is. And ours is a prompt-based experimentation. We call it PBX. And it's the full lifecycle of testing from ideation, build, analyze, to ship. And it's all done through AI agents.
00:10:52
Speaker
With behavioral science, how does the intersection of the digital environment with behavioral science work with with emerging AIs? Is there anything that you've seen that's successful, a big failure? How how are they working in the industry today?
00:11:09
Speaker
Yeah, so I'll answer basically ah ah maybe a slightly different question, which is like basically um how the kind of two two things here. One is when we're thinking about ai in general, it it actually serves as the same type of problem that most product people have always had, which is you have to get somebody to do something. So your chatbot, your you know magic feature needs to drive a user to do something. and I think we can kind of forget that in the AI world because you you don't you don't control as much. And so if you don't actually control the full user journey, you may forget kind of the core principles. And in our world, it's like, what is the behavior you want someone to do? So if you're engineering a chatbot and you're trying to get it to, at the end, the user does a thing like submit or accept something or
00:12:00
Speaker
ask a question. In an eval world, you say that is kind of the success metric, right? That is the thing. So in a kind of when you're designing AI products, you still need to have the thing you want someone to do. And then that is measured. And you say, can my AI product get me to do that?
00:12:17
Speaker
which is kind of where the theory comes in. So designing AI products is not that complicated in this in the way that you just go back to basics and you're like, look, look, I want to get a user to do a thing. How do I get a user to do a thing? And if you're testing the different prompts, system prompts to do it, great.
00:12:31
Speaker
If you're testing different ways to design the feature page, great. um so So I think, and that kind of comes back to like what is good. And in our Interrational Labs, we think a lot about kind of aligning teams and saying like, what is the thing that we think is good? So if the chatbot's responding in some way that doesn't increase trust. That's not good. But we, as a team, have to align that we want trust as a variable that we would measure. So there's kind of a level of agreeing on what is good that teams haven't had to do before because the output is so variable.
00:13:06
Speaker
And normally, the output is finite. And you can have a QA. And you say, is this good or not? and it's very clear to folks. I think the messy middle with designing AI products is just not clear many times what you're going for. And you have to actually predefine these categories, which is kind of what evals help folks do.
00:13:22
Speaker
And this is more for my personal curiosity. I'm like, I don't know if it'll make it to the pod, but I am curious. Are there AI tools that are focused on behavior and psychology right now? Like, are you using any? I'm i'm mostly curious.
Adapting to AI-driven Work Environments
00:13:40
Speaker
Just what what pieces are you using today? If any, do you think there is a space for AI in your field, specifically talking about psychology and user behavior?
00:13:51
Speaker
So a lot of our work is is really inserting psychology into that the idea of like behavior change um into different products and services versus kind of habit change, which is potentially more like personal behavior change. So kind of separate the two. um I think in in our world, behavior change is everywhere. Like if I want to just get you to do anything, the theory is I have to change your environment of decision-making.
00:14:20
Speaker
So I need to really just, instead of pounding over the head and saying like, I want you to do this, let's say you're a B2B app and you want to add, you want the admin to add some setting that unlocks usage for everyone. That's your goal. Like I don't need to bang you over the head and send you five emails. I need to change the interface of the app.
00:14:38
Speaker
I need to get you to do something different. And so I think fundamentally AI has changed how people work. Like ah the the the how I get work done has changed. It's like ever my whole workflows are changing. So at some level, like that is the intervention in with AI is that every single person is going to a different app than they did two years ago, five years ago to get work done.
00:15:06
Speaker
you know Our team is is figuring out how to like all work in GitHub successfully. That is very difficult. It's changing the function, the environment of our work, which means our output will be different because the functions are different. So um I don't think there's a particular, like you know this is the app that does behavior change well now that AI is here. It's basically AI changes the structure of our work fully.
00:15:29
Speaker
And the how we get stuff done is is changing, which means our behavior will change. Okay. that's i I'm very much enjoying that answer and I feel like I want to underline and highlight it and cut it for my own social media because i think that the environment is something experimenters kind of get lost in where we're really focused on the minute changes we're trying to make the habit change. Okay, what are we doing to change the environment, to influence the overall environment?
00:16:00
Speaker
behavior is, is real that's just a really cool snippet. And I'm going to carry that with me for a long time. um But I do appreciate your like, yeah, that there's no AI that's like really doing this right now. That's not really the point of what we're doing.
00:16:14
Speaker
I absolutely love that. I think the point of what you're doing is something that I personally love are your videos on LinkedIn. Can you tell us a little bit more about the teardowns that you do and what you look for when you're looking for some of these insights that you're generating and sharing with people at large, that's how people can learn from you in such a massive way is just tuning in to those videos you do. Can you tell us a little bit more about what you're looking for when you're kind of – I want to say finding your victims, your targets? Yeah.
00:16:45
Speaker
But I say it with the most utmost respect.
Behavioral Teardowns of Apps
00:16:49
Speaker
So these these videos are basically teardowns of apps or products that people normally use. So you can imagine if you're using these the Granola, if you're using Spotify, i go in and look with kind of this behavioral science lens and say, how would a behavioral scientist evaluate this app and product?
00:17:09
Speaker
um The way I choose them is just the ones that I use. And so it is way easier to have an opinion on an application if you've used it before. And in fact, anytime I'm doing a call with somebody new, if I meet them and they say i work on this app, I will literally you know off Zoom, like download it quickly so I understand what the heck this is. A lot of times when we're talking about an app or an experience,
00:17:36
Speaker
it's in your head and and as behavioral scientists we are like ruthlessly tactical. We just want to see the thing. And so the teardowns are really this like fine tooth comb of like going through every single step that it takes to get you to the thing you want to do in the app. So You know, in when looked at Descript, Descript is like a lovely kind of video editing, podcasting, editing sort of thing, um and done a couple teardowns with them. And one was just using their AI tools. And so if I want to be a successful video editor, i'm going to have to figure out how to use their AI tools. Well, how would I do that?
00:18:10
Speaker
And and so I think when you're going at things with a behavioral science lens, you're basically looking for the small details of life. You're looking for the very small things that change how I would behave if the button was different or the call to action on the top of the page was directing me in some different way.
00:18:27
Speaker
um As an example, you know Google launched their AI pricing very early and did a fun teardown of that, which you know for probably a Googler who's used a lot of AI tools, this may have been very intuitive for them. But for a normal person, the way they framed it was, you know had cognitive overload, um was had actually like a weird, they had the most expensive product in the middle. I've like never seen that.
00:18:51
Speaker
And then you just didn't understand the value prop. And so you're like, how would anyone go through this flow and be successful, which is what Google wants, to purchase something you know at the end of it So using these core psychologies can get you to these unlocks of like, whoa, how would anyone be successful? Or what are they doing that is like so successful that we should copy it and pattern match it? Which I did whisper flow onboarding. And whisper flow onboarding is like, I don't think I've ever done a teardown with no kind of like feedback. It was all positive. um And so you don't just have to learn from things that are bad. You learn from things that are are good of what are people doing that drives you actually to complete the flow. By the way, Whisperflow is like a really hard thing to do. You have to like download the app. You then have to like set up this voice. You have to like change your settings. Somehow they get you to be successful doing this. Let's study how they do that.
00:19:44
Speaker
I love your teardowns. ah It brings me to a point that i I think would be really cool if you ever wanted to try it, which is Chameleons PBX. You can use on any website. So if you're ever like on whatever it is, Spotify, I'm just going to make them our target today. If you're on Spotify and you're like, oh, no, I really hate – this and you explain it and you're doing a video and you want to see what it would look like as a response to your teardown, you can go onto any website and it just does it on your machine. Obviously it doesn't like launch the test. Right. But it'll show on the website. i actually did a funny one for Alaska Airlines on, um I just fly Alaska a lot. I'm based in Portland, Oregon. And for international women's day, i was like, Oh, we're still making like 15% less than men. wouldn't it be great? And I was like booking a flight to somewhere and i was like, wouldn't it be great if we got 15% off just the whole month?
00:20:37
Speaker
I'm like, I kind of want that. So I just went on PBX and I was like, make this 15% off and make the branding look like International Women's Day. And it turned it into this like purple and blue, like 15% off for women travelers website. Yeah, it was silly, but it was just like trying to prove the point that you can do anything in PBX to show your point. And if you ever, if you ever want to play with it, just let me know.
00:21:01
Speaker
ah Yeah, I'll set you up with it. Um, but you can go in and say, Oh, descript like I Google, why is this in the middle? Move it to the right. And PBX will move it in less than two minutes. Um, it's really fun for making your point land, but I want to go back to the one thing you said, which is that, uh, the, you learned from what was, you learned from what is good as much as you learn from
Cultivating a Culture of Experimentation
00:21:21
Speaker
what is bad. I think I see often in your communications, the fail forward mindset, um you know, there's, ah you're coaching really high stakes people. Like I just to call it what it is, you're coaching really high stakes people.
00:21:35
Speaker
A fail forward can be really scary. I'm curious if you have any knowledge and wisdom you can share with people listening on how to create a culture for failing forward and prioritizing that kind of learning.
00:21:48
Speaker
I think a lot of experimentation culture, comes from having early wins because you understand the upside of what you're doing.
00:22:01
Speaker
So when we're working with teams, we're not going to test the most risky thing in the beginning. We want to prove the idea that some experimentation can drive an outcome.
00:22:13
Speaker
um So that would be kind of the first hypothesis is you're going to take a place that isn't doesn't have as much downside risk. A lot of surfaces within products have downside risk. Whereas if you mess this up, you will put revenue at risk in a way that is fundamental to the business.
00:22:32
Speaker
We think people should start with more of the upside. you It is likely that if you change you know a button structure within your signup flow, you try four things, one of those things would likely drive higher conversion.
00:22:49
Speaker
Does that put a lot of revenue at risk? No. Is that fundamental to the business? Less, right? If you change one thing in the onboarding, you're not going to break or break the company. That said, people can get a high confidence self-efficacy that they can do some of these small changes that have bigger wins.
00:23:05
Speaker
um So one is just we don't put a lot of the business at risk for small experiments. The second is that we do a fair amount of pre-testing out of product.
00:23:16
Speaker
So if there is a big swing where we want to fundamentally change our pricing, we want to like switch how our feature set works and redesign kind of the the overall value proposition. These are big things. Doing A-B testing to this as like the first thing you do is risky. So instead of just doing qualitative work, which is what most teams do, we do what we call is out-of-product testing where We have four versions of this. And we recruit people from an online recruitment platform, let's say 1,000 people. And there's four versions.
00:23:49
Speaker
250 get one version. 250 get another, cetera. And then we ask them questions about like their likelihood to do the thing that we're trying to get them to do. um We also ask them, you know did they understand it? All these other things. But like it's mostly like, you know would you give us your email address, which is more of a, we call it dependent variable that would drive our prediction of which version would work the most. So we're basically doing out-of-product A-B testing.
00:24:15
Speaker
And we want to simulate the environment as much as we can, which is actually fundamentally so much easier now that we have AI, that you can just mock up websites and people can actually like play with the website.
00:24:27
Speaker
It feels real. And so the idea that you're comparing four different versions of these sites, you can kind of get a better intuition on which one would work. And so for folks who are like nervous on A-B testing would think about taking it out of product to get the confidence to launch something in product where you still would do some type of experiment. Yeah, i I often have this conversation, right? As a practitioner, I've been doing experimentation for many years. And every time I say the number of years, I feel older. Um, but but often the conversation is, okay, we're so afraid to even fail that like there there is too much risk. There's way too much risk associated with this. And I, I often say the biggest red flag is when the risk to experimentation is just not experimenting at all. Um, so you're kind of underlining that and I want to take this chunk and just like like plaster it on a billboard and just be like, you can start with what's good, right? You can start with where you're going to have a less amount of risk. It's a really calculated strategic way of approaching growth that is naturally de-risked, but people hear the word experimentation and they go, oh no, but what if it loses? So I wanted to make sure I asked you about that because it is just something that I hear ah time and time again. um
00:25:45
Speaker
i think that but When you're coaching teams, and i don't know how much you can share, of course, don't don't give us i don't give away all of your precious gold coins here, but when you're coaching teams, is there a certain trend or characteristic or trait that you can identify with the teams that do have some of the best growth practices and the best cultures of experimentation? What are some of those traits you find that maybe we aren't thinking of beyond you know really healthy data data rigor, right? Yeah. I mean, the the fundamental thing is, do they have the logging data to to be able to tell um something works? And can't everyone in the company pull that?
00:26:28
Speaker
So you know we we work with some companies that can't, and we work with some companies that can. And the creativity of folks who can the marketing manager can go pull the stats on how things are working, their creativity is just much higher because they're able to kind of understand and think with data about what is actually going on.
00:26:49
Speaker
So like base level would do more logging and from the start of the company, which is hard for startups to to prioritize, but but required.
00:26:59
Speaker
And then I think the second one is that ah these these teams understand that they might not be right. that That, like, there is upside to things. And they kind of say, like...
00:27:15
Speaker
it It would be really, really surprising if the first thing you did is the optimal of all the things you could ever do in the future. So the first time your designer designed this homepage or the onboarding or the dashboard was, opt it would just be really shocking. And so they have more of a humbleness to the world that says it is likely that one or two other things could be better. now Sometimes there's not a lot of upside there. Like if you've you know Google optimizing their homepage, there's just not a lot of up upside there. They've already done it. So there are times by which there isn't as much upside. Most of the time, especially if you're a new team, new product, new thing, there is upside to be had by changing a few things.
00:27:59
Speaker
So the teams that get this are just a little bit more humble about but about what they know about the world and their consumer. And so they're interested to try things. Okay, humility is is a big one. Got it, got it. No,
Balancing AI Uniformity with Creativity
00:28:13
Speaker
that's great. I realize we're ah going a little bit off a tangent here and it's something so important to me, so I want to make sure I get this question asked before we run out of time. the The words best practices… How do you feel about those words? Because as experimenters, we're like, boo, boo, best practices. They're just somebody else's guesses. And even if it's correct in one, going to use word environment because that's the word you used earlier. If it's correct in one environment, who's to say it's going to work for the other one? What about personalization, right? So when you're thinking about consumer psychology, how does the how does the term best practices feel to you? Yeah.
00:28:51
Speaker
So in some worlds, I like it to the idea that we don't want people to start from scratch. So a lot of times you kind of can come in, and and I like critique IDEO a lot for this, where it's like it's a blank canvas. You're like, brainstorm as if no one's ever thought about this problem before. And as behavioral scientists, we always start with literature. We're like, somebody's thought about this problem before, and there's likely a meta-analysis on the question. Let's go find that so that we're starting from a point of insight.
00:29:17
Speaker
Let's create hypothesis from the point of insight. We have now an insight-informed hypothesis. i'm We're not saying it's right, but we have an insight-informed hypothesis. We didn't have to like stand in a whiteboard together and brainstorm that.
00:29:29
Speaker
So we like that humbleness. this is if We're not the first people to think of the problem. um And ah you know many times what that looks like is, like look, if we're trying to do an e-commerce nav bar, we should probably look at people that we know do experiments, Amazon, Wayfair.
00:29:46
Speaker
Booking.com, we don't want to take a travel and put it in full e-commerce, but like these folks have experimented with NAV. Where do they land? ah And so getting generally like, if you know that a company is experimenting and they are in your domain and it's a you know small piece of the puzzle, it is likely you get some hypothesis from looking at other folks to see what they've done. We would not take a random startup and get ideas from them because they've not done experiments. They don't have enough sample size to do this. Larger companies do.
00:30:16
Speaker
This is like Google's homepage is just just, it's been experimented on. If we were to start a search company, we would not start from scratch. We would start from what has been experimented on. So there are times where you should do that and then times where you basically say, look, context does matter. I'm not drag and dropping something. It's my user's different. The context is different. i I need to have some clear hypothesis. The other thing is if you just reinvent the wheel for every single thing, you just won't have enough time to focus on the big things that do move the numbers for you. And so we are not opposed to saying, look, there's
00:30:52
Speaker
10 things you could focus on. Let's pick eight that where we just are like, we're just going to start from a baseline that we believe works. And the two that is going to drive your business, we are going to think very deeply about and and understand more.
00:31:06
Speaker
Okay, cool. I mean, it's refreshing to talk to somebody who's not like, boo, best practices. Because that's – I obviously live very firmly in the experimentation world. So this is great. I think – I have one more piece that I want to bring up to you because it's something that personally – don't know. It just interests me. I want to say irks me almost. But in the world of AI, AI is designing. AI is coding. AI is doing everything. Are we – is reinforcing the best psychological behavioral like requirements of users, right? I don't know exactly how to say that. But what I'm asking is if you if AI is designing every website, is every website going to look the same? And is it going to function the same? Whether it is outside of the context of industry or not, like do does AI absorbing best practices and what AI knows of behavioral psychology, is it just
00:32:04
Speaker
reinforcing the same things and then we don't get creative and we don't have these really unique use cases for specific needs. I'm just curious your take on the use of AI and i feel like there's a term for it that I don't know, but it's just the AI whitewashing of the of the same function, the same product. Is everything going to be the same if everything is built with AI? Yeah.
00:32:26
Speaker
It is ah is a good good question. um And I think as as more people become builders and more folks make things, um there's likely that they're going to take the default recommendation from AI and not necessarily change it. Now, I think in the world we're in right now, the default recommendation from AI is still worse than a designer's.
00:32:43
Speaker
it will change. It could get better later. so i don't I don't think we should hold on to that as like ah a statement of like, it won't change. So you can imagine a world where the default recommendation is actually better than a designer's, but it's very bland and vanilla.
00:32:57
Speaker
I tend not to believe in that world because if you have more noise you're going have more builders making more things, the world's going incredibly noisy. People are going to need to stand out. And so with kind of a noisy world, people do a lot of things to stand out. I like to compare it to Vegas, right? Vegas has no zoning regulations on signs.
00:33:18
Speaker
And so signs are crazy in Vegas because you just want to be bigger and better than the other people next to you. So in a noisy world, people do insane things to stand out. Now, most cities have regulated signs, so you can't do that.
00:33:33
Speaker
But if you don't regulate signs, every city would do that. So the idea that I think human creativity could get higher because we need to basically be the best flyer on the flyer pole for people to read us. I absolutely love that take. I think that's, and to bring it full circle, that's where experimentation comes in, right? Is to be the biggest and the best, test it before you launch it. but' it's like the biggest thing here.
Integrating Behavioral Science in Product Development
00:34:02
Speaker
The last question I always ask on all of these episodes is what is the Monday morning advice, right? I'm assuming somebody is listening to this on a weekend, but somebody who wants to integrate more behavioral science into their roadmaps, into their product builds, whatever it is, what is your recommendation for how they would do that starting tomorrow?
00:34:23
Speaker
Um, hire us. Just kidding. that's ah That's a joke. That was a joke. like i I set you up for that one perfectly. i mean, that's a, it's a funny, no, we, we think most people um have some insight about everyone's a little behavioral scientist. We're all going about the world, kind of observing things.
00:34:42
Speaker
And I think probably that's the double click that you would do is you'd notice more things. You're just in your environment and you're kind of noticing what is hard, what is easy. You're noticing where social norms come in. You're noticing what,
00:34:55
Speaker
how the coffee shop asks you the question of, do you want a small or a large? So as soon as you start noticing things, you notice that the environment of decision-making is changing your decisions.
00:35:05
Speaker
And a lot of times we think that, you know, information changes our behavior. If I just knew more about how good a thing was or, different details, like financial literacy is a nice example. If you're to change financial outcomes in the US, we would, we could think you could change financial literacy, get just more people to know about FICO and compound interest. Turns out when you teach people these things, it doesn't actually change their end behavior.
00:35:29
Speaker
It doesn't actually make them create a savings account or pay off their debt. The things that do are the small details of how a bank designs their credit card payment as the minimum, you know the first one or the last one. Is it defaulted or not? And then you can really change behavior. And as soon as you start noticing those small details of design that could change or influence your behavior, you'll kind of bring that back in to your work life. We've seen this consistently as we we have this boot camp that trains folks in behavioral science and we we really see them changing the lens on how they look at their external world and then bringing that back into work.
00:36:03
Speaker
So you're saying it's okay if I put little behavioral scientists in my LinkedIn? yes That's what I'm hearing. Yes, you got that. I think that's an excellent, excellent place to end. ah Thank you so much for being on the show. i can't wait to hear what people are learning from you from this and from following your LinkedIn because it's the place to be. ah but yeah, thank you so much for being on Unite Voices.