Transcript
Speaker: So coming back to you know your question about like ego and wanting certain stuff to win, it's a reality of digital product management and you know marketing and all that kind of stuff where there's going to be multiple voices. And sometimes they're all right. From the place that they sit, they're all right. And I think the biggest challenge for A-B tester is sometimes letting a good idea go or managing the multiple expectations and kind of you know maybe diluting the strength of the hypothesis in order to make sure that everything kind of satisfies everybody. so yeah, it's tricky.
Speaker: Welcome to Unite Voices, hosted by Katie Green. Real stories from the people behind today's most innovative experimentation programs. No fluff, just wins, failures, and the lessons in between.
Speaker: hi welcome to Unite Voices. I'm so excited to be joined today by Linda Bustos. I'm Katie Green, Principal Advocate at Chameleon. Linda's joining us today to share all of her UX knowledge. She was nominated for the UX Expert, or as we call UXpert category, of our Experimentation Thought Leadership Awards this year. And Linda has just a huge depth of knowledge on user experience. She also runs e-com ideas, which can be found on LinkedIn, as well as her website, where she posts really cool ideas in this space to share with fellow experimenters. So do follow her there. But for now, Linda, introduce yourself and tell us a little bit more.
Speaker: Well, thank you. Happy to be here. So yeah, I um i've run a project called Ecom Ideas, which is a website that's totally free for anyone to use who's in the space. And it's just a massive collection of innovative features and functionality inside e-commerce design patterns, all cataloged by you know type of page and type of feature experience, that kind of thing. So Yeah, basically, um I've spent quite a long time in covering the e-com and UX and AV testing space across various roles in different companies. And yeah, I'm just very, very passionate about e-commerce experience, web design, usability product.
Speaker: um So yeah, happy to nerd out on that today. And for people who don't know e-com ideas, it's really, i feel like it doesn't come from a total background of testing, right? Is is that fair to say?
Speaker: It comes from your background of UX and like, right? Right. Yeah, i think i think where it falls is that um everyone in the experimentation world knows that you can't just look at a site over there and go, that's cool, plug it into your site and call it a day.
Speaker: um You need to have some rigor and discipline and you know critical thinking around everything. And even stuff that works on your own site might not work six months from now once other elements enter the mix. So Yeah, I think that, you know, my job, I feel like is just to find and surface the ideas.
Speaker: It's up to you to decide whether it's a good idea in general. Some of the stuff is a little controversial. It's a little wacky. And that's where I, you know, I kind of feel like I i am creating content for different personas. There's definitely the experimentation and the people that are doing responsible for P and L and revenue and performance and stuff like that. There's also brand teams and that's a complete other animal, you know, like we want to represent the brand digitally. We want to do what's cool. We want our website to look like a fancy magazine or, you know, a video game in some cases. So I try and just find the ideas, put them out there.
Speaker: no comment. like It's up to you whether it's a good idea or a bad idea. I try not to post stinkers, but you know sometimes sometimes I think something's great. And I have a little blooper reel. like For example, um recently I posted something where the badges, when you're scrolling through the PLP, kind of were slipping behind the product card. So you would see like a badge and then it would sort of slip behind.
Speaker: It's hard to describe, but like That totally looked like one of the edgy, you know, design brand forward that you will see on like indie fashion sites and stuff. They really push the boundaries of design. I posted it. I'm like, look at this theme effect. And people were like,
Speaker: I'm pretty sure that that's broken. i don't think that passed QA. And I'm like, sometimes you can't tell because, you know, designers are really taking big risks and it's always fun to find those in the wild. And i love sharing that. So circling back to answer your question, is it just for CRO? Well, I think CRO should be involved with any design decision or at least, you know, weigh in with an opinion, have some kind of guardrails around stuff. But in terms of, you know, how do we bring digital experiences to life?
Speaker: There's a lot of creativity out there. Well, I think as an experimenter myself, right, it's one of the biggest things I struggled with is I'm staring at this page I've looked at for three years, right? actually That is not an exaggeration. I've been working on the same exact landing page for three years. And I am out of ideas. It's just where you end up as a human, right? You look at the same thing. I call it page blindness, right? I think multiple people probably use that term. But it's like you're looking at the same thing. You run out of ideas. I just wanted to call out like because you have such a robust experience in e-com, anybody who's dealing with that, who's been looking at the same PLP, PDP, navigation, I did see... a couple of your recent posts on navigation that were super interesting. um But like, you know, they should look to your ideas because we struggle so much with ideation and, you know, AI is becoming more involved in ideation. But having someone like you who just shares ideas, sees sees things out in the wild and says, this is why I think this is cool. I feel like it's really invigorating and inspiring for experimenters. So that's why I wanted to make sure we said that piece, which is like, no, this is not technically a test idea platform, but it can be.
Speaker: And that's how I would expect ah like our industries to collide immediately, right? I think there's more than one way there, but I just wanted to make sure I said that here at the top that experimenters should be looking to people like you and e-com ideas because we all know We just think differently. We have different perspectives. We bring that to the table. That's what makes CROW so exciting and cool is we're able to come up with these different ideas. So your observation without prescription, I feel like I've said seen those words ah when it comes to you, is it's a really important philosophy to take into ah conversion rate optimization in general. Yeah.
Speaker: Great. So thank you so much for letting us dive into that for a quick second. I realize we spent more time than we probably intended to on just talking about what Ecom Ideas is, but I wanted to make sure people understood the context. um If I can, I'm just going ask you one one quick question before we get into, and I realize this wasn't in our prep discussion. So this is just part of the off the cuff conversation that i I ask is when you're coming up with ideas, do you ever face that all websites look the same to you kind of thing? You're so good. I saw your Bulgari, your most recent post on Bulgari super navigation. And another one, I think was American Eagle. They had like a mini navigation type thing. And you were posting some of that I have personally struggled with navigation tests so many times. And so I'm curious, do you ever run into the issue that many experimenters do when it's like, oh my God, all e-com experiences are just the same?
Speaker: I don't know. Yeah, yeah. I think I could probably answer this in three different ways. um I'll start with the navigation thing. So ah I think in that post that I did today that you're talking about the Bulgari, put something that I say often that desktop has more fun.
Speaker: And we know that the majority of the e-commerce experiences are mobile, like the traffic is mobile and mobile first is a real thing. And it's an important thing. But once you get to a mobile surface, things like mega menus just become a lot less sexy. They become a lot more, you know, a stack of links.
Speaker: And it's risky to start putting what I like to call navendizing in there, like, you know, blocks of images or some people put animations and video or, you know, how do you handle your hierarchy and your taxonomy? Is it an accordion with another something of links and that sort of slider, you know, go three or four levels deep and back.
Speaker: um So mobile navigation is kind of vanilla. One thing that really surprises me is a lot of brands just don't give it a visual hierarchy.
Speaker: So it's the same set of, you know, whatever font you've chosen, whatever color, everything is all the same. So there's not like, you know, is there a featured category? Is there, you know, somewhere where you want to direct people? So I do find like some people are or people Some companies you know do a little bit of twist on that. They push some actual product recommendations in there. you know If you're a small catalog and you have like only a couple of hero products or something that 90% of people come for, then that's a great strategy. That doesn't work for everyone.
Speaker: We can have banner blindness in the menus if you put like you know two or three like featured categories. It's just like the homepage. You're sort of like... taking up real estate, hoping to funnel people into a category and who knows if that works. So a lot of testing opportunity for menus, but they do tend to look the same.
Speaker: um And in terms of just websites looking the same, especially since we're in a Shopify world, right? Like a lot of apparel sites just have a regular theme and they don't add a lot of their own stuff on it. You get the theme up, it has the same standard. The PLP looks the same. Your PVPs essentially look the same.
Speaker: um Homepages are where you see a little bit more brand storytelling. They're like more, you know, like brand specific content. But the rest of the site and of course, checkout carts and checkouts tend to look very, very much the same. So it is harder for me now, especially since I've been collecting stuff for e-com ideas for three years, just to actually find stuff that's truly novel and truly different or that truly solves a problem.
Speaker: um Websites start looking the same to me. But I take for granted that if you're not looking at 100 different websites every single week, that something that I think, oh, I've seen this eight times or 100 times might actually be the first time that someone else has seen it.
Speaker: um So for me, it's a little bit tricky ah to try and pick featured ideas because, you know, they Sometimes something shocks me or I think that that's really cool, but I know that it's not an actual value add to the user experience, right? It's just that this designer did something different in a very unusual and uncommon way.
Speaker: ah Maybe that's worth sharing, but um yeah, it's, especially in certain industries, like themes tend to look the same. You go to like snacks and CPG and beverages and stuff like that. They get really creative with colors and,
Speaker: you know, theme style and stuff like that. But they all kind of have a similar kind of vibe to where they're a little bit more cartoony. They're a little bit more, you know, and that's great. You would never see that on a, on an apparel site. No, this is great. And I think,
Speaker: I would like to talk a little bit more about the ai piece of it all because you're kind of i in the Shopify world. i i realized like the Shopify checkout experience is so homogenous and like they're saying this is the most streamlined Shopify experience for everybody, right? And it's starting to get more personalized as you get more advanced, right? It's kind of how I'm seeing it.
Speaker: But I want to talk a little bit more about AI because more and more people are using ai to... design to vibe code, whatever it is. Are you seeing any trends in UX as it aligns with the elevated abilities that AI have on websites?
Speaker: So... I am a confessed vibe coder, like the new iteration of e-comm ideas that I launched like a month and half ago, purely vibe coded. built it in lovable. And one thing that I learned about design with AI, especially if you're building a product or anything is like, it's not really a respecter of design systems and it's not really consistent with element, you know, UI elements across you're building a new experiment and it's a it's generative engine and it can kind of like invent stuff on the fly and it just keep a record. Now someone could probably like educate me and say like, no, there's a proper way to kick off a project where you actually canonize what the design system is and enforce it, you know, that it's always used everywhere. But I just found myself with designing product that I had to go around and around around in circles. And once you get enough
Speaker: um chat history, what what AIs and LLMs do is they compact it. So they actually don't keep keep track of everything that you've ever asked or told it or yelled at it for. it just kind of compacts it into a summary, hands it off to the next like batch that you can talk to it. And sometimes it drops and loses context. And that can be very, very frustrating. And that's for designing products. So like if you're trying to vibe code like Shopify theme, you might experience the same kind of um pain.
Speaker: When you're talking about just, you know, using some of the AI tools to do one kind of thing for like an experiment, I think that that's a completely different unlock because you can do stuff, you know, you're working on one little component and that can really empower someone who is more of like, um,
Speaker: CRO is a discipline where you have to wear so many hats, like you have to be good at the heuristic evaluation and understand UX. You have to understand data, you have to understand statistics, you have to understand, you know, like the methodology of testing. And oftentimes, like, I mean, this is my own experience, like working with AB testing programs when I've been doing that consulting. because I really relied on support from a designer and a developer.
Speaker: So not every test that I thought would be a good idea could be implemented because maybe that's not how the data was layered in inside Shopify. And you know the uplift was was not a simple CSS show hide situation. Sometimes it was like, well, we don't have a design system. If I'm working with smaller companies, we don't even have a design system yet. We need to get a designer first.
Speaker: You know, developers didn't want to go and like code something that wasn't going to last. So I think that this um I think that this can empower marketers and business folks and people that kind of have an eye for what they want to test out, but without having um their own coding skills to be able to do that and to be do that quickly and also, you know, create more variations.
Speaker: So I think that that's really good. It's a good unlock. um But. You know, you have to make sure that the AI is keeping within a design system.
Speaker: That's the tricky part, right? That's where the human in the loop is so important in ensuring that the AI isn't doing, isn't out running off doing the things you don't want it to do. So it's it's really helpful to hear your perspective because I know a lot of people, something that, you know, a big part of my job is talking to people and saying, hey, you know, what's going on in your world today? I talk to our customers. I talk to prospects. I talk to people who don't even work with us at all. And I'm like, I want to understand what's going on in the experimentation industry at large. And something a lot of people are hearing is use AI.
Speaker: Okay, how? Right. that They're like, I don't know where to begin. Am I wasting my time with AI? So it is it is really interesting. But the the thing that I think comes it comes down to is that AI enables more people to have access to testing, to design, to UX, like things you're saying. And it still needs the person to implement that rigor. And I think that all comes back down to kind of like your first point, which is like you you have to make good decisions. I can't help you make good decisions. I can show you ideas, but I can't help you make good decisions. um So, you know, it's like training yourself to be able to to do that. But I think what's beautiful about the testing industry overall is that
Speaker: It tends to be low ego, right? We're here because we have a genuine curiosity for what is or is not going to work, right? That's what experimentation is. So I'd love to hear your perspective on people with UX ideas, right? Maybe they're very passionate about these ideas Testing then becomes more how do I prove my hypothesis right versus how do i genuinely learn and fail forward. So do you have any opinions or advice for people who might be dealing with some of the higher ego approach to UX? How do we make sure UX and behavior online is aligned with a culture of learning and continuous improvement?
Speaker: I think you opened that with an important point that people inside the CRO industry generally accept that you have to be curious, you have to let the data speak and you know you just never know. No peaking, no trying to manipulate tests in your favor by not letting it reach the right confidence interval or like stretching for the long time. like Methodology and test design and everything you know is important in that rigor.
Speaker: Not everybody with access to an A-B testing tool um has that background. Anyone can set up an A-B test. And if you're in a smaller company or a less resourced company, you know you might be the marketer or I've been in this position. I've been the VP of e-commerce and you know like I have a lot of ideas, but I need to actually sit down and talk with the team. like There's more to consider than just, hey, want to do this idea because if a if it needs to be developed, and sometimes what you want to test is really a net new feature.
Speaker: So then it becomes a decision of like, do we expect that this is going to be enough of a lift to warrant putting it into a development sprint and maybe knocking off something else important or urgent? you know, in in order to accommodate that in. And then you get a designer that has a point of view, you know designers and creative people and anybody who's really a good thinker.
Speaker: And this goes for like, analytic people to, you know, math heads, design heads, development heads, everybody kind of comes from their own background of expertise. And when you look at it through a different lens, everybody can be right.
Speaker: But then you're dealing with trade offs. So, you know, like a development effort or, you know, we shouldn't do this because this is this reason, accessibility, web performance, you know, stability of your build brand, you know, like I remember really one of my first kind of AB testing sort of experiences or back. I worked with a company called Elastic Path and we ran in house the Vancouver 2010 Olympic store. So ages ago, but we had a challenge with AB testing because we had to answer to the Olympic committee, which is an international committee, which is so um
Speaker: trying to find the right word committed to brand. Brand is fixed in stone and it doesn't matter, you know, if you want to try something or like whatever it has to fit. It has to fit the international guidelines. So sometimes you can even just hit up against constraints.
Speaker: So coming back to, you know, your question about like ego and wanting certain stuff to win. It's a reality of digital product management and, you know, marketing and all that kind of stuff where there's going to be multiple voices and sometimes they're all right.
Speaker: you know from the place that they sit, they're all right. And I think the biggest challenge for an A-B tester is sometimes letting a good idea go. um or, you know, managing the multiple expectations and kind of, you know, maybe diluting the strength of the hypothesis in order to make sure that everything kind of satisfies everybody. So, yeah, it's tricky. it's a real I feel like it's ah a question neither of us could answer in a podcast. You know what i mean? Like, I feel like I almost set you up for failure there, but I didn't mean to. You know, it is it's a very complicated problem. Like, I, as an experimenter, have gone into a room where
Speaker: you know, people who have 10 more years of experience than me are telling me, they're guaranteeing me something's going to be right. And then I'm like, okay, we can test it. I'm telling you it's not going to work, right? Like, because I have thousands of tests in my backlog, this context they don't have. It's just this really interesting kind of all comes back to we all bring different perspectives. But I did, i i kind of want to back up the workflow. Second to last question here, but take a step back on the workflow we're talking about, which is people coming to the table for a roadmap discussion, let's call it, with ideas. So what would you recommend people do to make sure their ideas aren't just flashy and new or adding noise? What guardrails can people put in place before they even get to, hey, crow expert, like I want to test this thing.
Speaker: What can they do before they even get there to say, and here's a really strong case why beyond I feel like it's pretty. what's interesting, um I think that that is a tricky question to answer. Because if you default to what every consultant says, it depends, you know, like everything kind of depends a little bit.
Speaker: But I think I think sometimes things that look like bad ideas actually win, like, I think we've all seen the case studies, or you and your own experience,
Speaker: Like I remember pushing back on a team that I was working with that wanted to put a scrolling ticker at the top. And I just find those incredibly annoying. And I never would have thought that it won, but it definitely won the test to have like a scrolling ribbon across the top. And then we did another round of testing with shoving emoji into it.
Speaker: And for this particular brand and customer, that was a second round of lift. So so I think if you if you've experienced that once or twice where you've seen something that like you really thought that it wouldn't because like this will never work.
Speaker: I think it goes both ways. The things when you think are really great and then you test it and you're like, want we can also go the other way. Things that you think are really terrible. So for that person, once again, circling back to what your actual question was.
Speaker: You know, what do you tell someone on Monday morning? Like, yeah, I think I think it's nice to knowledge share and just show like if you do have experience, like giving a practical example can sometimes, you know, open somebody's mind. Right. But I think the other thing, too, if you don't want to push back and if you want to foster a no bad ideas culture,
Speaker: There's always ways that you can kind of mitigate your risk. You can test something on a PDP that gets medium traffic, right? You're not your best converting thing, not your hero product, but something that could, you know, you can do it on a handful of products so that you can get the test through without introducing too much risk.
Speaker: Or you can do the throttling method where, you know, we'll send it only to one in every five visitors and, you know, try and protect the rest of the 80%. There's workarounds there. um And you can test it in those kind of ways. So I don't like to shoot down ideas.
Speaker: um I like to keep an open mind. But yeah, for somebody who's just bright eyed and bushy tail showing up on Monday morning with a handful of of great ideas, you can always also hide that it's deprioritized because we're super busy with other things. You can always also say that.
Speaker: Defer the decision to later. That's my favorite method personally is, you know what? Let's think about that tomorrow. Yeah, exactly. That's my favorite. No, that's great. I think that a lot of crow people, you like you said, there's all these different personas in crow. And i think for my very last question here is,
Speaker: ah just some of your advice for somebody who might not be in more of the UX space on Crow, somebody who might be more of that, as you put it, math head or data head or engineering head. Maybe they do want to integrate more of some UX ideas into their day-to-day practice. so kind of how I'm tangibly thinking about this is an analytics person who is so focused on goals and, you know, maybe they want to come to the roadmap meeting with more than just, Hey, we really need to increase click-through rate on this button.
Speaker: how How would they, for your final question here, how would you recommend somebody who doesn't daily work with UX? What would be their first step in getting their feet wet with some of the stuff that you work in every day?
Speaker: Yeah. I think usually when people are data driven and start with the numbers, you want to protect the numbers. So there's a KPI that, you know, sometimes there's a KPI that you've been told is is important and now you're trying to figure out, okay, what does that mean and and what can we do there? And then you do have to look at the features and functionality and you do need to not just look at a page and consider what you could do, but like get a little bit more curious, like what are user...
Speaker: actually doing. So you have the quantitative data, which you're seeing in your analytics reports, but can you then also go look at Hotjar or whatever you're you' using for your tracking and then actually watch people. And then that might flip your understanding of like just the metric. And then you might want to layer in, let's show a prototype or let's get some user testers on the call.
Speaker: And, you know, test our hypothesis there, like A little bit more curiosity than just jumping to a conclusion from a number to a diagnosed problem. And then looking at things in context, because once you actually look at a page, then you might start seeing the points of friction.
Speaker: um But yeah, like if you're just a pure numbers person, it's just communication and curiosity and engaging with your designers and your developers. And I think everybody today, like the web is mature. And we all kind of know what usable and unusable is just from our own use of the web.
Speaker: We know that you know everything animated is annoying. We know that page refreshes all the time is not as nice as a smooth transition. We know that um struggling with a mobile menu for, we know we know what we don't like. And yeah, so I think it's just communication, teamwork.
Speaker: Communication and teamwork. That's what we're calling the episode. Period. Communication and teamwork. No, that's great. I just want to make sure because you have such a breadth of knowledge. You've been in leadership positions and having to work with people who are from all over the spectrum of personas when it comes to Crow and UX. Yeah. Thank you so much for offering a little bit of that advice because I can imagine somebody listening. They might be like, okay, yeah, I'm going to set up a one-on-one with my designer. Maybe I don't work with that person all the time. But that would be like the metric for success of this output is if we could inspire one person to have a conversation with somebody they might not have previously just to get a little bit more into one of the other personas that is critical for Crow UX, huge win for us. But Linda, thank you so much for your time. i realize we're at the end of our episode here. um If anybody wants more ideas from Linda, you can follow e-com ideas on LinkedIn. Linda, is there any other places that they should follow? Substack, YouTube?
Speaker: Right now it's LinkedIn and the website. Um, Yeah, there is a newsletter sign up if you want to get in on that. I haven't published one in a while, but once that's revived, that's a third channel.
Speaker: There you go. I tried to get on Substack myself and I was like, whoa, this is really intimidating. I'm going to leave this for tomorrow. And I started that in February. So, you know, yep, I hear you. No, that's great. Everyone follow Ecom Ideas and Linda on LinkedIn for more thoughts on UX from our UXpert. You know, it's great to hear from you. Thank you so much for your time, Linda.
Speaker: Thanks, Katie.
