AI is like a teenager: 100% confident and 70% right. So who fills the other 30%?

Katie Kelly and host Katie Green (double Katies) dig into what AI actually changes for CRO teams, and what it can't touch. You walk away knowing where to automate safely, how to protect the creative and research work that still needs a human, and what to do tomorrow to grow your own profile in the field.

Katie Kelly has worked in conversion rate optimization since 2008, across agency marketing, operations, and growth. She also built Kameleoon's Who's Who, having reviewed thousands of experimentation professionals to spotlight voices the industry tends to overlook.

Key takeaways

 1. Simplify a process before you automate it, because automating a messy workflow just scales the mess.
 2. Protect entry-level talent and diverse voices, since fresh perspectives and creativity are exactly what AI cannot replace.
 3. Keep qualitative user research human; a 10-minute conversation with a real customer beats a synthetic audience for original ideas.

AI does the boring stuff: Why creativity still wins in CRO | Katie Kelly, Fractional Business Partner, Subject Consulting

Most teams blame the platform when experimentation stalls. The uncomfortable truth is that it's usually people and process.

Paul Davidson was a Senior Product Manager at Expedia Group and Senior Product Owner at NI (National Instruments). He spent years as a product manager on an internal experimentation platform, working on the roadmap and alongside the analysts and engineers using it every day. He has watched thousands of tests, and he knows the patterns behind the ones that go nowhere.

This conversation breaks down how a program drifts into an anchor, and the specific moves that cut one team's test cycles from six weeks to two. You walk away knowing how to simplify your metrics, validate earlier, and hold the trust of stakeholders when real money is on the line.

 1. Trim your secondary metrics and move your primary metric closer to the change you're testing, so readings come back in days instead of weeks.
 2. Validate early and accept imperfect feature work, because a test that fails fast on instrumentation saves the weeks you would otherwise lose validating at the end.
 3. Protect financial goals with circuit breakers and guardrail metrics, then lean on proven proxy metrics so not every test has to move revenue on its own.

Has experimentation become an anchor? Testing for learning, not validation | Paul Davidson, Product Manager

What if your users already know exactly what they want, but your product is still too hard to use? Behavioral science has a word for that problem, and it's not "friction."

Kristen Berman is a behavioral scientist and CEO of Irrational Labs, the firm she built after running Google's behavioral economics group. She studies how people actually make decisions, and she spends a lot of time showing product teams why their assumptions about users are off.

 In this episode, Kristen walks through the gap between what users tell you in research and what actually drives their behavior. She and host Katie Green dig into how behavioral psychology can sharpen experimentation hypotheses, de-risk big product bets, and help teams build a genuine culture of learning.

Key takeaways

 1. When building an experiment hypothesis, start from a psychological theory about why behavior isn't happening, not just from what users say in qualitative interviews.
 2. Out-of-product A/B testing, such as running four versions of a pricing page with recruited participants before touching the live product, lets teams take bigger swings with far less business risk.
 3. The teams with the strongest experimentation cultures tend to share one trait: humility. They assume the first version of anything is probably not the best version.

Why past behavior is your best A/B test signal: Lessons from behavioral economics | Kristen Berman, Behavioral Scientist, Irrational Labs

Stopping someone who already has a finished experiment is a lot harder than saying no to an idea on a whiteboard. Marcela Gutierrez figured that out, and it changed everything about how Fossil Group runs its experimentation program.

Marcela Gutierrez leads digital analytics, experimentation, and consumer behavior at FOSSIL Group, where she has spent 13 years across the US and Switzerland. She is the kind of analyst who thinks just as hard about how data gets communicated as she does about what the data says.

In this episode, Marcela shares how FOSSIL shifted who builds experiments and why that single change unlocked speed across the entire team. She also breaks down how her analysts think about metrics, storytelling, and why human judgment matters more in experimentation programs as AI takes on more of the build.

Key takeaways

 1. Building a test before asking for approval changes the conversation. Coming to a sprint review with something nearly finished is harder to stop than rejecting a rough idea, and FOSSIL uses this dynamic to run more experiments with less friction.
 2. A primary KPI needs a secondary one. Marcela's team pairs each main metric with what she calls its "BFF" metric, so results tell a coherent story rather than just a flattering one.
 3. Analysts are not data processors. The most valuable work they do is understanding organizational context, cleaning data before it goes in, and translating insights for the right audience at the right level of detail.

Harder to stop a moving train: Inside FOSSIL's high velocity testing program | Marcela Gutierrez, Director of Digital Analytics and Experimentation, FOSSIL

In this episode,  Katie sits down with Rhys Mohun, founder of Formentor Labs and creator of the Room to Think newsletter, to discuss why the traditional "expert" mindset is becoming a liability in the age of AI.

Rhys shares his philosophy of being the "wrongest in the room": a strategy that prioritizes psychological safety and evidence over ego. As AI begins to automate the administrative "boring stuff," Rhys argues that the true winners will be the leaders who lean into human-centric skills like facilitation and empathy.

ABOUT OUR GUEST:

Rhys Mohun is the owner of Formentor Labs, a niche consulting practice teaching growth teams how to think bolder, innovate faster and use feedback from experiments to amplify their strategy. The team with the most ideas wins.

Rhys has 16+ years experience building growth teams that practice science and empathy to learn from their customers. 

At Intuit, he developed Canada's experimentation program welcoming 14M visitors and $65M revenues into a world-class practice that became the model for global teams.

He's led growth from seed to Series A for two Toronto SaaS and fintech startups.

Finally, Rhys loves to teach. He offers his signature Evidence-Led Strategy workshop for SaaS and DTC clients, mentors young startup founders at Wilfrid Laurier University StartUp Lab, and facilitates the Digital Growth Marketing program at Toronto Metropolitan University.

The wrongest person wins: How intellectual humility accelerates experimentation velocity | Rhys Mohun, Founder at Formentor Labs

Welcome to UNITE Voices, a new podcast by Kameleoon hosted by Katie Green. This isn’t your typical tech show filled with industry fluff; it’s a space for real stories from the people building today’s most ambitious experimentation programs. 

We’re sitting down with product managers, engineers, and designers to dive into their wins, their failures, and the cultural shifts required to scale a truly data-driven organization.

Come learn with us.

Episode 0 - Welcome to UNITE Voices

Unite Voices is a show where marketing and product leaders share the moments that changed testing at their company, and how they use AI (or not) in their experimentation work.