Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Hero Engine
36 Plays3 months ago
The latest episode of The Commute pulls back the curtain on the Hero Engine as your host Noah Goldberg sits down with Robbie Sherman, revealing the data secrets that drive our success!
We explore:
- What the Hero Engine is and how it fuels our loyalty programs.
- How we use data to understand our customers and personalize their experiences.
- Success stories and the amazing teams behind the Hero Engine.
Recommended
Transcript
Introduction to 'The Commute' Podcast
00:00:00
Speaker
Hello, everybody. Welcome back to another episode of The Commute. I'm your host, Noah Goldberg. My pronouns are he and him. um This is the second season of the podcast, and I think we're off.
00:00:15
Speaker
I think I know we are off to a
Robbie Sherman and His Role at LTNA
00:00:18
Speaker
great start. Many people have come up to me in the hallways of this beautiful office of LDML and are really getting into this cast. I think, Jacob, I think we're starting to get that momentum that we're, you know, we were hoping for after season one.
00:00:33
Speaker
um Well, speaking of momentum, um you know i wanted to bring in guest. you know I'll say a little different. I'm going to say the majority of you don't know him, ah but I'll let him introduce himself and we'll go from there. So I have someone really special with me.
00:00:51
Speaker
Who are you? Oh, hello, LD land. I'm Robbie Sherman. um My pronouns are he and him.
00:01:02
Speaker
And I actually work on the LTNA, also known as Loblaw Technology and Analytics Team. I support the loyalty program from that perspective.
Insights into PC Optimum Analytics
00:01:15
Speaker
um And it's funny, you it's called The Commute. I didn't even know that. ah This all started actually this morning when I bumped into Noah and he's like, Hey, want to do a podcast with me today? And I'm like, a podcast?
00:01:28
Speaker
Well, good thing I have a face for radio, ah because otherwise that might be an issue. um But yes, my name is Robbie Sherman, um and I've been at Loblaw in August. It will be three years.
00:01:45
Speaker
um I joined John Wallace's team back it to three almost three years ago ah doing PC Optimum analytics, which was a blast.
00:01:59
Speaker
You know, I learned a lot ah about how we use our data um and how do we, you know, improve the customer experience, how do we improve improve the business um through that data.
00:02:14
Speaker
um So things like you would hear of Hero Engine, um that was under my purview, ah supported by amazing leaders such as ah Chelsea Wang. If you don't know Chelsea, you should reach out to her. She's super
Noah and Robbie's Connection over Loyalty Programs
00:02:30
Speaker
awesome.
00:02:30
Speaker
um And Yiqing Liu. um And so we did that. I did that for a while. um And then i had the opportunity to also um take on the technology stack um for loyalty.
00:02:46
Speaker
um And so what does that mean? um You know, a lot of our integrations with a vendor called Eagle Eye, um how the data flows through our systems, um making sure the customer experience is good for the frontline.
00:03:04
Speaker
ah So I have was doing both of those. And then i now all i now support the technology and loyalty operations teams ah on how campaigns come to life.
00:03:17
Speaker
All right. that's a That's a journey. That's quite a three-year journey. um So it's funny. Before I moved on to loyalty, I didn't know Robbie.
00:03:29
Speaker
um And then I remember ah when Lauren first took on loyalty, she kind of did like a...
Data Personalization in Loyalty Programs
00:03:39
Speaker
Um, similar, like it was just like, uh, an info session, if I could call it that or an onboarding.
00:03:45
Speaker
immersion. What do we call it? Immersion session? Something like that. Immersion session. Um, but I mean, it was good timing cause I had like, it was my first week on it too. Um, and I remember like meeting you for the first time i was like, oh, cool. And then you know, actually, you know, we started playing what we like called Jewish geography. Um, and then, you know, we just knew the same people. turns out that you knew my nephews, which was cool. And then, you know, outside of that, ah we both have, we both share, robby and I share a very similar passion of,
00:04:17
Speaker
you know, getting shit done. um And I think we both with Robbie, you know, Robbie mentioned working on data and understanding how our data works um and is used.
00:04:29
Speaker
You could tell that like, you know, he just wants to move our amazing loyalty program even further. So Robbie, you know, Let's, you know, you've had such a really, you've had a really cool journey in, know, three years here.
00:04:43
Speaker
um You know, I will plug, you know, for those at LDML, um if you haven't met Robbie, he is on Slack. hey Okay. I'm learning. I'm learning. I'm learning. Yeah. ah But you should absolutely grab coffee with him. really knowledgeable about the enterprise, knows a lot about loyalty. um I still learn from him. every Honestly,
Personalized Customer Experiences and Future Innovations
00:05:07
Speaker
every week I learn something new about how it all operates. But with the data, like what and what was the as you were getting into loyalty, um and we'll get into Hero Engine and um in a few minutes, but like what is what was the most fascinating thing you found like about how we use our data to power our loyalty program?
00:05:30
Speaker
Yeah. Um, that's an interesting question. And I would say in general, um, ah The most powerful thing, it depends how you look at it.
00:05:41
Speaker
If you look at it from a customer lens or a business lens, it depends, right? um You know, one of the most fascinating things to me, which is actually not that complicated ah in the grand scheme, is how we can take our data to personalize it to a customer without spending promotional dollars.
00:06:05
Speaker
Right? Like how do you drive, you know, first we we talk a lot about offers and I think that's super important because that incents behavior, but it's also about how do we actually use that to um just make it more personalized for the customer. So let me use a very simple example.
00:06:24
Speaker
I think one of the most... You know, a couple of years ago, to me, one of the most personalized experiences we had was in the PC Optimum app, which is called Flyer Engine, which it just populates, you know, your most purchased products. And there's some waiting on price and everything like that. That just makes it easy for you to know what's on the flyer, which we're going to invest in no matter what.
00:06:49
Speaker
And how do we show that to the customer? Right. That is years ago that this came ah on board. Right. And, you know, you can you we've actually taken it so much further now ah with that. Like, I know we're working on how do we digitize the flyer and personalize the entire flyer versus eight products for those that just ah like, you know, let me just make sure i everyone understands what the flyer engine is.
00:07:13
Speaker
I can't whip, right? You can't show it. No one sees it. They only hear our safer radio voices, not faces. um The flyer engine, if you go in the PC Optimum app and you go into your offers, ah you will see a carousel that has says flyer deals for you.
00:07:34
Speaker
Your grocery flyer deals. There you go. yeah um And that is the flyer engine. It essentially takes... Our current flyer tied to your home store, so the store you shop at the most, yeah and it will so it will put the top eight products that are most relevant to you. So we aren't creating new offers. We are just making sure that our members are aware of the most relevant products that are on sale that week for them, right? That's right.
00:08:03
Speaker
Love it. Not complicated, like in in theory. Yeah. I nailed it, Jacob. You nailed You did it. You know your app. You know your Apple. You know your Apple app. I'm sorry. Yes. Yeah, yeah, yeah. You
Hero Engine and Targeted Offers
00:08:14
Speaker
know, and like that's rudimentary, I would say.
00:08:17
Speaker
um And, you know, where where I get really excited about it is like, how do you take it to the next level? Right? Like, I truly believe that we have all this data. Right.
00:08:28
Speaker
Every flyer should be personalized, right? Why do I need to have, you know, as an example, I'm past the stage with having a little baby. So like, I'm not really interested in diaper off, like diapers are not really a thing for me.
00:08:44
Speaker
Why do I need to scroll through the flyer and even see diapers, right? Like give me what I might buy or what I do buy. So like, I love that, you know, today, um I saw in the PCX feedback channel,
00:08:58
Speaker
Uh, that the personalized experience, the one-to-one personalized experience have come out. Like to me, that is like a perfect use case for how you use data.
00:09:09
Speaker
Agreed. Right. Like I was super pumped when Amir posted being like, it's live, like. Can I have some feedback? And like, I right away downloaded the new app and I went to my, my zone, my zone. i don't know what we're going to call it, but, uh, and it nailed it.
00:09:24
Speaker
Like solve dinner this week. Yeah. Like that is, are you listening to my, my voice at home being like, what's for dinner tonight? What are we doing for dinner tonight? And what do I have? Chicken fajitas.
00:09:36
Speaker
Like perfect. I couldn't be any better. bit So is there Taco Bell in there? there's There's not Taco Bell. Like one of the pieces of feedback I would have is that it has like whole wheat tortillas. Like I'm not eating whole wheat tortillas, but I think, I think my significant other is the one that's ordering the whole wheat tortillas. Now you're ordering. Yeah.
00:09:57
Speaker
um But so like, to me, that is like using data to actually improve the customer experience without investing dollars. have it But I also think, you know, you mentioned about things like hero engine or targeted offers.
00:10:09
Speaker
Like that is how you actually drive value for both the business and the customer. Like we know how much you're spending and where you're spending it every period, every day, every week.
00:10:22
Speaker
How do we actually create experiences that are personalized from an offer perspective? And that's where things like Hero Engine comes in You know, I know the market team is doing some amazing things with um matching, price matching, like ad matching, right? Which is like being able to, and I know this is an actually partnership with the LD, is like scraping flyers on a Wednesday morning and then actually being able to be able to create offers online.
00:10:49
Speaker
for those products and have it live for the in the customer's app for the next day. Yeah. Like really like helping the customer instead of them having to do it. Like we know you, we have a pretty good sense of you. Here's what we're going to do for you. And what that does is from a business is a good value for that customer.
00:11:06
Speaker
But what you don't do is have to re-rate the base. And what that re-rate means is, yeah, we could probably, you know, i'm going to pick an example. Maybe it's the tortilla shells that I was talking about are, is what we, is on the flyer of a competitor.
00:11:21
Speaker
You know, one option is you change the price for everyone. And so even the people that are going to buy it anyway, are still going to get that discount, but rather you remove that pro promo dollars and just focus on the people that care about price match and are going to buy that product. So it's a, it's a really good, like minimizing the re-rate in our base.
00:11:43
Speaker
I think, well, one, um big shout out to, Liam Charu and that whole team. to get praise from Robbie that they're doing it right. Oh, man. I already saw Liam this morning. I saw Liam this morning. I saw Liam this morning. I'm like, look at him. You solved my dinner problem for tonight. Yeah, I love it. um So, Robbie, you mentioned Hero Engine.
00:12:06
Speaker
And, you know, for those, so a couple of things. but One, maybe you could tell us what Hero Engine is, um why it's important to the loyalty program, and, you know,
00:12:20
Speaker
let I don't we again to have to get into the details, but like, know, high level how it works. Yeah, sure. So what it is, is it's a bunch of algorithms that determines, you know, what offer per targeted offer we should give a specific customer in a specific week.
00:12:41
Speaker
And there are few categories of offers that would fit in there. So, you know, one would be your total spend. So, you know, spend $200, get 20,000 points.
00:12:52
Speaker
um there are category offers. So, you know, we know you. are in the baby world. So, you know, spend $45 this week on baby, get 5,000 points, or it can go all the way down to, you know, targeted MOPs, which is, you know, could be ad, like ad match is an example, but it could be like, we know you're going buy this product.
00:13:15
Speaker
Here are the, you know, here's a discount on that product. So that's like the the general of what it is. And is it right to say that when we talk about targeted The audience that we're mostly focused on are are fully engaged customers. that Great point. So they like the who is generally where we where we see our best response rate, um which is customers who actually redeem the offer, is is on our most loyal base. So yes, the ones who are actively engaged in the app um are the ones that we know.
00:13:51
Speaker
I haven't seen the data recently, but like we were in the upwards of close to 20% response rate on those customers. And so how it works is, you know, and we're getting even better with, you know, I'm going to give a shout out. i don't know if Min Pan, um, on the hero engine team listens to this, but he's actually building a budget optimizer because the the complicated part with targeting, um, for offers.
00:14:17
Speaker
is managing to a budget and an investment level, right? i I like to use the word, I don't like to use the word cost because it is an investment because the return on it is very positive. Yeah.
00:14:29
Speaker
Right. So it's, you know, how do we get the best return on our investment without going over? Right. And so. You know, the challenge is, is like forecasting on a week to week on who's going to redeem and that not whatnot.
00:14:43
Speaker
What the team is in working on is called the Hero Budget Optimizer, which is to take that investment of what we want to invest that week and ensure we get the most out of it.
00:14:54
Speaker
Um, so that is like, to me, like the magic, right. is being able to put the input of how much you want to invest and what you expect, what you want your return to be and choose the customers or our members that are going to get the us that
Measuring Loyalty Program Success
00:15:13
Speaker
returned.
00:15:13
Speaker
Right. So for for everyone um who uses the app, what when they look at their offer feed, yeah what is a hero offer?
00:15:24
Speaker
How will they know something's going to offer? um So you can look in your hottest offers. Like I'm looking at mine right now, Noah, and I have Real Canadian Superstore.
00:15:36
Speaker
20,000 points when you spend a total of $200 in a singular cumulative purchase. Oh, wow. That is, would be a hero. and That's a gimme for you, I feel like. Huh? I feel like that's a gimme. oh No, no. You're a family of four?
00:15:50
Speaker
Yeah. Okay, are we... and so yeah yeah okay then you do a lot of uber eats i guess right oh yeah exactly there you go i am an uber one member um and my you know my kids also have like hot lunches every day i'm like about not about making lunches uh this is not my forte um you know i would argue another one would be Like right now, i if you go to like your home store offers, so right below hottest offers, you'll see whatever your home stores shop the most.
00:16:23
Speaker
Yes, offers where you shop the most. I see here a farmer's market muffins and it's pay $5.50 and get $6.50. Right.
00:16:34
Speaker
So there was algorithms that are determining why five 50 and why chocolate chip muffins. Um, and I know the SDM team, like lot of their offers are, are hero, call it hero-esque.
00:16:48
Speaker
Um, so those are some examples that you would get. Very cool. Yeah. Um, but what's also really cool. Can we talk a little bit like while we're in the app? Cause like, I do want to give us another shout out to, these to the Apex team.
00:17:00
Speaker
Love it. I was going there next. Oh, you're going there. Let's go there. So wait, let's, okay. Is there anything else on Hero we want to talk about? Do we want to talk about how we measure?
00:17:12
Speaker
I was going to say, so why don't we talk like very quickly on like um how it's built. Um, and then again, how we measure is probably the like, let's finish off with that. So it's a bunch of algorithms that run yeah on a weekly basis and we continue to tweak the algorithms yeah to make sure that we're staying with within an investment level and getting the return. yeah And then, you know, I always like, I look at things in like three lights. Like you have like the propensity of a customer taking something, you have the value, we have a value of
00:17:45
Speaker
what the customer is going to get. And then there's the S and the S is strategic. So there, you kind of always need that strategic lever, which is essentially break the P the, the model. And like, we need to focus on example of that.
00:18:02
Speaker
You know, we, like an example would be, we need to like amp up support for a specific banner or for a specific group of customers to try to drive it. So it might not our category even, it could be a category, right? It's like, how do you tweak it? So it might, you might reduce your return.
00:18:21
Speaker
yeah But there's other reasons, business reasons, economic reasons that we're going to do something. a way to drive tonnage in a category. Exactly. Like it might not be the most intelligent use of our money, but there are other business needs to be able to tweak it.
00:18:34
Speaker
Interesting. Right? So you kind of always, you want to be careful on the S. Yeah. Like the S is like, is dangerous if you just like always have that. it. But there is that part of it. Let's, so measurement. So this is where, I think that's one of the, despite like, this is the spiciness out of loyalty, right? Because this is where we talk about,
00:18:51
Speaker
incrementality, yeah right? And Hero Engine's incrementality is measured on a test to control, the' correct? And I think that's always been like, yes, you know, the argument about test control is that if you, well, if you just remove Hero Engine, the dollars would go away, so it actually is incremental.
00:19:09
Speaker
um You know, but then there's also, like, the test, like, you don't necessarily see that incremental in a P&L every week, right? And that's the, um,
00:19:21
Speaker
That is the spiciness of it, right? And like, it's really interesting because, you know, coming from, so my, prior to coming to join a Loblaw, I spent a lot of time in telco um and managing the customer bases um at both TELUS and Rogers.
00:19:36
Speaker
Um, and so what does that mean is like, you know, making sure customers spend more with us, making sure customers don't leave us. Um, and it's all done targeted. Yeah.
00:19:48
Speaker
Right. And all offers are targeted in like, in that industry, it's very well known of how to measure incrementality at targeting control. I think as I've came to grocery, worst we were still, I think we've come a long way, even three years. We were still in in the mass type measurement, right? Because we our whole industry is based on that flyer on a weekly basis, which is looking at, you know, what was the pre-sales or the baseline and the sales we got after we went on promo.
00:20:18
Speaker
yeah And the advantage of doing targeted Which is you select an audience, you carve out a chunk of that audience that you wanted to give that offer to, and you don't give it to them and then measure the performance of it.
00:20:35
Speaker
Right. Right. And it's like, it's very hard to get your head wrapped around that concept, yeah but that has how every targeted audience. offer or anything targeted, whether it's an offer or not, is like, that's the advantage. Like, that is the best way you're going to actually measure incrementality. yeah You really got to take your, got to get your head wrapped around
Apex Engine and Personalized Offers
00:20:56
Speaker
it. Super full disclosure. When I first joined, I was like, wait a second. and Like that's, but once you actually understand why and how we do it, um you do get around it. Okay. Let's, um, apex.
00:21:10
Speaker
Oh yeah. Um, yeah. very So Apex is essentially our baseline offers, right? Like our base offer for grocery. yeah um Yeah. Maybe we should explain what that means. Yeah. So I guess Apex is the set of offers that you get that are tied to your home store again, ah but they're the offers you get on your everyday products. That's right.
00:21:36
Speaker
So shoppers today ah has a base point. Right. So it's two dollars two points for every dollar you spend. Grocery does not. Instead, grocery says, you know what, you buy cucumbers every week. We're going to give you 100 points for cucumbers. Correct.
00:21:51
Speaker
um We've built an engine that assigns those products to each customer. Right. um Give us the quick, you know, the TLDR on that. Yeah, TLDR is, um, okay.
00:22:06
Speaker
First, you can get your Apex offers at Shoppers. So it is eligible. Um, so just so you know, and where you would find it in the PC Optimum app is under the section called offers pick for you.
00:22:20
Speaker
And the, if you remember, I mentioned the, the, the gentleman by the name of Min Pan, who's building Hero Budget Optimizer. He built Apex and the the TLDR is we used to have a third party vendor that did it. It was a, you know, a dark box.
00:22:37
Speaker
We didn't understand how they did it or what they did. um And, you know, it was the right decision when we started the program, but we felt that, you know, we could probably do it at least as good Probably better for a lot less cost.
00:22:53
Speaker
And so last year, the team went on an adventure and was like, we're going to prove to you, Galen, that we can do this ourselves. And what they did is they built um this personalized engine. And the key thing here is Back to the investment is we don't want to blow our budget every week for giving customers stuff they're going to already buy.
00:23:16
Speaker
We want just enough so they're getting points on their receipts, right? And they're engaging the program, but we just don't want to give away the
Technology Stability and System Reliability at LTNA
00:23:23
Speaker
farm. So the key to this program is yes, being able to identify the products, giving enough quantity so people feel that, you know, we're investing in them, but also not blow the bank. And that's what they built.
00:23:36
Speaker
So the the key thing here is around, again, the investment level and being able to, you know, say we want to spend $300,000 this week, go to all the customers and we know we're going to spend $300,000 week.
00:23:51
Speaker
yeah Very cool. I think those are the two. like those So we covered all the engines, basically. There's another one called Playbook that my team uses, but that's... Come on, talk about it. What is there? is there? That's for a different time. um Let's so we've talked a lot about loyalty, yeah um but I think this is really important. You know, a lot of people look at loyalty and they just see this program where you're in points. But we cover a lot of like how the points are earned and why we do certain things. i really hope everyone appreciated what like yeah all those things we just talked about.
00:24:28
Speaker
Quick switch. So you mentioned so you're part of LTNA. I am. ah What is one thing about LTNA that you would want everyone at LDML to know?
00:24:42
Speaker
Oh, that's a good question. now I would say the the one thing if I had, I wanted LDML to know is that the most important thing for LTNA stability of our infrastructure.
00:24:59
Speaker
Right? I think it's the right thing. And so we're are obsessed with system stability overall, right? And we take it extremely seriously across any changes we make.
00:25:13
Speaker
Um, anytime there is major incidents, cause it is technology, are we recovering quickly and are we understanding, um, what. What went wrong and how do we prevent it from happening again?
00:25:26
Speaker
It is something that like it's instilled within the entire LTNA organization. Um, you know, every, every couple days a week at 7 30 AM, m we're on a call to go through all the changes to make sure everything's.
00:25:42
Speaker
All good. And it trickles down to everyone. Right. Like, so for me personally, how that translates to me is i am always constantly looking at our performance with Eagle Eye.
00:25:53
Speaker
Right. And we had one issue this year and I'm obsessed. Oh, Eagle Eye is our loyalty vendor who is essentially the central location for all our points. So everything loyalty that's points related touches eagle eye. Um, and so like for me, what that means is like, I literally look at my dashboard to make sure that everything is working as, as it should be.
00:26:18
Speaker
Love it. Yeah. Um, okay. That was awesome. Uh, okay. Quick hitters. What are you doing this weekend? What am I doing this weekend? and I am driving my children around left, right, and center. Dance, gymnastics.
00:26:32
Speaker
ah What else do we have? Tudor. So lots of stuff in the calendar yeah that that I look at. Yeah. All right. I'm going random here. Favorite control brand product.
00:26:44
Speaker
Is the chicken burgers. Which ones? The breaded ones. I know, but PC or No Name? Oh, PC. Okay, got it.
00:26:54
Speaker
um
00:26:57
Speaker
Last but not least, um what's one thing about loyalty that we didn't talk about that someone should book a coffee chat with you to learn about?
00:27:09
Speaker
Oh, um I would, I would, i love talking about what we can do to improve utility of the loyalty program.
00:27:21
Speaker
Right. ah I like got a book of coffee chow with you. Yeah. Like I think, I think as an organization, everyone likes to go to offers because I think it's like, you know, yeah points and earning. But like, I think that's to me where, where, where we lack is like making it useful. Why a customer should go into the, the app if you like for not just points, you know, I do want to give it like a shout out to LD team because like an LDML, like,
00:27:46
Speaker
the The stuff you guys are coming up with and the speed at which you move and the collaboration is is unbelievable, right?
Podcast Conclusion and Coffee Chat Invitation
00:27:55
Speaker
Like I use most of the apps frequently and it's like such a great experience.
00:28:02
Speaker
ah You guys should all be extremely proud of of the work you guys are doing and and and like the culture you have on on the LD team and LDML. Okay. Love that. Well said. and we're going to end on that note.
00:28:13
Speaker
Robbie, no one's man. Thank you. Have a great weekend. You as well. Do this again soon. Hey, everybody, book that coffee chat with Robbie. Extra large, one milk, one and a sweeteners. Please go. yeah From Tim Hortons only though, right? Tim Hortons. Got it.
00:28:28
Speaker
All right, folks. Hope you're enjoying that commute. Hope you enjoyed this cast. if If you're listening today, enjoy the weekend. If not, enjoy the rest of your day. Love it. Thanks, everyone.