Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
AI Visitor Scoring for a Future without Cookies | Constantine Yurevich of SegmentStream
66 Plays2 months ago
SUBSCRIBE TO LEARN FROM PAID MARKETING EXPERTS 🔔
The Efficient Spend Podcast helps start-ups turn media spend into revenue. Learn how the world's top marketers manage their media mix to drive growth!
In this episode of The Efficient Spend Podcast, Constantine Yurevich, Founder and Chief Storyteller of SegmentStream shares the challenges in marketing measurement and innovative approaches to attribution and optimization. Constantine discusses insights into overcoming legacy systems, leveraging AI-powered visitor scoring, and navigating the evolving digital marketing landscape to maximize ROI and business growth.
About the Host: Paul is a paid marketing leader with 7+ years of experience optimizing marketing spend at venture-backed startups. He's driven over $100 million in revenue through paid media and is passionate about helping startups deploy marketing dollars to drive growth.
About the Guest: Constantine Yurevich is the Founder and Chief Storyteller at SegmentStream, an AI-powered platform helping fast-growing startups and DTC brands optimize marketing analytics for rapid growth. With over six years of experience building innovative solutions, he is passionate about transforming digital marketing strategies through advanced machine learning and data-driven insights.
VISIT OUR WEBSITE: https://www.efficientspend.com/
CONNECT WITH PAUL: https://www.linkedin.com/in/paulkovalski/
CONNECT WITH CONSTANTINE: https://www.linkedin.com/in/yurevichcv/
EPISODE LINKS:
https://segmentstream.com/about
https://segmentstream.com/resources/success-stories
https://martech.org/martech-topics/
https://www.marketingevolution.com/knowledge-center
https://www.adroll.com/blog
https://www.adroll.com/features/digital-advertising-platform\
Recommended
Transcript
Facebook's Role in Machine Learning
00:00:00
Speaker
50 people were wearing something luxury, where you can contextually identify that this is a relevant audience, but only two made the purchase. The question is, now Facebook, like Facebook is a machine learning platform, AI is the most powerful thing, Facebook asks you, send back some signals to me, like, was the audience that I've sent you was relevant or not?
00:00:23
Speaker
And most of the brands gonna send two signals. And in these signals, maybe people also maybe so some person was not even wearing other luxury brand. It was just some crazy person who just won in the lottery and decided to buy some a unknown brand, which is not relevant, just random. One of the person was like maybe some girl.
00:00:45
Speaker
around 20 years old. And now Facebook thinks that okay, probably we should build machine learning algorithm based on these two signals and bring you more customers like this.
Marketing Measurement Challenges
00:01:02
Speaker
I'm really excited to have a discussion around meeting mixed optimization with you and all the great work that you're doing at SegmentStream. In the marketing measurement landscape today, there are a lot of challenges that that we're trying to solve for, and a lot of software solutions are trying to solve for.
00:01:23
Speaker
You mentioned a few that SegmentStream is working on, but I'd love to to hear from from your perspective. When you look at the measurement landscape, what are the systemic challenges that you're really you're really seeing marketers played by and you're working on solutions for at SegmentStream? Yeah, so I would say digital marketing is a very dynamic space, but it's so dynamic that for some advertisers, it's quite challenging to adopt to to how quickly everything changes like yesterday you could have you can you can track everyone you can know exactly which websites are visited tomorrow you have different legal regulations you have different technical restrictions and it's very hard to adopt especially
00:02:11
Speaker
When like you learn to do something, you learn about different terms, like multi-touch attribution. like You have understanding what is multi-touch
Attribution and ROAS Issues
00:02:20
Speaker
attribution. And even though this term is very, very broad, and if you go to Wikipedia, you're going to see that it's it's a very broad term. And in a sense, MMM falls into MTA. like Many different approaches fall to MTA. But at some point, it became very, very narrow. And now people consider MTA being something like like a conversion that is stitched back using cookies and now you can assign credit to every single touch point and that's it. The same way as incrementality measurement, the same with MMM, like all this, the same with ROAS. So many people say ROAS while they yeah in reality mean some metric, which is reported in some platform, which has nothing to do with real ROAS. And I would say the biggest challenge is to adopt and to leave all the old stuff that is not working behind.
00:03:10
Speaker
and to open up to something new that can really help you solve real challenges. And and when you you think about that, you know I think there's there's a lot of challenges with with attribution and cross-device tracking, the the deprecation of of cookies, essentially less data for ah sort marketers to work with. right And as I think about it,
00:03:35
Speaker
you know As a marketer who's responsible for managing a seven-figure media mix, I have to leverage multiple ah data sources to make decisions that around where to deploy that that capital to essentially grow the business as efficiently as possible. right now base the The decisions that I make are based on the measurement lens in which I'm looking at the channels. Logically, I will make certain decisions based on that measurement that could strain me down a wrong course.
00:04:12
Speaker
I'm wondering what your perspective is on that when you talk to brands, when you talk to marketers, given the current attribution landscape,
Navigating Marketing Without Metrics
00:04:21
Speaker
right? Like what are the what are the negative implications of making decisions based on that data? Yeah, so this is exactly the topic I've started. So we have a lot of different, I would say legacy measurement approaches. And all of those measurement approaches have names.
00:04:40
Speaker
marketing mix modeling, like multi-touch attribution, et cetera. But there are so many approaches. And one of the approaches that we use, this approach doesn't have a name, right? There is no name invented yet. It's not a market standard. And probably what you're talking about with different lenses is when you apply multiple legacy solutions to be able to see like what reality is. And a good analogy will be, imagine you're currently in Dubai.
00:05:07
Speaker
And you're looking for the coordinates of San Francisco where you would love to like run your company install and and make a startup. And you don't have a good GPS and you start like this very and popular word right now, triangulation. And you take three GPS which are not working well. And one GPS says that San Francisco is actually in China. Another GPS says that San Francisco is is actually in New York.
00:05:32
Speaker
And the third one says it's actually in Africa and you do this triangulation different lenses and you end up in Cyprus and you continue building your startup in Cyprus. And but for some reason, like it doesn't scale and there is no proper environment for this startup. So my approach is different. Like I am, it might be controversial, but I am against all these triangulations with different approaches and different lenses. My idea is.
00:05:58
Speaker
to test and to fail. So first of all, go to China and figure out that China is not San Francisco. Figure out this fast, like in three days, probably you will be able to figure this out. Then go to Africa, figure out that Africa is not is not San Francisco as
Last Touch Models and Incrementality
00:06:13
Speaker
well. Then go to New York.
00:06:15
Speaker
It's better, but definitely not San Francisco. And finally, you will find San Francisco. After like this three, four, five tests and trials, you will be in San Francisco. And even though you might be losing losing like two or three months to figure out where San Francisco is, but you're not going to be stuck in Cyprus, right? So this is my approach to to this this methodology.
00:06:36
Speaker
that's yeah that That's funny. You know, I and have to push back a little bit because from from my perspective, marketers are running their budget based on existing paradigms.
00:06:53
Speaker
and that's gotten them so far. And so there are objectively thousands of businesses that are still running their media mix primarily based on last touch and seeing they've been able to scale with that paradigm, even though that landscape might be, even though that measurement has its its flaws, right? This was our case where prior to iOS 14.5, we were operating primarily based on Last Touch. we had you know a We had our CAC targets and our high-level revenue targets that we were hitting. We were still making channel-level decisions based on on Last Touch. And we were doing good enough. I was 14.5, kind of shifted that. We adopted incrementality testing. We started to do some different things. Now we're adopting M&M. And now we have this mix where we'll go into a a Monday meeting. We'll say, OK.
00:07:52
Speaker
This is how much we spent. This is how many customers we got. This is how much revenue we got. This is our CAC, so ROAS. And then within each of the channels, you know we have last such attribution we're making decisions on. Sometimes we have some incrementality tests from a given channel we're making decisions on. We're going to start to have more and more MMM results that we're making decisions on. And when I think about triangulation, I think about,
00:08:17
Speaker
like How am I directing the ship and how am I looking at all these different data points to direct in that, in the right way? So, and then, you know, to to your point, perhaps all that is wrong.
Beyond Local Maximums: Broadening Approaches
00:08:29
Speaker
Perhaps that, perhaps, perhaps that type of triangulation is getting me to, to not a ah definitive answer.
00:08:36
Speaker
If that is the case, how do marketers, you know, adopt something that's a completely new paradigm, right? Yeah. Yeah. So the idea is that like, I personally really love last touch and it has, it's like it, depending on the scale of the business, it still may work for you, but.
00:08:56
Speaker
problems start when you start to scale and at some point you reach like maximum that you you have you can in Google ads in paid search in all the like so-called lower funnel channels where you still can observe these signals and those signals are quite fine, like quite deterministic. At least you have some correlation between the click and the conversion. So problems start when you start scaling, when you start investing millions, for example, 500K or million per month. So this is when the problem starts. And last click can still, so with last click you still can reach local maximum.
00:09:33
Speaker
So you can play quite safe and you can still reach some local maximum. With triangulation, on the other hand, you might not even reach local maximum. You might reach somewhere in Cypress. With Last Touch, you can you can reach and settle in New York.
00:09:49
Speaker
and do quite well, even though a little bit further is San Francisco. But with triangulation, you can get some absolutely misleading results. It's like with misleading numbers of averages, like when people say that many years ago, people lived like average age of people was something lower than 30 years, right? And everyone says, okay, so it means that everyone was sick, everyone was dying, and most of the people were dying. So it feels, the perception is that everyone was dying before 30.
00:10:18
Speaker
But the reality is that like most of mortality was related to children before even reaching five years old. While anyone who reached five, like most of the people who reached five, sometimes lived till 80 because they like it was like,
00:10:33
Speaker
like a filtering mechanism of
AI-Powered Visitor Scoring by SegmentStream
00:10:35
Speaker
the evolution. Like, and, but the perception is that like people were old at 30 and then they died the same way as a train. So this is the problem of all the averages. And I would say, I would say our approach was to find an absolutely new way to zoom out from all existing approaches, like zoom out and look at the problem from absolutely different angle.
00:11:01
Speaker
How can we properly measure the impact of our ads and how can we properly optimize them to reach maximum revenue? And and I can give you maybe another metaphor. Imagine you decided to create a brand new luxury jewelry brand.
00:11:20
Speaker
And you opened your shop in the center of New York and you start running your ads. And no one is, so so some people still come into your shop, but you decided to start testing different channels. So you put, I don't know, 500K into Facebook ads.
00:11:36
Speaker
and you see that 100 people actually came into your store. And out of this 100 people, 50% were actually wearing something else, something else, luxury, from other luxury jewelry. On some of them you've you've seen you've seen Cartier, on some of them Bvlgari, some of them were wearing Hermes shoes, some of them were wearing Balenciaga, doesn't matter. And after all, like you you analyze, like you've just, you've just invested 500K. So these people come 100 people, but only two people made the purchase. So 100 people came to your store, 50 people were wearing something luxury, where you can contextually identify that this is a relevant audience, but only two made the purchase.
00:12:20
Speaker
The question is now Facebook like Facebook is a machine learning platform. AI is the most powerful thing. Facebook asks you send back some signals to me like was the audience that I've sent you was relevant or not?
00:12:35
Speaker
And most of the brands going to send two signals. And in these signals, maybe people also maybe so some person was not even wearing other luxury brand. It was just some crazy person who just won in the lottery and decided to buy some unknown brand, which is not relevant, just random. One of the person was like maybe some girl.
00:12:57
Speaker
around 20 years old. And now Facebook thinks that, okay, probably we should build machine learning algorithm based on these two signals and bring you more customers like this. And Facebook starts bringing more customers like this, but those are irrelevant. Because this ah like segment was so small and you were so focused on, they need to buy, they need to buy immediately. So that eventually you ruin your whole marketing and you ruin algorithms of Facebook.
00:13:25
Speaker
What segment stream proposes instead, stop focusing on conversions at all. Conversions don't matter. Your money in the bank matter, after all, in the certain period of time when you test different hypotheses. If you see that like 50 people wearing other luxury brands.
00:13:40
Speaker
who you like qualified as relevant audience, who came to your store, they asked about the product, they asked about your brand. They started, they asked different questions about different jularies. Someone brought their wives and they told that there's a wedding and they're looking for some brand, but they feel hesitant on buying a brand. So all of these people, you have 50 people who were qualified by you somehow based on either their behavior or different contextual features.
00:14:06
Speaker
If you're going to send 50 signals back to Facebook and scale your budget, Facebook will bring you much more people like this. And at some point, the model of Facebook will be trained. And at some point in the long term, more and more people of your relevant audience are going to know about your brand. And at some point, your revenue is going to grow.
00:14:23
Speaker
So this is our approach and it's quite simple. And we what was required is just to zoom out from these approaches, like let's focus on conversions that happen immediately whenever we launch something. Instead, focus on whether it was relevant audience or not relevant to audience and measure everything only based on this.
00:14:43
Speaker
Right. and And that is the very unique, one of the unique differentiators to to what you're doing and what you're referring to as what you call AI powered visitor scoring, where We're not looking at the conversion. We're looking at the visit or the session and the quality of that visit or session, right? And then you're doing modeling on top of that to understand
00:15:15
Speaker
Based on the quality of that visitor session, what is the probability of that customer to convert or to turn into revenue? Yeah. Or at least whether this was relevant traffic, well-targeted traffic, traffic that was really interested and interested in the product you offer, or it was some random traffic who just clicked, or it was some bot traffic or some irrelevant customer. Because especially when it comes to display,
00:15:43
Speaker
So like with paid search, like you still have some terms. Sometimes you might have like misleading terms, but in most cases, like you already have intent. But when it comes to display and paid social, like all of these activities, are in a sense, display, people can just like the catchy ad. So this like clickbait ad, it's really nice. it's ah Really beautiful picture of very beautiful girl advertising a specific product and they just wanted to click but once they clicked and once they saw Prices on the website in your luxury brand shop. They immediately ran away
00:16:20
Speaker
with a confusion and our goal is to identify this. So instead of making black and white based on whether this customer purchase straight away, we make all these shades of gray between black and white to identify like who was relevant, who was the most relevant. And this way our clients can target, can define the threshold themselves. Like what is the relevancy? And then, then they can and can build both their attribution and also media mix optimization and also smart bidding optimization based on these signals.
Systemic Challenges in Digital Marketing
00:16:50
Speaker
I want to get into the scoring system a little bit. I do wonder, because that data is based off a visit, how do you solve for the challenge of view-based traffic? This is another very interesting topic about view-based. If you run your ads, even if those are upper funnel, like if those are display and videos, if your ads do not generate any clicks, this is a signal.
00:17:18
Speaker
This is already a signal and what I've seen with most of our customers for for who have run many incrementality tests as well, because our platform also supports incrementality testing, geo holdout testing, that in many cases, upper funnel campaigns that don't generate clicks with engaged audience are usually not incremental.
00:17:39
Speaker
So one of the, one of the last clients who've run POC with us and finally they trusted us and they were able in just in two months to increase revenue from 1.5 million pounds to 2.3 million pounds. They were running a lot of ads in DV360.
00:17:55
Speaker
And then we measured the traffic that was coming. So last click was showing zero. The V360 interface itself was showing hundreds and hundreds post-view conversions. We've measured the quality of the traffic and our assessment was like There is no quality traffic at all. Like people spend like few seconds, they're not interested in the product. Most of them come with some strange IP addresses, countries, et cetera. So we measured like, okay, incrementality of the channel, like they they invest. like so So our assessment based on our approach was like also zero, like zero, sorry. There is no engaged traffic. Most of the traffic is just some bot traffic that immediately bounces. And then we launched incrementality test and incrementality test also showed zero.
00:18:35
Speaker
so What happens with this post view that everyone is so obsessed with? and like Some people even include this in optimization. For example, Facebook allows you to use like seven days post click, but Facebook also allows you to use like seven days post click, one day post view. so what is What is the problem with this and where where is the biggest like misleading thing in this? All these platforms have Pixel on your website.
00:19:00
Speaker
All of them track your existing audience. And the issue is, whenever someone comes to your website, from Google, from your organic search, from the channel that really drives incrementality, these pixels, like DV360 pixel, Facebook pixel, they immediately pick up this audience and say, oh, this is a tasty audience, this probably will convert.
00:19:21
Speaker
And they immediately start, like not tomorrow, they immediately start showing ads to this audience again and again and again. And then there are some conversions and they show, okay, this is a post view conversion and we are responsible for this conversion because we showed an ad and eventually user converted. You don't see any traffic from this ad,
Non-Click Channel Measurement Challenges
00:19:40
Speaker
you don't see any quality traffic of this ad. So users the users already made up their mind and in many cases they might might be already purchasing by that time.
00:19:49
Speaker
But this platform just shows you these post-view conversions, which in a sense you can perceive as retargeting. So it's super lower, lower funnel conversions. And in many cases, they are not incremental. Even though, like for example, we ourselves run a lot of retargeting.
00:20:06
Speaker
campaigns, but because we are B2B when we promote content, et cetera. But in many cases for e-commerce, for B2C brands, like if you're retargeting campaigns, Drive clicks, yes, you return customers back to website. Sometimes incremental, sometimes not. Usually incrementality is not that high that it's compared to what is reported in your last click, but at least there is some incrementality in this retargeting activities. What we've seen this with post view, in most cases, it's not incremental. I'm not saying that it's always the case, but in most cases and most of the upper funnel activity that are really incremental, that really drive value, we still are able to measure them based on customer behavior because like upper funnel means that you can reach wider audience, much cheaper, much broader.
00:20:52
Speaker
And like if in Google search, you have really high CTR, maybe in those campaigns with upper funnel, you have much lower CTR, but you still have a huge volume of clicks with lower CTR, with lower engagement, but we still capture this engagement. And we understand whether someone was interested or not. If if there there was no valuable behavior at all, in most of the cases, I doubt about the influence out.
00:21:16
Speaker
Understood that makes sense. And then for what about for channels that do not have a click? Yeah Yeah, yeah. Of course, there are some channels that like made in a way that there is, there is no click possible at all, like TV, for example, or like but podcast we are recording right now. So those channels are different. I'm not talking about those channels. And these channels should be measured differently. Of course, there are many different methodologies, but right now I'm talking about mainstream belief that like Facebook ads, display ads, YouTube ads are really driving
00:21:51
Speaker
incrementality while there are zero clicks. So in this concept, I don't believe at all. Understood. Yeah. And I think i think that that is that that is something that might be, there might be skeptics around. To me, it's a spectrum, right? Like to me, there are certain mediums where you can leverage the click-based data to learn more about the quality of the traffic while making an assumption that there will be some incremental nature of traffic that comes in beyond the click, but your your thesis is still very much based on visitor quality. Yeah, there is there is still some fraction, and for sure like there is some fraction who clicked, but I mean, if you see a channel
00:22:37
Speaker
where there are no visitors or where for example, you you invest there like 50% of your budget in YouTube and brand awareness. But based on visit scoring, you see that like there are 10, 5, 15 conversions. Of course, we always have this because it's it's still predictive. We still have some margin of error. It can be, let's say even plus 100% or plus minus 100%. It can be wide like error margin.
00:23:04
Speaker
But still when you do make decisions in your budget allocation, you still should play within this margin. You cannot play within margin like 100,000%. We cannot be 100,000% incorrect. But I see a lot of brands who allocate their budget this way and they still believe in brand awareness. And so for me, like if someone was not even interested to check out what but the product is, so probably it was irrelevant. they were watching their favorite podcast on YouTube and they just waited three seconds to skip your ad. like They just hated this, but they don't want to pay 10 pounds a month and not to see ads. Sure. Yeah. that That makes sense to me. let's Let's talk a little bit about your your scoring system. right so you and You assign a probability score of zero to one to every visit based on engagement and intent. What does that system look like?
00:23:59
Speaker
So in a sense, like first of all, we can call it in a sense, single visits attribution. Of course, when we can stitch everything, we do a lot of work in terms of stitching still. We still stitch everything that is possible based on cookies, based on user ID, especially for software service businesses. ah We stitch a lot based on IP address when users give consent to collect their IP address. We also have.
00:24:25
Speaker
methodologies, how to solve the biggest problem I would say in upper funnel is in-app browser.
Limitations in Data Stitching Across Platforms
00:24:32
Speaker
When most of the like Instagram, LinkedIn X, for some reason, they don't open Safari when you click on the ad, they don't open Chrome. Instead of the, instead of that, they open their in-app browser and people just browse, like view a few pages and then they switch to default browser. And some people say, oh, this is not.
00:24:51
Speaker
Not the case, because it's a small fraction of users. Actually, we we we've analyzed and it's like sometimes in Facebook, it's up to 25% of users who actually start their journey inside Facebook in an app browser and then switch to Safari.
00:25:07
Speaker
So it's it's already like 25% that can be restored and we can stitch it back. So we stitch whatever whatever is possible to stitch, but still like everyone is talking about identity graph. Like it was working a long time ago when third party cookies existed and life ramp made a fortune in this.
00:25:25
Speaker
But this is not the case. And even with everything that we are able to stitch, we stitch like small fraction, like maybe 1%, 5% of users with all these things when even but like only when people give consent, of course. ah But after that, like what to do with like 95 or 99% of sessions that we're not able to stitch. So in a sense.
00:25:45
Speaker
we use this data that we are able to stitch to pre-train some model that can recognize behavioral patterns that eventually lead to conversion because there are still some conversion that happened within the stitched customer journeys. And then we have algorithms how to extrapolate this knowledge and this patterns to all the traffic that was not possible to stitch. And in a sense, it's like a single visit analytics. Whenever whenever Click happens on the website. We analyze customer behavior. And at the end of the session, you already have a score. So it means like there are no problems with cookies here because within one session cookies. Always consistent. It's one session. This session is essentially build based on cookies. So yeah for for for one session, you always have a traffic source and it's impossible to lose it. So that's why we create conversion immediately after the visit end. And we immediately attribute it to the traffic source.
00:26:37
Speaker
This way, like you have like hundreds or millions of clicks and for every single click at the end of the visit, We give a score from zero to one and depending on the score, like, and after, for example, one week or one month, you have the scores assigned for every single click for every single channel. And then we take total number of conversions you have during one week or one month. And now we redistribute those proportional to the scores that was assigned to every guest. And this way you have like this view and assessment how, how your channels perform.
00:27:10
Speaker
Right. it's it's such It's such a unique system and it's so different. Yeah, yeah. Do you know any other software company or anybody else who's working on something like this? No, not at the moment. And there are so many pitfalls because the biggest pitfall is like how to train this machine learning model. Because there is no data to train because like in ideal world, you have all this beautiful customer journeys that are stitched together and you can train machine learning model based on consistent data. But we have fragmented data. We have incomplete data.
00:27:44
Speaker
And this is one of the biggest machine learning challenges to train a good model that solves business challenges based on incomplete data. So at the moment, I don't i don't know any any other solutions that use this approach. And from one from one side, this is an advantage. From another side, is it's a challenge because as I've mentioned, ah like marketers, they stick to the past.
00:28:07
Speaker
When they go to Google, they search the multi-touch attribution. They, they, they searched for, and at some point like we beat in Google ads, if you search for multi-touch attribution, you will see segment stream. If you search for MMM, you will, you will see segment stream. And on our website, we like, we really struggled like, what should we put? Because sometimes like there is a big company and like strategic management tell them, we finally need to implement multi-touch attribution to solve upper funnel problem.
00:28:33
Speaker
And they go to the website, oh, it's not multi-touch. So, but we are looking for multi-touch. So we started putting, okay, it's in a broader term, it's a multi-touch distribution. If you look in Wikipedia, what we are doing is a multi-touch distribution because multi-touch distribution is assigning a credit to every single marketing touch point. And this is exactly what we do. The only difference is that we assign this disc credit not based on conversions.
00:28:56
Speaker
Right. it's It's predictive based on the the model that you kind of explained earlier. I want to go back to the Facebook example because I thought that was really interesting.
Machine Learning Challenges in Marketing
00:29:07
Speaker
so be and and I think you know you're you've been you've been kind of clear this The use cases for this are you know much more based on graphic sources that have a click or visit component to them. um And this is where we should focus. So let's take Facebook, for example. I'm running a Facebook campaign.
00:29:28
Speaker
I'm targeting a broad art audience and I have a couple of different options on what I can bid towards, what I want Facebook to find. I might say, okay, I'm going to optimize towards a purchase. You mentioned that what happens is that I can serve a number of impressions.
00:29:47
Speaker
the Facebook might find the two purchases that I capture, infer some things about what type of audience that I'm looking on based on the unique attributes of those purchases, and then it'll go out and find me more of those spokes. But there's challenges there because what if those there's some randomness to to that, right? and Another challenge in in machine learning is also can be over-feeding.
00:30:09
Speaker
Like you you have these two really good signals that purchased and you overfitted your model. Right now it's not good in predicting other purchases because it's overfitting based on two signals. Now, that the other option and the the one that we're exploring a little bit more now is to optimize off of predicted LTV.
00:30:29
Speaker
which is to say that we assign a predicted lifetime value to a individual conversion and then we send Facebook that data and then we're trying to maximize our ah ROI based on that, which is still based on the the purchase event kind of happening.
00:30:51
Speaker
Yeah, yeah, exactly. exactly yeah Can I comment on this on LTV? Please. but So yeah, let's get back to our metaphor with you opening a luxury brand. So imagine like you have these two customers and now you have only two two signals that you can so send to Facebook and you're already overfitting the model. It means yeah you say, I only want girls of 20 years old, and also this also everyone who wins the lottery. I want only these people. But now you go even further into overfeeding. Now you say, okay, conversions are not enough for me. I want to optimize for LTV. And now you're like, I says, okay.
00:31:29
Speaker
This guy who who won in the lottery purchased just once, but this girl, she came again. And now you like essentially say you are discriminating this guy and sending Facebook more, like a stronger signal based on LTV that we predict that LTV for this girl will be higher. We more want more people like this.
00:31:48
Speaker
So first of all, you overfeed algorithms of Facebook based on sending only like purchases that happened. And then you overfeed it even more that based on this purchases that tracked by last click, you sent LTV, which is even more random.
00:32:03
Speaker
And when I say random in your past data, it's not random. It's a fact, it's a fact that these people but like have long, like high LTV and these people have low LTV. But the amount of data compared to the audience that Facebook possesses is very small. So it means that you can train your model really well based on past data.
00:32:25
Speaker
But it will perform really poorly based on the future data. So this is a main principle of overfeeding. And we see these problems all over the place when we, even like bigger companies, bigger startups, run incrementality tests. So why we decided to create our own incrementality testing. There were companies out there that specialize on incrementality testing, geo holdout testing, et cetera.
00:32:46
Speaker
But they come into the same trap. They do overfitting. They choose control market with like small number of conversion. Then they apply like coefficients to build synthetic group. And now the synthetic group is pretty aligned with the historical data, but it's overfitted.
00:33:01
Speaker
And then it's very inaccurate and very bizarre in the predicting future. So LTV, we have an approach for predicted LTV and probably it will not fit into this podcast. Maybe we will create make a a second episode focusing on businesses. like Because for me, like the most interesting, because because for e-commerce, everything is simple. like yeah We even have self-serve pricing plans for e-commerce that to work out of the box. It's super simple business. ah So no discrimination, but like in a sense, e-commerce is very simple.
00:33:31
Speaker
You have traffic, you have a conversion, conversion happens immediately, everyone is happy. But with lead generation businesses and with software service businesses like FinTech, it like it's an absolutely different story. It's like we don't even propose self-serve pricing plans for such businesses because those businesses are very complex. They have really huge offline component and not all leads are made equal and you need to make this distinguish between leads.
00:33:55
Speaker
But applying LTV based on last click conversions is very dangerous because it can overfeed the machine learning models of Facebook and
Synthetic Conversions and Optimization
00:34:02
Speaker
Google, etc. And it should be another another approach should be applied that we can talk about maybe next time. Yeah, for yeah for sure. And I think also the I think one of the the challenges with but predicted LTV is that You are making a number of assumptions on the gross sell rates, the you know retention of a given customer when you have a longer sales cycleor cycle or longer revenue revenue cycle of when you're actually when you're actualizing that that revenue. and so There's a lot of challenges with that. To speak to your
00:34:40
Speaker
platform specifically, instead of optimizing towards a purchase and giving that data to to Facebook, what you're basically saying is we can send them inversion data that's based on the visitor scoring. Can you kind of walk through what that looks like technically, specifically for Facebook? Because I think that's a really interesting use case.
00:35:05
Speaker
Yeah. So this approach is applicable for some clients. For some clients, it's not applicable. So if you opened your jewelry store and at some point you have not 100 people coming to your store, but like thousands and thousands. And now you have like at least thousand people who actually purchased those, those were not just wearing like jewelry, et cetera. They actually made a purchase. you You already have enough data, enough strong signal, and you can send this data to Facebook and you can be sure that Like when you have at least like 100 conversions per week in the campaign, probably you are not overfitting the algorithm. Algorithm of Facebook is really good in picking up signals. So this approach is applicable for businesses that have their Facebook campaigns and they see like five, 10, of course you should remove post view. You should remove it. Of course, now never train smart bidding on post view, but imagine you removed post view from, from Facebook and now you have like.
00:36:03
Speaker
15 conversions, and you understand that now you're overfitting. But you can you don't want to optimize for website clicks, because I've mentioned website clicks can be just clickbait. Like, oh, it's a nice ad. Let's make let's make like more beautiful ad, more catchy ad, and people are going to click. It doesn't necessarily lead to high-quality traffic.
00:36:20
Speaker
another another option Another option just to interrupt you. so and so you know You don't have a lot of money, you don't have a lot of conversion volume. You could create a custom conversion of like, I only want to go after people that have spent X amount of time on site, or you could create these things in Tag Manager to kind of like optimize towards a more engaged visitor in a sense. so That is an option. But it seems like what you're doing is a more sophisticated advance kind of yeah based on that. yeah I can go into this topic.
00:36:49
Speaker
Very deep, like especially when we see how people decide. like First of all, you need to make a decision. like how many How many minutes should your customer spend on the website so that you identify them as qualified? and Do you know how most of the people pick up this number? Yeah.
00:37:04
Speaker
They just go into Google Analytics and they see like average duration on the website. And here we're coming back to, and here we're coming back to the like flows of average numbers, like with these people who like most of the people die before reaching five years, while all this actually leave till 80. So in reality, you can pick up like this number.
00:37:23
Speaker
But in reality this is just average it might be that most of the people who spend ten minutes or fifteen minutes or one hundred minutes on on your website are those just who forgot to. Close their tab in google and just started browsing for another website while another portion of users can be just bought traffic and you and you again like. Having this triangulation and and you land in the wrong spot.
00:37:44
Speaker
both not not not filtering bad traffic and not filtering good traffic. So in a sense, like for human, it's very hard to pick up this number. And this approach is quite valid, like how many minutes someone spent, like how many pages and like for businesses who don't have budget and when not millions of dollars are at stake, this might be a viable approach for some businesses that invest like 5K a month and they're still ah growing. For segment stream, it's a viable approach because we we ourselves cannot build machine learning models based on our data because we don't have enough data. So unfortunately, we cannot use our own technology on our website properly. but the But the main idea that there are many nonlinear patterns. There might be some users who spend eight minutes on the website and browse only two pages.
00:38:30
Speaker
But they did specific actions, they they viewed contact page, they browsed on the fashion store and they browsed all images. Or there are some visitors who spent like, I don't know, I know from my girlfriend that some, some women spend like hundreds of hours on the fashion store website, just adding something to wishlist and looking for references. They want them to buy somewhere else. So there are many, many patterns that are misleading and might not lead to the conversion.
00:38:56
Speaker
With machine learning, we can feed algorithms with hundreds of signals, with hundreds of events, with hundreds of contextual features, and figure out like all these nonlinear patterns that eventually lead to conversion. So I would say this is this is the same approach, but a little bit more sophisticated. Sure. And and explain that technically right. So how how does that work in Facebook? What exactly?
00:39:19
Speaker
ah the the bidding like the the bidding and optimizing based on your segment stream visitor scoring. Imagine you you run your Facebook ads.
00:39:33
Speaker
And you don't don't observe enough final conversions. Maybe you have very complex product, expensive product. This product takes time to evaluate. Or for B2B, the most complex thing, when there are many decision makers. One comes and makes a decision while another one makes a purchase. So it's not even possible to connect this. So someone clicks on your ad, and they start browsing your website. They start reading about your product. They start reading description. like They do all the research.
00:40:01
Speaker
They maybe download some brochures, maybe, I don't know, they do all these things that usually people who are really interested in the product do. And then they leave your website without a purchase, but Facebook needs this signal. So at the end of the session, we create a synthetic conversion. It's not real conversion. It's synthetic conversion that we called like models purchase or whatever.
00:40:22
Speaker
And then we use like a use conversions API, like server to server and immediately, like whenever the visit is ended, we immediately send it back to Facebook so that Facebook could start optimizing immediately to bringing you more customers like this. are Is that data only for visitors that have a score of a certain percent, a certain number? Yeah, it's possible to set up in the platform. So we usually set up the threshold.
00:40:50
Speaker
to make sure that we are not overfitting the algorithm of Facebook, but also that we don't send signal like low quality signal. So it's always a balance between the volume, the number of signals we send and the quality of the signal. So we need to find this perfect balance where we are not overfitting the model, but the same time we're not sending low quality signals.
00:41:11
Speaker
Yeah, for sure. I mean, it's, that is, it's such a challenge in in paid marketing. And even, you know, when you're managing a multi-channel mix and you're making bets and you're making, you know, you're trying to pull budget in a bunch of different ways.
00:41:28
Speaker
This happens every quarter, Constantine, where we'll go through the planning process and we'll say, you know, we're on 12 channels right now. We want to test these 17 things and we have X amount of budget to do it. And you're like, well, okay, we actually need to only test three because we just don't have enough to spread across all these different areas. And so there's this constant balancing act of I want to learn as much as possible, but I also want to give each thing enough to get appropriate learning and then also give it enough data to optimize off of. This brings us back to like triangulation. You can like spread a little bit everywhere or you can throw like all in into China and you figure out, okay, China doesn't work. Then you all in in.
00:42:19
Speaker
New York, New York doesn't work. And finally you will find the channel because the problem, like I've seen this a lot among new clients, they have really like huge amount of channels. And and like, if we look at like we have this feature in the platform, not to see conversions, but also see like percentage of revenue this channel brings.
00:42:38
Speaker
And they have like maybe 50 different channels and everywhere like 1%, 1%, 1%, 1%, 1%. And then like we, of course, we, we, we already can measure this because we, we measure traffic quality. So this is already good enough, but also we have one step forward. So that one of our, some of our advertisers can do, we can also calibrate this attribution. So we can run geo holdout test.
00:43:02
Speaker
And if we see that, like you've mentioned that there might be post view effect. We might
Marketing Investment Decision-Making Challenges
00:43:06
Speaker
be wrong. Like margin of error clock can be like plus 100%, minus 100%. So we want to run additional assessment for them, but we cannot because.
00:43:16
Speaker
When the channel has 1% incrementality, there is no way to run a test because this incrementality will be indistinguishable. It will be just statistical noise. like And we've seen so many... like For example, across the US, like US is the best in terms of... like All other countries are much worse, but in the US, at least you have states, you have many big cities, you can still find some regions which are similar to each other.
00:43:40
Speaker
For example, compared to UK, it's crazy. like London, what are you going to compare to London? like okay Very hard to find like something similar. but For example, in the US, it's possible. But even within the US, like for most of the tests, even with really big advertisers who spend $1 million dollars a month, minimal detectable effect of incrementality is 5%. It means that like when your assumption is that your channel is less than 5% incremental, there is no way to measure it. It will be just statistical noise.
00:44:08
Speaker
So my my idea is like usually my advice to to companies that really want to understand the criminalities and then invest across Snapchat, Pinterest, TikTok, Facebook, DV360, et cetera, just pool all the budget.
00:44:22
Speaker
and make a firm decision like what they want to test. If it's TikTok, go all-in in TikTok. If it's Facebook, go all-in in Facebook. Run. Fail. But the problem is that most people don't don't want to fail because in some companies failure failure is considered to be a bad thing.
00:44:39
Speaker
While in my opinion, in marketing, failure is an amazing thing. Even if you run incrementality test and it doesn't show like statistical significance or incrementality, at least you know that this channel is not incremental. You can move on from China to New York, but most most of the people like perceive like this diversification across channel as safe when in reality, they keep wasting money for one, two, three years. Yes, it's not visible, but at some point,
00:45:06
Speaker
At some point, business stops growing. Yeah, I mean, I consider diversification to be a requirement and important, but there is, like anything else, a spectrum of diversification and there is this such thing as being too diversified. I was literally listening to a ah YouTube short that ah Warren Buffett was talking about how many times in his investing career, more than I think he said 70% of his portfolio was in a single stock.
00:45:35
Speaker
Now, that doesn't mean that 100% of his portfolio was in a single stock, but that means that he had conviction in one thing. That's why he's so successful, because he has a skill. like Diversification is fine. You can play like everyone else. You can be average. If the the goal of your company to be average, like S&P 500, that's fine. You can be average. You can just play safe, like $100 across all channels. But if you want to be like Warren Buffett,
00:46:04
Speaker
you need to get a skill, you need to understand what really works, what really doesn't work. So all this knowledge, like about post view, we've got this knowledge just because we failed many times.
00:46:15
Speaker
Because like the hardest thing in building product like ours, like the hardest thing, it's not about like writing code, it's about understanding what works, what doesn't work. Because like we started developing our product in 2013 and like imagine like we come up with this algorithm and there should be some crazy client who will trust us with their budget to test this algorithm. And but and we need 2, 3, 5, 10 clients. We need to fail many times before we're going to find what works now. Like, okay, now for most of the clients, we connect them and in most of the cases, like everything is perfect, like ah revenue is growing. But it took us more than 10 years to fail many times.
00:46:55
Speaker
Many clients return because of this, just because like we didn't know ourselves, many things. and But with with our our culture is not to act safe. We don't act safe. We want to find the truth. We want to find the truth. And at some point to find an approach that will work for most of the advertisers. And we're really we are really grateful to all our clients who work with us already for five, six years. like We're really grateful for them because they We had our ups and downs in terms of like testing, but we agreed together that we want to test it. I would i would love to ah to chat with you about investment philosophy, probably offline, because I think you have some interesting thoughts and i'd I'd love to chat about it. i to To close out the podcast now, though, I think I want to just ask one more question, which is a bold one. um What is your long-term vision for SegmentStream?
00:47:51
Speaker
Long-term vision is to become like this is not an accident when I told you about like luxury brands, etc So the idea is that our platform is not for everyone. So yeah, I would say when we talk about apparel you have you have Zara you have United colors of Benetton, but also you have I don't know, you have Hermes where you are not even allowed to buy something or you have Ferrari where you are not able to buy Ferrari because like you need to understand what Ferrari is and you can buy owned like used one, but you cannot buy a new one. So in a sense, we've tested, like we've've we've spent a lot of time to find our perfect ICP, perfect client profile. And we understood this profile. We want clients who are as passionate as we are.
00:48:36
Speaker
about finding the truth, growing their business, not being worried about being fired because they made a mistake. But when we finally some point like when we reach these clients, and finally like we have some really amazing clients who come to this journey with us, so actually we want, at some point, all these ambitious startups, companies who want to grow and who don't want to settle in being average to become our clients. And this is our long-term vision. I love it. Krasanti, thank you so much for being on the show. Thank you. Thank you very much.