Transcript
Speaker: The initial version of Apollo was answering those questions well, but in a way that was highly bespoke and very difficult to scale. Apollo is not a cost center, but Apollo is a a profit center for our client partners.
Speaker: Today's episode is brought to you by Omni. Most companies I speak to want AI analytics, but are failing to put projects into production. That's where Omni is different. It's the semantic brain that grounds AI in the heart of your business logic, giving you governed answers, whilst also the depth to identify root causes. It's intelligence everywhere, from your spreadsheets to their chat feature, even within your product. Omni moves you beyond the dashboard. Don't just take my word for it. Trust teams like Booklexity and Synthesia that are already using Omni to deliver intelligence that people trust.
Speaker: Check out them in the show notes or visit omni.co. That's O-M-N-I.co. Now, back to the show. Hello everyone, welcome to another episode of the Stacked Data Podcast. I'm Harry Gollop, your host, and today I'm joined by Adam, the head of data engineering at Media Lab, a independent media agency that has won multiple awards over the years for some of the tech and products that they have built. And Adam is going to be talking today about
Speaker: Media Lab's proprietary platform called Apollo and how they have been able to build a platform that is able to serve 50 plus clients without having to have 50 different versions under underneath, giving them the agility and the ability to scale their client basis whilst giving them that their clients the bespoke feel of a proprietary platform.
Speaker: Adam, it's great to have you on the show. are you doing today? Yeah, great, thank you. It's exciting to be here. Thanks for all of the chats leading up to this, but yeah, looking forward to getting into it.
Speaker: Brilliant. Well, look, um I've obviously given a bit of an intro, Adam, but for those that might not know your yourself or have heard of Media Lab, um yeah give us an overview of of maybe your career and a Media Lab as ah as a business.
Speaker: Sure, I won't spend too long here. I know that folks want to get to good stuff. So yeah, I'm Adam. I'm currently the head of data and engineering at Media Lab. I look after um all things to do with Apollo, our marketing intelligence platform, which I'll come onto in a little bit. um I started in media doing search advertising.
Speaker: and I quickly took a side step into data and tech and I built a career from there. So I've held various roles in large network businesses, i several roles within GroupMWPP, always kind of hands-on technology leadership roles,
Speaker: and I joined Media Lab in 2021 and spent the last five years building out what we've got here and yeah really really proud of the progress we've made and kind of what the capability we have today it's um a significant capability so that's a bit about me and yeah Media Lab you've already introduced it but we are a leading independent media agency we are we are currently campaign and Media Week's Independent Agency of the Year.
Speaker: We've won the last three Agency of the Year Awards on the trot, which is great. um We've had consistent growth, but particularly over the last few years, it's been very significant. And I think Media Lab is founder-led business, um very, very values-driven organisation.
Speaker: and yeah we're a full service media agency focusing on UK clients um across a range of sectors so that's a bit about who we are and Apollo has been a key differentiator um we have to be able to feel good about where we're spending our clients money and we have to know that the money they're investing is driving the outcomes they want and Apollo is the engine that gives us that confidence so yeah big stakes but we're really proud of and really exciting journey Excellent. Yeah, so like you you teased a little bit there, and like I suppose the whole the whole hour of podcast is going to be around how you've built this marketing intelligence platform called Apollo. um
Speaker: Could you give us a bit more details on what role it actually plays inside Media Lab? Sure. um Two halves really. First half, it is a proper grown up, scaled out data platform.
Speaker: So it acts as a single source of truth. We're collecting all of the data that is relevant to evaluating our clients' marketing activity and often beyond that. In one place, we have a foundation for decision making.
Speaker: And then secondly, we have developed and are constantly iterating on a suite of measurement services and other forms of marketing intelligence that allows us to really activate that data and understand where where is media being valuable, what's the right balance of paid, owned, earned and different media channels. so First and foremost, it's a source of truth for decision making, allows us to be accountable for how we're spending clients' money.
Speaker: And secondly, it's really about unlocking that intelligence layer through a suite of marketing intelligence services, which major around kind of measurement and evaluating effectiveness.
Speaker: Awesome. So I think marketing spend is a huge part of business growths and particularly in product and tech based startups, um acquisition of customers um and i suppose and and enterprises. Acquisition of customers is is key and the landscape's been changing a lot in the in the growth space.
Speaker: When you first joined Media Lab, like what was the state of Apollo? um And yeah, it'd be great to sort of start hearing about but how the the the platform has evolved.
Speaker: Sure. Yeah, when I joined Apollo was pretty well established amongst the existing client base. It's kind of um emerged from need and really the need was the two hearts I've just described.
Speaker: Single source of truth so you can actually not spend time wrangling data but get on with decision making. Before Apollo existed, that was the typical combination of spreadsheets, ad hoc referring to several platforms to make decisions.
Speaker: All of those things take away from you know taking the right actions quickly. And the second is a deep understanding of what's really going on because it's not a straightforward job in marketing. Like many domains, understanding what the kind of causal impact is of some investment.
Speaker: You can't just take what the platforms tell you what's happening for granted, you need to dig deeper. um So those two products drove the initial kind of version of Apollo and the initial version of Apollo was answering those questions well, but in a way that was highly bespoke and very difficult to scale.
Speaker: So when I joined in we had a really happy customer base with Apollo, a really engaged internal team, but i but we had big growth ambitions as a business and I sort of suggested that we're going to change that approach if we want to maintain that service but continue to scale. So it was nice to have that engagement and adoption already there, but clearly it needs to iterate on the product we had.
Speaker: Interesting and I think the whilst we're going to be talking about Media Lab and Apollo's as a platform i think there's gonna be a lot of lessons here which can be applied to just building out a data platform or a marketing intelligence platform um your yourselves but equally I think you know Apollo's been built for for a reason right so um Adam, when you you sit down and you joined, you've seen some of these limitations, you you probably started thinking about what does the what does this new architecture need to look like? What are the principles that you wanted to really achieve from this platform? um
Speaker: Help share how you thought about that from like a yeah i suppose combining that so ah the the the needs of the commercial needs and the the technical challenges that you wanted to unlock.
Speaker: Sure. um I suppose we had three kind of core design principles. One was transparency. This is this is a value for Meadlab anyway. You what we see is what we tell our customers and we want to be transparent um in every way.
Speaker: So we needed Apollo to be able to surface data from what's going on and acknowledging that there are multiple sources of truth when it comes to evaluating the effectiveness of marketing. We wanted all of that to be accessible.
Speaker: We wanted the kind of data-driven optimisation discussions to have all of that available for consideration. A second was technical excellence. I've been in and around this kind of problem for 10-15 years and the hardest part of this problem area is scale.
Speaker: And I don't just mean technological scale but kind of process and operational scale. This isn't a low business complexity problem um and with complexity comes a lot of risk. Are you able to maintain and scale this?
Speaker: So it was clear we needed some proper technology chops in this area. We kind of couldn't get by on a series of string and sticky tape solutions so that was a priority from the start um and the third is i kind of already mentioned it there but real operational scalability we knew we needed to be quite grown up in terms of how we actually going operate this i'm talking support enablement engagement internally and externally and on the market So those three kind of things, you know, transparency, technical excellence and operational scale were considerations from the get go. And I had to balance. I mean, it's kind of a, I suppose, a a typical typical academic exercise in agile project management.
Speaker: Got to move forward and MVP mentality all the time. whilst holding the the big vision in your mind, i'm mean, I want to sell that in because you've got to get buy-in for where you're headed, but you've got to get there in steps that prove value along the way. So they were the things that were immediately on my mind when I joined and I started to form a plan after did a bit of listening and absorbing. Can you um ah can can you expand on what you mean by like an MVP mentality?
Speaker: I think be good to actually understand like What does that mean in practice? Yeah, and I think what it means in practice with a data platform is quite a nuanced question, actually. People can kind of understand what it means in other areas of software. So MVP, this idea of you've got to ship a minimum viable product so you prove value quickly. You can't go into a dark room and spend months and months and months and a large amount of investment before you kind of come up for air and put something out there and get feedback. That is a quick way to spend a lot of money and a lot of time building something that doesn't meet grief or the requirements moved on. So I think the MVP mentality is quite
Speaker: agreeable as an idea but as you say in practice it's tricky and i think if you're building some sort of saz application with just a bunch of features it's quite easy to imagine how you can have a you know a slim version of an app that you then add features to over time with a data platform you do have this kind of minimum um set of progress you have to make for you have anything to show you've got to build a foundation you've got to get that plumbed in The balance here is we started to be really clear on what are some analytical use cases that we can deliver quickly.
Speaker: And we chose one that had relatively low complexity. We knew that if we joined three or four particular high value third party data sets together, whilst not core client reporting, would provide insight that could accompany the existing reporting happening at that time and could demonstrate value.
Speaker: And we worked backwards from that kind of minimum goal and figured out, okay, what does the ingestion pipeline need to do what version of a transformation pipeline needs to exist, what sort of monitoring would we need to operate just that scale, and you build to that spec.
Speaker: And you get past that milestone and you kind of come up for air and you say, okay, where to next? So that has really been the mindset from the start And I think you get more leniency once you've got to a certain ah degree down that journey where you've got a bunch to show, then you can start to move a bit more into ah a roadmap. Still kind of iterative, but knowing that the kind of core feature set is out there. So yeah, that first part the journey is definitely the hardest.
Speaker: And that's something that i hear a lot um of, you know you what we're talking about here is like the core foundations, right? um And being able to invest in the core foundations, I think every data person knows is really important.
Speaker: But, you know, showing that ah ROI to the business and allocat getting the investment and the time to be able to to work on the the bits which you're not gonna see ah ROI from straight away, or you know the the ah ROI is more that it doesn't break. um So how how did you, was there was their top level buy-in already? And if not, yeah how do you structure actually getting that um investment buy-in to enable you to focus in on stuff which, look, we're going to have to build this, but yeah you know we're not going to see some value from it for X time.
Speaker: Sure. Just coming past them already, let me give kind of give you two answers to that question as well. On the technological side, you've got to make smart decisions, and the benefit of a sort of greenfield project where you but you're thinking about this kind of MVP mentality,
Speaker: is you can build something with a modern architecture. You make sure that it's got the right interfaces and boundaries and it's pluggable such that you can change components later.
Speaker: So you can design a beautiful architecture and then actually kind of cheap out on certain components, knowing that they're good enough for an iteration and then swap them out later. So choosing the right design, using cloud native tech, using modern kind of API interfaces,
Speaker: meant we could actually reduce amount of work required to get to that minimum foundation. you know It's not true that um or just because you need to get to minimum foundation that therefore it's a huge enormous job, you can make smart decisions to minimise that.
Speaker: And then secondly, yes, we I did have good buy-in.
Speaker: there was an already an understanding that this was going to be a significant area of um growth for business. And they Media Lab were looking for somebody with conviction and a plan to suggest how we get there. So I had to manage expectations, I had to set a path for how we would do it, but I was supported all the way along and when I was challenged it was a very healthy challenge.
Speaker: You know, can we do this quicker? Can we borrow that bit rather than build it at least initially and maybe swap it out later? That would cause to think, okay, have i got the right architecture to have that kind of plugability? So those two things kind of came hand in hand.
Speaker: um And maybe we'll come onto this, but I think that simultaneity of making the good design decisions and doing the buying and vision bit, both those things are absolutely critical. And you've got to be fairly rounded or at least be a rounded team to cover those spaces.
Speaker: um I knew what getting myself into. So in many ways, I walked into kind of an open door really, but doesn't mean it wasn't there it wasn't a challenge. jump No, no, no, that's great. And I think the thing that I picked up there is like that ability to have that vision, but also understand the trade-offs along the way. And I think that's one of the things that We did definitely see um why why why people succeed is when they really understand the trade-offs of impact, architectural decisions, what what they should do, what are the consequences of it.
Speaker: um So i think that's really really interesting insight. and One of the things that we we spoke quite a lot about as well, which I think is often overlooked when you're talking about a a data platform and and data products is actually the importance of ah user experience um and the user interface, right? These people, analysts, and self-serve users need to be able to use the platform in a way that is intuitive. um how How do you guys go about achieving that?
Speaker: Yeah, that's a good question. And a lot of people want to sidestep UX UI because they think I'm building a data system. Why is that relevant to me? And in the early days, we didn't have any user interfaces apart from the third party BI application we were using.
Speaker: Fast forward to we've got lots of proprietary interfaces doing different things. So our kind of current UX UI has got a more obvious answer. But back then, I suppose we could put UX and UI into the same category as ergonomics. And In order to scale what we had, it was clear we were going to have to generalise some things.
Speaker: And when you generalise some things, you lose some ergonomics. You lose the kind of custom bespokeness that leads to intuitive behaviour. Maybe your metrics have got more generic names now, so people struggle to know what they mean in kind of more general contexts.
Speaker: We invested a lot in, you know, our premise was we've got to generalise, we've got to take things that were custom and make them into one thing that's highly configurable. For each one of those design decisions, we have workshops. Okay, how do we mitigate the impact? What are the things we need to do um to make it feel like a bespoke curated solution still?
Speaker: So I think that was really what it meant early on. um And you have to have a proper plan for that and you have to have the right layers. So our choice of the the eye layer, for example, a critical set of decisions was that we could customize things at a level um beyond what we were doing in the data model because we knew we'd need to rely on some of that kind large smart customization.
Speaker: We didn't lose that feel of personalization despite the fact the model is more generic. So I think Engineers are very good generalising things, that's kind of really what I think programming boils down to.
Speaker: um But maybe not so good at thinking of, okay, how do I preserve the look and feel to the user, whatever interface may be visual or not. So, bit of a generic answer perhaps, but that was I suppose a core thing we were aware of from the get-go and were always trying to mitigate. And think that remains true today as well actually.
Speaker: And when you're trying to sort of understand what would be an intuitive sort of user experience for your clients, for the users of the platform, like how how did you go about understanding that? Is this something that you're looking to just sort of think, just you're spending time thinking about? Did you do research? like how How do you actually yeah There's a listener sitting here thinking about, okay, I'm designing this platform. how How do you go about thinking about user experience and making sure it is right? You've got to involve users. but That's an easy thing to say. You've got to think hard about who and what's representative.
Speaker: It comes a lot down to the environment here. We were very fortunate. um Media Lab was and still is based in one location in London, so being physically together um meant getting together a representative group of cross-departmental stakeholders was doable.
Speaker: We assembled an Apollo Council really early on, which was like an internal forum for decision making and us bouncing ideas off. um Also, because Apollo is a managed service, not a SaaS product, we were already doing a lot of service delivery. you know We were effectively evolving the engine behind the scenes, but the outputs to users were great and we wanted to preserve that.
Speaker: And so the resources that were involved in that last mile service delivery were critical to us getting the evolution to the engine right. So you've got to involve the users, you've got to figure out who the right people are. um And yeah, I think that spirit has been maintained. We're quite a lot bigger now. So our whole kind of enablement and engagement plan is kind of multifaceted. We've got, you know, self-paced training, classroom-based stuff, knowledge base, customer success forums, temperature checks and touch points.
Speaker: We've got a much more rounded strategy now. But I think the thing that's always been consistent is that you've got to have a trusted set of engaged users from the start so you can get some feedback on your decisions.
Speaker: Excellent. so lot I'm sure people are interested to hear. um what what What does the Apollo sort of architecture really really look like? um yeah Talk us through and the the sort of core layers that you've you've built out.
Speaker: Sure. Of course, there's some secret sauce in here, so i'll keep some things a bit more top-line than others. but Effectively, it's a modern data platform. So um we have a data collection layer, a data ingestion layer, data transformation layer and a data consumption layer.
Speaker: ah that is the way that data flows through the system. We rely on a few third parties to cover the data collection area. So um domain specific connectors with APIs, maintaining those and all of the version changes is not our core business.
Speaker: Not to be one to get involved in, we're leaning on people to do that really well. Data ingestion is something we're particularly proud of. um Again, making the right architectural choices. We use Avro.
Speaker: as a file format which all data we ingest into the system follows. It means all the data that arrives has a degree of validation built in, it's typed and it can be processed very efficiently.
Speaker: So our data ingestion pipeline which is responsible for taking Avro data, landing in a storage bucket and updating a table in our warehouse is an embarrassingly parallel piece of architecture that scales ad infinitum, extremely low cost.
Speaker: I think that's an example of solving a technology problem in a very purest way and reaping all the rewards there. Data transformation is a bit that's really hard in a system like this.
Speaker: Landing batch structure or semi-structured data into a warehouse is not a difficult thing to do at scale with with some right kind of design choices, but going from that raw data, let's call it, into business information joined up, KPIs derived, aggregated, ready for the use cases is an enormous job and a job that's hard to scale because that doesn't scale ad infinitum. You can't embarrassingly parallelise that.
Speaker: you're trying to produce some small number of outputs, that's going to get slower the more rows of data you've got and the more complex your logic is. We've used DBT here um and our DBT co-base is, um I've not got a stat to back this up, but I suspect it's one of those comprehensive DBT co-bases out there.
Speaker: um It is enormous and gets a huge portion of our engineering attention. So we have got awful lot of patterns in there about how we have generalized using macros, um how we enforce data of quality at multiple levels, how we do kind of pipeline retries and only process change data every every time data changes.
Speaker: So um that is our transformation layer. And then ultimately, we rely on power of modern clouds to be exposing all of this in a highly performant queryable layer, right? So we're using BigQuery and Google Cloud as our storage.
Speaker: That's where we keep the result of the ingestion pipeline and the result of the transformation pipeline. But that then means you've effectively got a versioned interface that all of the tools and modern data consumption use cases can pull from.
Speaker: So yeah, that's sort of the quick whistle-stop tour. We're quite agnostic about the data consumption layer. um Well, i should I should qualify that. Ultimately our users, our clients are buying the data consumption layer so we give that an awful lot of focus. But what I mean is architecturally, if those things change, if those technologies need to change over time, they can do because we've got the data model sorted and that really has been the key bit.
Speaker: So we've got a very very significant BI presence, we've got some very significant proprietary marketing intelligence services that are top of the data. But fundamentally if those things need to change and evolve as they will with our roadmap then We're very confident we can do that with the existing architecture Amazing and you've said obviously like the you know the hot the Apollo serves 50 50 plus clients with different data sets different um Approaches the transformation layer and designing a data model that yeah, ah
Speaker: that is is hard enough without that complexity and without different clients. Have you applied, like, is there like a a unified but approach or how do you scale that sort transformation layer in DBT when there's such varying sort businesses and, and you know, the the nuances of their own data um that's coming in?
Speaker: Sure, and that is a secret resource. But yeah, I can say a little bit about this. So, This standard versus custom thing is an enormous challenge that you have to get right, you have to keep iterating on.
Speaker: We do customization as late as possible in the transformation DAG. um Part of the reason we're able to do that is because we have created a domain model. So I skipped this when I was talking about the data regression pipeline, but we don't just willy nilly bring in any sort of data into the platform.
Speaker: we go through a ritual of defining a data feed, which is a meaningful thing that produces you know specification for what we're collecting. And where it's a common set of data that we would pull for many clients, so to each client would be their own data, that's coming through in the same data feed. So the schema is is guaranteed identical.
Speaker: and that means that the transformation layer there is certain processing we can do on top of that data that is generic before we get into client-specific processing. We do that without compromising any of the security considerations and the total logical segregation of data but it means we can do custom later and that is a big part of it.
Speaker: Another big part of it is What does it mean to say something is is custom versus standard? um There is a black and white answer to that, but you can generalize 15 things that are custom into one thing that's generic with a configuration.
Speaker: And that is our constant pattern. We're constantly be building things that are custom. And then six months later, we've done a few more that are like that. and we go, hang on a minute, there's a pattern there.
Speaker: We need to generalize that. And so kind of pull it up a layer into a configurable module. So our kind of macros folder in our dbt cobase is I suppose where all the magic happens, but it's this design obsession of if you're going to generalize, you've got to think about the ergonomics and you've got to watch that list of custom stuff because that's going to prevent you from scaling operationally and technologically. So sorry, I long asked your question, but that is effectively how we've done it.
Speaker: No, no, it's really interesting. I think the yeah it ties back to that that scalability and modularity point in your data models. And it seems that you you you maybe you wait for some of when there's new use cases and say you you just you you custom build, but then and when they repeat,
Speaker: you can spot you're spotting them trends and then looking at a way to to standardize that. and i think yeah whether you're building a marketing intelligence platform or or ah you know financial financial data modeling, I think that that type of mindset could definitely be the approached. You want to try and spot opportunities to to reduce and reuse um areas. i think that's a really interesting point. Just a really small thing on that as well.
Speaker: This is where your early design choices either limit you or enable you here. Something like DBT, it uses ginger syntax. So you've got that kind of dynamic ability from the get-go.
Speaker: If we had chosen something more rigid, we maybe wouldn't have that possibility. So you've kind of got to make those early design decisions with the full vision in mind. I think that's that's a real challenge. like There is a tension between MVP what it doesn't tell you in the academic textbooks is you can develop MVP, but you've kind of got to already know what the end game looks like, or at least enough for some of the key design decisions. So just wanted to kind hark back to that.
Speaker: Nice. what That's obviously in the transformation layer. What about on the ingestion point? Because you're again, many different sort of ingestion points and you need something that is very scalable and isn't going to break because you're ultimately, so you know this this platform is a is is a line on the P&L for both for you guys and also for for your clients. so you can't have things breaking. how What were the engineering decisions that allowed you to build sort that a fault tolerant, highly scalable, low cost pipeline for for ingestion?
Speaker: Yeah, good question. I've been burnt in the past by not having enough flexibility there. Ingestion is an easy problem to solve, but if you've been accruing data for 50 clients for 10 feeds for two years, pulling some history every single day, that is a lot of accrued data. And if you now decide you need to make a change to that,
Speaker: That can be non-trivial. This is where tech debt can explode if you don't have a plan for it. So a really key decision, and it comes back to the Alvaro point, is choosing a file format that can handle schema evolution.
Speaker: So we have an ability to evolve a data to feed from one definition to another backwards compatible definition. And we've had to use that capability probably 100 times the last four years.
Speaker: It would probably be narrow side to presume we wouldn't have to use it quite a few times again as well. Our common platforms bring out new data points as they innovate. And we want to get these into our single source of truth because they're useful for decision making, reporting, whatever the use use case may be. So that's a key thing.
Speaker: and Another is Data warehouses can often get very frail where everyone is terrified of losing data and obviously that is a good mindset to have.
Speaker: We wanted a design where we could theoretically rebuild the data warehouse by replaying events. You know, in the worst case scenario where Google Cloud don't meet their disaster recovery guarantees for whatever reason, can we re-populate the warehouse and relatively quickly by replaying events?
Speaker: We built an architecture that would enable that. So having full lineage all the way down the pipelines, you know, exactly what came from where, or when, and having the source data archive so that you can reprocess if you need to was a ah key decision. So that's a little bit about how how we do it there.
Speaker: Very, very interesting clearly important, right? um I think the The final thing on your architecture approach is this 80-20 model, loan which we've sort of covered, right, of having this sort standardization um for 80%, but then you do need to have the customizable um elements to be able to move quickly and to be able to, you know, there's not always one size that that fits all. so Yeah, for for folks that are ah thinking about this in any type of platform context, um how does your the approach work in practice?
Speaker: Sure, I suppose maybe the very first thing is you've got to make the call right on that scene, that kind of the and you need outstanding analysts. um It's the sort of thing that perhaps has been a role some people have suggested ai might replace and we're huge advocates of AI, maybe we'll on to that a little bit, we're using it all sorts of ways, but the ability to really form relationships, get under the skin of a business problem and understand the requirement, often you have to extract it from conversations, it's not given to you,
Speaker: and then to have enough of a knowledge of what the engineering team are doing to to be part the conversation where you can make decisions on that standard versus custom. The right way to approach a use case that looks very unique is is the be all and end all. So we have got some fantastic analysts that are excellent at that.
Speaker: um and they know And they have to be domain oriented. So these aren't kind of business owners that we've flown in from somewhere. These people deeply understand our domain. Many of them have come from kind of digital marketing careers and kind of pivoted towards this role.
Speaker: So we've been very fortunate there. I think it would be extremely difficult if we didn't have those roles. um And I think for the 20%, not all of the rules go out of the window. So you might be to standardise everything.
Speaker: Okay, but can you standardise your approach? Can you standardise your principles? Can you standardise the way that you deliver the stuff that's custom?
Speaker: what are the appropriate documentation, what sort of training you need to give your internal people to sort of positively indoctrinate them into that way of thinking. So even though the 20% is only 20%, we've probably spent much more time working on how do we do that bit because we know that's the bottleneck. So if we can get that as well as it can be, the sort of 80% will take care of itself if you kind of follow what i'm saying. So that's been ah a key thing.
Speaker: Very nice. I think the building it is one thing, right? um And then you once you've built this platform, how do you then look after it and maintain it? um to make sure that it's going to be working reliably in production. um what What have been the sort of um yeah technologies, approaches that you've put in to ensure everything stays stable?
Speaker: Sure. Yeah, you're sort of um heavily implying that there, but I would completely agree. Operating it completely blows building it out the water in terms of effort and time.
Speaker: um You've got to have really, really good support. You have to invest time in centralisation of monitoring and alerts. If you don't do that, you are um hampering your support team's ability to get on top of stuff.
Speaker: So we've invested, we probably spent more time building tools for ourselves than we do building features in the platform because the operating is so important and that customers have a great experience if the outputs are what they expect them to be and they're accurate and up to date and if there's any problem someone's on the line to fix it. So we've invested a lot of time in giving our support team the right tools.
Speaker: um Customer success and enablement and training and all those kind of advocacy touch points have been a big part of it too. And across the Apollo team right now, we've got a real spread of resources. here We've got data scientists that work on the the measurement side of things.
Speaker: We've got data engineers, we've got back-end engineers, we've got data analysts. We've got analytics domain people that are kind of technical customer success managers. We've got support analysts, support engineers.
Speaker: So you do need a real span of roles with a really positive, collaborative atmosphere for this to work really well. um I think we communicate really well as a team. Being in one location helps.
Speaker: Everyone being kind of Slack natives has been really important. All of our monitoring, alerting streams into there as well. so kind of creating a home where the team lives, a virtual home,
Speaker: as a concert and then you've got to keep talking to users. You can't assume that everyone is happy all the time. You need to be actively finding out what can we be doing to solve net new use cases, other areas where we're kind of falling down in training or enablement.
Speaker: So it's just maintaining a focus, I think, on all those things. You've obviously said a few, you mentioned AI previously as well and like the team that you've got and there, he sounds like an incredibly talented team, lots of diverse skill sets, but from my understanding, it's been a relatively lean team in the building of such an impressive platform.
Speaker: um What challenges has that crew provided? And yeah, how how do you you build lots with ah with with such a lean team? And I'm sure that's what we're seeing a lot in the industry anyway with ai Leaner teams are appearing and and and yeah the I think gone are the days of these big massive of centralized data teams, but can you see how you're seeing that evolve and how you've built something like this with a ah relatively lean team?
Speaker: Yeah, I appreciate the question because I did make it sound like we've got this enormous team. It's not the case and the size the team we have now has grown because of the size of the demand. Sort of five years when i joined, it was a much, much, much smaller team.
Speaker: Actually, that lean team thing has been a real blessing because that also puts some healthy pressure on some design choices. um We have made sure that our kind of core developers can do everything. So we don't have separate DevOps, we don't have separate front-end engineers, back-end engineers.
Speaker: We've chosen technologies and architectures that can be operated from one small team. so and the engineering team today is still pretty small and we do everything from you know building the components building the uis front ends back ends apis storage layers infrastructure testing ci cd all of that is done by core team with core skill set that centers around python and sql and we've chosen technologies that can be um learn quickly by people that skill set. You know, our front ends are not built in JavaScript, for example, and not because we have a problem with JavaScript, but because that was a skill set that kind of wasn't part of our core team So I think the Leaner team is a blessing. Those architectural choices have absolutely backed out.
Speaker: um And I think Growing people and nurturing talent has been something that I've personally found huge joy from. It's been a big energiser as we've gone ah gone across the journey. So whenever there's been a new use case, the first place that we've looked is into the team internally, who wants to upskill, who wants to learn, and there's more than one hand goes up and you you figure out how you do that all together.
Speaker: um So there's a lot of homegrown talent and choosing technologies that are very accessible. So I think that's a big part of it. And even with ai we have brought in a couple of specialists to help accelerate our efforts here, but that's not at the expense of upskilling a lot of our data people as well.
Speaker: um And I think the focus we've had, the obsession we've had on getting the data right has ended up being of critical importance for the ai agenda, right? Because all of these AI tools just need nicely structured, clean data, which is kind of where our focus has been.
Speaker: It's really interesting, isn't it? I think it's often overlooked sometimes when you're in that design phase of, you know, we've already spoken about like the reliability and the maintenance, but like what tooling and what choices are you going to make with the team that you've got? and what is the what is the cost of that that that team and also going out to hire, you know, if you're you're going to choose some some very obscure um yes skill sets and technologies that are are pretty niche, you're going to have struggle hiring um and bringing people into the team and that means you're vulnerable to to talk to when your some of your specialists do leave. So i think it's a point that's definitely overlooked sometimes when people are are setting out.
Speaker: um so interesting that you've had to and it's been a ah strength of of your team's build um i think it will also tie to this you know we spoke about at the beginning with the uh you know to get the investment um for to build a platform but then know now you've been here you've been with what on five five years or so not building this platform What's now the ah ROI that you're you're seeing and how has that affected, how has the platform like Apollo affected the the business and your your team?
Speaker: Sure, and I guess there's sort of a series of connections here, right, from how Apollo works to kind of how Media Lab presents themselves on market, how we work with clients. Apollo is not a cost center. know you mentioned you can have a line on the P&L later, I suppose that is literally true.
Speaker: um But Apollo is a profit center for our client partners. um and And the way that we sell in Apollo to clients is based on the tangible returns that it drives.
Speaker: um So, you know, not having this is sort of not an option um when we want to be driving outcomes for our clients.
Speaker: I think it's been a relatively easy sell in that sense. because we've been very, very clear about what Apollo enables rather than treating it as a cost of doing business.
Speaker: It unlocks decision making that allows us to deliver the outcomes a client wants at better rate than if we didn't have it. So Apollo is, I think, fundamental to that story that Media Lab tell our clients, which is absolutely true. It's something that we is core to our DNA.
Speaker: And I think we find when we're prospecting for new business, um many of our clients have never had a system that even has all the data in one place, never mind access to the sort of intelligence layer in the measurement suite. it's so that there is this remarkable...
Speaker: um tension between all of these AI use cases that are very, very loud in market. And actually a lot of clients are just struggling with having the data in the right place, to kind of hygiene stuff sorted to able to exploit that AI.
Speaker: So I think it doesn't it doesn't take away from the need to do that. And that's where we've prioritised. Awesome. And look, that's a huge part of the business and yeah your USP essentially.
Speaker: What's next for for you and your team and the ah the Apollo platform moving into the future? Secret source question again. suppose i suppose We are wanting to unlock more of the value from the data we've got in Apollo.
Speaker: It's used heavily for reporting and measurement. It informs our planning decisions, but with um marketing services becoming increasingly automated, increasingly AI decision-led, we want to make sure that the unique layer of intelligence we've got, that kind of cross-channel intelligence,
Speaker: tied first party outcomes, that we can explore that right way across the marketing campaign lifecycle. So we are looking at the agents that we've already built a bunch of agents for our internal operation of Apollo.
Speaker: um But we're identifying what are the things that the kind of the real time signals that we should be making buying decisions off the back of that Apollo uniquely services. So AI for us is an accelerant to exploiting those things and problems that used to be slow and deterministic can now be kind of not fully deterministic, but you can have a human reviewing recommendations and making decisions. So we're kind of cutting down the time from data to insight, data to action.
Speaker: um So yeah, we most our roadmap for next couple of years is very, very focused on how we activate the intelligence in Apollo ah more quickly, for more people, in a more automated way,
Speaker: um Fundamentally everything we're doing is how do we maximise investments that clients make in marketing. So using the data to drive the outcomes they're trying to drive, measured in the right way, working with the right kind of partners to make those changes mid campaign, that's kind of where we're at. And that point of transparency, you see what we see, we can move budget around if we need to because we're independent, is kind of key, I suppose, to making the whole thing real rather than just sort of dream.
Speaker: Awesome. Yeah, I think it's ah it's dynamic world, what we're moving into faster than we've seen. But i think it sounds like you've got a a lot of proprietary assets, which you can really start to squeeze with the tools that are coming out, which ultimately yeah passes on value to to your clients.
Speaker: Adam, before we were almost a time, but I'd be keen to get your view on what is the yeah key skills that ah data um data professionals in general need to be working on in in the the current state of play and and preparing themselves for for the future?
Speaker: Gosh, high stakes, no pressure. um I think I would probably ah give two different answers for dead professionals versus data leaders. I think data professionals need to just keep on digging deep around the the modern stack.
Speaker: I think the modern data stack is fairly well known these days and all of the emergence of new tech doesn't take away from just being a solid operator of of those core tools.
Speaker: So I would say that. um I think data leaders have just got to be really rounded. I think more so than ever, you've got to be able to design the system, get the architecture right.
Speaker: You've got to build the team. and The team is absolutely everything. You have to foster the right culture. You have to develop and nurture people. you' got to hold on to people. And you've got to you've got to manage up and you've got to manage across. You've got to sell in well, gain buy in,
Speaker: be building relationships with other leaders in the business. You've gotta to kind of do all of that at the same time. So I think data leaders have got to be really rounded. There's a bit of pressure there. um And yeah, data professionals, I think the the core stuff still applies and lot of people being worried about being left behind with AI.
Speaker: you've clearly got to engage in some of that emerging tech, but you can do it in steps. People get, I think, overwhelmed by how far they are from some of the stuff they might see being shown on YouTube.
Speaker: You know, they they can install a code assist in their IDE and they can take a step in that direction. So, um yeah, I would say don't freak out if you are kind of on the ground.
Speaker: Just, yeah, keep digging and and look for the thing that pushes you further along that track. Excellent. And look, i before I let you go, Adam, I think it's always good you to to reflect back. um what What have been your biggest learns from this journey that you've been on and and what advice would you give to, um yeah, I suppose anyone that is looking to build out a platform platform capabilities?
Speaker: Gosh.
Speaker: I think the thing that comes to my mind I can't get out is you've got to just be really passionate so you can dig deep when it's tough. um I've absolutely loved the journey the last five years. i knew it would be a big challenge.
Speaker: It has been a challenge. It's been the most rewarding chapter of my career. um i have relied on, deeply caring about what we're doing, both in terms of an obsession with the problem we're solving, but also being really brought into the environment, really enjoying the people I'm working with, being totally aligned to the vision of the board. You know, that's been a blessing, a lot that's been external, they've not been my choices, but I think if you're going to do it, you've got to be you've got to be very, very sure that you want to step on that journey and you've got to passionate about it so you kind of dig deep.
Speaker: Amazing. Adam, it's been a pleasure to have you on the show. I think it's a real insight into engineering excellence, but also pragmatism of building a platform that clearly is driving tremendous so ROI.
Speaker: um And there are some real nuggets in there, I think, hopefully for for our audience to to apply to to their worlds. And yeah, I suppose it was equally a ah really fascinating insight inside Media Lab um and what it would be like to to work for someone like yourself as well.
Speaker: Amazing. Thanks for having me on. Really appreciate it. Really enjoyed it. Brilliant. That's it for this week, folks. We'll be back in a couple of weeks. Thank you and goodbye. Hi everyone, just a quick one from me.
Speaker: If you've enjoyed today's episode, I'd be so grateful if you could hit that follow button or leave us a rating. Even better, pass the show on to a friend who might also get some insight from it. It really helps us grow the community and continue to share amazing conversations.
Speaker: I also wanted to take a minute to talk to you about Cognify. Those of you that don't know, Cognify is the leading recruitment partner for modern data teams. We help some of the world's best organizations scale data and drive real value from the hires that they make.
Speaker: you're thinking about building a team or making a hire and you're struggling with talent or just want some insights on the market, then I'd love to jump on a call with you and tell you a bit more. Equally, if you're looking for a job and want to find your next dream role, then reach out to myself or any other Cognify team.
Speaker: We'd be happy to see if there's anything on our books that we can help you with and give you general advice on the industry. Finally, big thank you to Omni, this season's sponsor. If you'd like to learn more about the AI analytics that Omni can deliver you, then check out the link in the show notes or come speak to me. i can happily point you in the right direction.
Speaker: Again, thanks for listening and look forward to seeing you a few weeks time.
