Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
045 - Wise: How a lean team of Analytics Engineers supports 300+ Analyst
197 Plays30 days ago
Most data orgs scale their AE team as they grow. Wise didn't. Moritz Kerstan runs a lean team of 17 supporting over 300 data practitioners — and he's got a clear thesis on why that's the right call. In this episode, we dig into how Wise structures their analytics engineering function, how the team operates without hierarchical authority, what good prioritisation actually looks like, and the honest truth about building with LLMs inside a large regulated fintech.
Recommended
Transcript
Scaling Challenges and Building Sustainable Models
00:00:00
Speaker
So the scale is really mind blowing. And think a lot of the things that work at smaller scale really, really stop working at this level. Over time you realize, okay, i actually need to build an incentive system for other people to be able to do that quickly. I need to give them tooling that makes it quick for them to do it. I need to give them guardrails and like a design system. That means they build these models in a sustainable way in the first place. And that's for me really the only way to deal with the scale.
Introduction to Omni and AI Grounding
00:00:25
Speaker
Today's episode is brought to you by Omni. Most companies I speak to want AI analytics but are failing to put projects into production. That's where Omni is different. It's the semantic brain that grounds AI in the heart of your business logic, giving you governed answers, whilst also the depth to identify root causes. It's intelligence everywhere, from your spreadsheets to their chat feature, even within your product. Omni moves you beyond the dashboard. Don't just take my word for it. Trust teams like Perplexity and Synthesia that are already using Omni to deliver intelligence that people trust.
00:00:57
Speaker
Check out them in the show notes or visit omni.co. That's O-M-N-I dot co. Now, back to the show.
Meet Moritz Kerstan: Leading Agile Analytics at WISE
00:01:05
Speaker
Hello everyone, welcome to another episode of the Stacked Data podcast. Today my guest is Moritz Kerstan, the Analytics Engineering Director at WISE, which is one of the world's leading fintech companies, moving billions in money across borders every day.
00:01:23
Speaker
Moritz leads a deliberately smaller, should we say, um an agile AET team supporting an incredibly large data organization, um thinking 150 plus analysts. um And he's built a clear point of view on why he believes that's the right model to make it work at the scale that they're operating at.
00:01:47
Speaker
um and what it actually takes to drive impact without a top-down mandate.
Moritz's Journey into Analytics
00:01:53
Speaker
We're going to dive into his philosophies behind keeping the team agile and how his team operates to empower and units across the organizations, um their abilities and how they're using and building LLM integrations inside a large enterprise and all the bits in between.
00:02:14
Speaker
Moritz has been a friend in the industry, he's been a big advocate of that at data meetups.
Empowering Agile Teams at WISE
00:02:20
Speaker
We'll actually be in WISE's offices for for one later in July and their team have also spoken at few. So it's been a long time coming to have you on the show, Moritz. Yeah, thanks for for joining me.
00:02:32
Speaker
Thank you very much for the warm welcome. Thanks for having me today. Excellent. Well, I've obviously given a bit of an intro, Moritz, but it'd be great for the audience to get to understand a bit more of your career journey and yeah how how you come to leading analytics engineering across WISE.
00:02:51
Speaker
yeah Yeah, of course. sir Pretty much by by accident, like all whole good careers, I would say. So how this all started out really was after was the uni and like my my first job in ah and the startup, I was looking after there marketing.
From Marketing to Data: A Self-Taught Journey
00:03:04
Speaker
So I wasn't, I never made a career plan like, hey, I love data, I want to go into data. i Didn't really study anything in uni in this direction or anything like that. And my my job there was like, hey, do some good marketing, right? Like get some customers for for a company. And to do that, i I'm glad to use data, right? like I realized pretty quickly, actually, the way we're doing this right now doesn't really work. You you need to understand and the results of the campaigns we run. You need to understand your customer behaviors.
00:03:32
Speaker
And from there, it really started very, ah very simple in a self-taught way of like, maybe I should put this in the bubble sheet. and wait, people talk about this thing called CFO. And I kind of kind of started doing more and more of that and then also realized, okay, wow, this is really my thing.
Finding Motivation in Impact
00:03:48
Speaker
You even get the short-term dopamine hit of like, oh, I built this really cool model and it really helps me make these decisions now and work much faster than ever before. And the good thing is I was in a team of other people who I was doing marketing and very quickly was able to help them with the stuff that I built as well.
00:04:06
Speaker
And that That really motivated me. I really felt quite happy and intrinsically motivated about that. So from there, I kind of always followed this path of what's the biggest impact I can have right now with the skill set that I have, with something that kind of day-to-day intrinsically motivates me to do. So went back then in the time school. was still in the marketing and analytics team of realm.
00:04:32
Speaker
I built a team there over like three, four years.
Data as a Tool for Achieving Goals
00:04:36
Speaker
ah doing i always have been very, very motivated by the the end outcome, right? Like i don't I do enjoy building models, coding and stuff like that.
00:04:46
Speaker
But what really, really gets me out of bed in the morning is like, okay, we're trying to achieve something new for our customers, or I'm trying to achieve something for my colleagues. And I've always seen data a bit more as like a tool for that, not like the intrinsic reason I do this. It's more like, okay, there's another intrinsic reason, i something I want to achieve. Data is kind of the way there. um And then over time, while working in the marketing, it's real there I kept complaining to to my lead about how bad all of processes were and how tooling was and all of this, all the stuff that frustrated me day to day. And then eventually said, okay, go and build a team and try to fix it. And that's what I'm still doing now, more or less successfully.
Focusing on Impact over Vanity Projects
00:05:27
Speaker
Excellent. i think that impact focus is... yeah I mean, it's come, I think it's such an important part of the of a data person's skill set, which is often undervalued and I think people don't necessarily get so caught up in the technical and and the building that they forget maybe what the impact of what they're building and why they're building. I know from working with you, that how important that that that is. What what a drew you more to that sort of analytics engineering piece rather than um you know data engineering or BI i and how this is all sort of, you know that why are they glue in the
Flexible Roles at WISE for Greater Impact
00:06:05
Speaker
middle?
00:06:05
Speaker
Yeah, it's an interesting question. and We intentionally kind of keep these lines very fuzzy and general advice or in or my team especially, I try to keep roles very open-ended. I never want to end up in a situation where I have someone sitting there who's like, I see a problem, I know how to fix it, but it's not my job title, so i can't do it. um So generally, im I think I'm probably kind of going in between all of those job titles, and as you say, kind of doing the glue in the middle.
00:06:31
Speaker
um To me, that was where the biggest gap was. like I think we we were very strong in like kind of the individual pieces of the puzzle. Oh, we have a really good air flow hosting or something like that, for example. But what was missing was this kind of end-to-end. If I'm an analyst at the company, I want to at the end of the influence the decision here. And that means I need to get some data live quickly. um And to do that, you had to kind of navigate all these different systems and teams and different parts of this like almost user journey.
00:07:02
Speaker
And that's what I wanted to fix because I saw that's the biggest gap. Like no one else is trying to own that at the moment. No one else is trying to do everything together.
00:07:12
Speaker
And we've mentioned it at the start, um but you guys at WISE work at a pretty large scale, I think it's fair to say. and Yeah, help how'll contextualize that for the audience.
00:07:24
Speaker
Yeah, for sure. so um we have about 8,000 WISE, so people working here, and we have a data team of more than 300, that's analysts and data scientists. um In general, we work in empowered teams. So the idea here is that the people closest to the customer understand the customer's problems most and best, and they are empowered to make decisions on how to solve those problems. And they also work in a team that has different skillsets, right? So we have analysts, data scientists, engineers, PMs working together and these like embedded unit units.
00:07:58
Speaker
and That's generally the scale. So we have lots and lots and lots of um kind data producers and and data practitioners and they're embedded all over the company. They're not like in a central team. Combined with that, data is also really democratized. I'm not just saying that. i think we 5,000 weekly active users on a BI tool, something like that. So the scale is really mind blowing.
Sustainable Scaling through Incentives and Guardrails
00:08:23
Speaker
And I think a lot of the things that work at smaller scale really, really stop working at this level. um it's It's super interesting the of change of skill set almost that I also have to go through for for a long time and develop a lot on where at smaller scale, some things are just totally not promised because you can very directly influence them. You're like, oh, okay, we have like 10 models that are really bad. Cool. I'll just rewrite them. That'll take me, you know, half a week or something like that. And cool, we fixed that problem now with AI half a week. Back then it would have probably taken me a couple of months. and
00:08:59
Speaker
And over time you realize, okay, I actually need to build an incentive system for other people to be able to do that quickly. I need to give them tooling that makes it quick for them to do it. I need to give them guardrails and like a design system really. That means they build these models in a sustainable way in the first place.
00:09:17
Speaker
And that's for me really the only way to deal with the scale, especially in comparison with what you mentioned, where we're running a pretty lean team.
Advantages of a Lean Team
00:09:25
Speaker
So the reason for why we're running a lean team is kind of twofold. So on the one hand, what I've noticed is that this way you really, really end up with people who are intrinsically very, very motivated and do really high value work all the time.
00:09:41
Speaker
I think what I've noticed is like, if I give a team, data engineer or analytics engineer, that team will find work for them. Now, is that work always really high value? I wouldn't say so, right? like At the end of the day, there's so much demand for data that you will be busy, but you won't necessarily have like customer impact if you're those roles.
00:10:00
Speaker
Now with a more lean team, it doesn't happen so much because people are really aware of like, okay, this is a precious resource. We need to be very, very i clear on where we're deploying that resource, the time of people who work in this realm.
00:10:13
Speaker
And you end up with very, very high impact work that has pretty clear customer impact at the end of day, or at least a very clear story internally for the stakeholders, like what has improved. um that's That's one reason for it, it's like really stay on the high impact side.
Analysts Taking Ownership with DBT
00:10:27
Speaker
The other side is kind of how WISE runs. So at the end of the day, we're all about making a really fast, lean transparent product for our customers.
00:10:38
Speaker
And a big part of that is this kind of motto we have called minimum fees, where we basically say, okay, the way we price is look at our unit costs, then we're a profitable business. So we have a stable margin on top of that.
00:10:52
Speaker
And everyone here is quite focused on reducing that unit costs. What that actually means is become smarter about the way that we use all of our systems. So that's across the company, but also part of data, right? Like we don't really have the luxury to say, oh, just forex the warehouse of data is a bit slow.
00:11:08
Speaker
We need to come up with a way to do that. And we pass that on to our customers, basically drop pricing for our customers over time when we managed to drop unit costs. we get more customers that way and so on. So that's our flywheel.
00:11:22
Speaker
Part of that then also means my answer to our customers, me me saying, hey, I need money from our customers to do something in data i can't really be, oh, I'll just 10x the team, right? and Because that won't really fulfill our mission. I need to come up with a smarter way that scales.
00:11:38
Speaker
Interesting. So how many people are actually in your team? ah Currently, it's 17. The latest numbers on the analytics engineering side.
00:11:48
Speaker
and We actually, for a long time, like for two to three years, I think, had a team of about five only.
WISE's Approach to Prioritization and ROI
00:11:54
Speaker
Because initially, we were very centralized. We were kind of building out the basic basic platform stuff. We weren't using dbt before that. That was our first big project. It's like, well, we need to introduce dbt.
00:12:05
Speaker
And not only do we need to introduce the VT, we actually need to make the majority of our pipelines incremental and we need to get our analysts to build pipelines that are incremental. Advise, the analyst role is really wide, like the ownership is really, really wide. So you're expected to influence really business decisions and and a better product with with the rest of your team.
00:12:27
Speaker
but you are also responsible for the upstream transformation of your data and to actually be able to take kind of raw data coming from product services or external data and make that actionable and useful.
00:12:38
Speaker
And so our team's ethos has been to enable analysts to do that more quickly, right? For them to worry less about this transformation bit and dbt was one kind of piece of that puzzle.
00:12:50
Speaker
um But yeah, we're pretty, ah if you look at the ratios of like how many data practitioners per analytics engineer we have, and I think we are very, very lean. Yeah, yeah. I mean, there's and definitely, in terms of percentages, there's definitely ah plenty of other organizations that operate ah similar scales to WISE where they, you know, maybe at their their AE teams are probably seven, eight times the size um of of what you guys operate.
00:13:18
Speaker
ah How would you, know, these other fintechs and wider businesses maybe that have, you know, 70, 80 plus analytics engineers, obviously that you'd expect maybe their time to value or maybe that's the wrong terminology, but the speed in which they can deliver impact and, you know, not be a bottleneck is probably, might met might be quicker. how How do you guys balance that internal when, you you know, you mentioned you're such a scarce resource?
00:13:46
Speaker
yeah Yeah, I think partially this is course a little bit about definitions of like what specifically do we call analytics engineering and what do we call analyst work where like the work that a lot of the analysts do here are kind of in the analytics engineering realm. So some of it is like teams can unblock themselves because we give them the tools to enable that.
00:14:05
Speaker
The other bit is that we have to be incredibly clear on prioritization and this is also something and I ask everyone in the team to be incredibly clear about. So in general, we're in this environment as a cross-functional almost team where we can, or at least cross-domain team, where everyone has really strong opinions on what we should be doing for good reasons. Like a lot of people have actually really good ideas of what we should be building next, what we should support them with.
00:14:31
Speaker
And the vast majority of these things have positive ROI. They're like really useful things to do. And it's our job to look at all of that and say, okay here's the highest ROI project that we should be doing next.
00:14:42
Speaker
And that's incredibly difficult, as you can imagine. It really, really strengthens your strengthens your muscle of prioritization. though And what do I mean by that is that If we are not internally incredibly clear with each other, of fact why are we working on this and not this other thing, then we won't be able to go to the organization and say, hey, here's why we're not working on your project. It's not because we don't like you. It's because we looked at our kind of project sizing and impact sizing. And we've basically come away from that saying this will move our KPIs more than this other thing.
00:15:13
Speaker
and And then we're very happy for other teams to kind of pitch to us in our KPIs and say, hey, um actually, here's this other thing you could be doing. This would be the impact of it. And you end up doing a lot of um as at least yeah have quantification of projects and and you really end up trying a lot to have more of an intellectual argument than, ah oh, the most highest person in the room or or the most senior person just says what goes.
00:15:40
Speaker
Same for me. I lose arguments all the time with the team. on what I believe we should be doing versus what the data shows we should actually do. So there's this ongoing constant prioritization process.
00:15:52
Speaker
I think that's a a really interesting point. and I think with with the speed of which people can now build with the likes of AI um and the speed that organizations are expecting and data teams are being expected to move, it can be easy to get caught up um and not know what to work on. and I definitely see that when I speak to people that maybe don't don't know what impact work they're working on.
00:16:18
Speaker
What's your advice then, Maritz, to someone that's in the space? I think you've mentioned like tying it back to to KPIs, but yeah, what... Other than I suppose just tying stuff back to KPIs, how how do you how would you coach someone to be better at prioritisation and to apply that to their world?
00:16:35
Speaker
Yeah, really, really good question. andm I'm not sure I'm fully believe there, but I can i can teach i can i can talk through some of my learnings. um I think early on, i once put a a plan together in marketing analytics that was kind of like a long, long laundry list of all the things that people had asked us to do. And I got really good coaching, like, this is nonsense, delete all of this and you tell me what you should be doing and why. um And I think that's maybe a first step in a way of like really fully owning and fully feeling the responsibility and accountability of your time and really at the end of the day, like the the cost you create for the customers of the business. It's like, okay, you really, really should be feeling in charge and accountable in in the system of like, every minute i invest in this thing isn't invested in this other thing. Is that correct on the opportunity?
00:17:22
Speaker
cost trade-off that we do.
Quantification in Decision-Making
00:17:24
Speaker
More practically though, I think what I annoy my team a lot about is quantification, even if it's not really possible. So what I mean by that is a lot of data work, I think is if we're being real, pretty difficult to actually put a number on like can't really A-B test, giving half the company bad data and the other half good data and then say, oh, this is the impact that analytics engineering had.
00:17:45
Speaker
and We can do, I think, a lot of the lot of like kind of napkin math, though, that gives us like an almost on a logarithmic scale, like an idea of, okay, this thing at the end of the day will save two analytics engineers three hours or a week. This other thing will affect 5,000 weekly users, right? Like immediately you kind of get into these, you have a lot of like, it's not very precise, but you still, you kind of get an idea of there different levels here. And in my experience, I would say the the kind of If you do this in a very disciplined fashion, then you realize that's probably one or two things you really should be doing that are kind of like at least 10x of all the other long tail, the smaller stuff that you could be doing.
00:18:26
Speaker
um What I mean by quantification is, yes, tie things to KPIs wherever you can, but there will be lots of things that you can't actually measure. But I still want you to explain the kind of logical chain, like in a perfect world where you could measure everything. Oh, you could measure way that people make decisions. You could measure every minute that your data stakeholder spends on Looker and you could immediately tell like everything.
00:18:48
Speaker
Even if realistically we can't or we can't disentangle that from the overall, like we can't disentangle that from the decisions that the business that makes that then lead to growth, we should be able to explain it in theory. And that has helped us a lot.
00:19:02
Speaker
like Like that kind of makes 80% of projects kind of seem insignificant immediately and helps a lot with prioritization. So that would be my advice to like, A, really own it and B, quantify even when it's really uncomfortable to quantify.
Ownership and Career Growth in Data Roles
00:19:15
Speaker
I think that's, ownership is such a big thing and I think you have for someone's career being able to, one, you're going to move the needle in in your own organisation, but being able to demonstrate that you can work autonomously, that you can take things end to end and then linking it back, I say, to that value is is really, really valuable to to you to to yourself and to the organisation.
00:19:38
Speaker
um So what what does all what does the day-to-day look like for analytics engineers in in WISE? say You you hi it seems, for this quite wide breadth of of skill across um maybe AE, a bit of DE, etc. But yeah, what what does the life look like of ah someone in your team?
00:19:57
Speaker
Yeah, yeah, a very ah good question because it's quite quite varied and it really depends on the the domain and pocket working in. it Like we put a lot of focus on really working closely with our stakeholders and understanding what tooling-wise, like what do they do day-to-day and that kind of look very, like what a high-impact activity in a domain is can look very differently, right? Like maybe in some places, oh we need to work out these very, very core data sets and in other places more like, oh, we need to ah kind of influence people into writing better code or something like that. So I think the most surprising bit is how little code we sometimes write. Like I sometimes get people um even internally applying for roles whose story it is a little bit like, I don't want to deal with stakeholders. I don't want to have to like soft power everyone. I want to write more code. And often have to tell them, you're not going to be very, very happy in this role because of course this is like,
00:20:46
Speaker
kind of we take that a given, right? like Like you need to have the tech skills to execute on on projects in this realm. But what that skill in isolation is useless. If you're not good at getting people to actually use what you built, or if you're not clear on, Hey, here are things that I can't just by myself fix. I need other people to change their behavior and how they build model and how they look at data, even in things like how they presented data in the eye. I need to influence that and that will be real impact to our end stakeholders. like At the end of the day, who we think of as our like internal customers or stakeholders are all the people using data, making decisions with data. um And oftentimes, we kind of have to influence for an analysts, for example, to present data in slightly different ways or to to build slightly more scalable models or to add a lot of metadata to those models.
00:21:36
Speaker
for a positive effect to reach the end stakeholders. um So yeah, your day-to-day could really be, oh, maybe today I'm building a model, but it could also be like, I have a bunch of meetings and I need to really do clear user interviews. That's a lot thing that we do a lot, like really side by side, sit down with your stakeholders, look at what they're doing on their screen, let them rant a bit on everything that sucks about what we've built. and And then we kind of have this feedback loop between, oh we figure out in the individual domains, like really working closely with the stakeholders, what what are the recurring problems?
Building Scalable Solutions
00:22:09
Speaker
Like what slows us down?
00:22:11
Speaker
We come together centrally and then we build these kind of almost task forces of like, okay, we realized our Slack allotting is really bad for when our pipelines fail. So we then build a task force that centrally tries to fix that, right? Like build that stack alerts or the technical challenge with that. So you would go away code for a bit and then you would come back and test this with a small user group at first, but then you would scale it to the entire organization.
00:22:35
Speaker
And that's why our team has a lot of leverage, right? Like the of thesis of return on investment on our team is really we raise the floor for everyone. So in the past, we would often have power user analysts figure out the stuff really well in their team. Like, oh, this one team has amazing slack alerts, but it would never reach the rest of the organization.
00:22:53
Speaker
With us, it's like, OK, we take that and then we scale that to the entire organization. Really. So I think the point around locking yourself away and building code, that role ah in my opinion is is just going to to disappear. And if you stay with that mindset, I think you're you're really going to struggle over the coming years. We can get on, I think, towards the end of the episode about yeah where where the industry is heading and and and and AI and how that's affecting it. but I think you the
00:23:24
Speaker
The importance of being close to your stakeholders, close to the problem, is has never been more prevalent as execution becomes becomes easier. um And yeah if you're looking for a varied varied day and um yeah spinning ah a few different plates, then it it sounds like your team's a very good place to um to to to be One of the things I think, you know, AEE's been, it if data modeling's always been around, right?
Measuring Success with KPIs
00:23:52
Speaker
But DBT coining the term back in 2018, 2019, and it growing as a discipline from there.
00:24:00
Speaker
think one of the areas that a lot of teams across data engineering and particularly analytics engineering do struggle with is How do we actually contextualize our impact? You know, you've spoken about these KPIs and how you tie stuff towards the the business impact. But I think as an AE team, what what are your KPIs for success? And how do you how you look back know, a quarter and say, we've've we've done a good job?
00:24:25
Speaker
That's ah another really good question that I've been asked a lot and also have asked my my myself a lot. and Especially this looking back thing, right? like I think that is where you really realize, like ah like often forward-looking planning is like, oh yeah are we're going to improve this KPI, we're going to improve that KPI. If you then look back at your projects, it's like, did that actually materialize? lot of the time it didn't, which is not an issue. It's just a learning opportunity to kind of retro, but it is it is definitely something top of mind for me.
00:24:55
Speaker
um To more directly answer your question, our KPI tree kind of splits into two areas. So on the one hand, um i would say broadly we're trying to optimize productivity maybe, like maximize productivity. What we mean by that is can people who want to make decisions or often it's also quite operational use cases actually, for example, trying to monitor um where our customers wait the longest in a queue to get in contact with their customer support agent or something like that. Like there are lots of lots of different use cases that we actually support. Some of them more analytical, classical analytical, like, oh, what's our conversion rate? And some of them are actually quite operational. and
00:25:34
Speaker
we We look at all of that, kind of call that decision making roughly, and we say, okay, we wanna make that fast and quick and snappy. And then we trust that if we can do that, we basically build a better product for our customers. Like we understand our customers better, we build a better product. And luckily that trust, to be honest, I don't have, i have zero evidence for that.
00:25:52
Speaker
<unk> happening This is just an axiom that I take as granted. In the organization we have a lot of trust in that. like Luckily I don't need to go around and kind of get buy-in for like, oh, it does make sense for us to understand our customers better via data. It's very much ingrained in the organization. So I'm in luxurious position there. But high level, that's what we're trying to optimize, maximize the productivity with data with the constraint of not increasing prices for our customers, right? Like a lot of problem, for example, we have said, oh, data loads very slowly. I could solve that tomorrow by kind of 10x in our warehouse or something and that. But that would really break our cost constraint of like, yeah, you need to do this by becoming smarter, not just by throwing money at the problem most of the time.
00:26:35
Speaker
and in The productivity on the productivity side, productivity, no one can agree what that means, right? It's very, very intangible. So our approach to it has been to kind of triangulate metrics. So we look at survey based stuff. So we we call that analytics NPS. It's like any other net promoter score you would do for any other product. We don't ask what do you recommend is to your friends, but we basically ask how much do you trust the data you find? Like, does it look for you really? how How easy is it for you to find data?
00:27:05
Speaker
um and we kind of calculate the score from that. We also have kind of more harder metrics, maybe more more more direct measure with the metrics. For example, the lead time, like the first time that an analyst tries to ah build a model kind of locally in their preview environments, etc., to the time that it's actually live in production um connected to their sub-KPI's. They're like, okay, how long do they wait for CI checks? How often do these CI checks fail and stuff like that? um how How long does it take for data to actually load? It's basically ah with too many KPIs, have to definitely admit that. We're basically trying to get an impression of are we doing a good job from looking at the combination of all of those.
00:27:44
Speaker
And then specific projects, we often try to tie to one or two of those metrics. um This has some weaknesses and practice in practice for sure. For example, on the survey-based stuff, if you don't get really good sample sizes, it's incredibly difficult to actually measure and attribute improvements, that like small improvements especially that you made.
00:28:05
Speaker
In a way, that's good because in a way we want to make we want to only work on projects that have such a large impact that you don't have to worry about precision or anything else. thats Like, oh yeah, okay, you can clearly see the step change here.
00:28:16
Speaker
But in practice, you often end up like, okay, we know this is a smart thing to do. We can't really measure like 100% of like, oh, I don't know. What to do yeah what was about what was our incrementally, well was it what was our part of moving the needle there?
00:28:27
Speaker
Exactly, exactly. But yeah, that's how we're we're approaching our KPIs and how we understand and communicate that impact from our team. Really interesting. And and I was going to say, how how do you deal with then communicating that to and yeah leadership in the wider business? I imagine and and know why is is obviously a very sort of data savvy and and very data literate business. But yeah,
00:28:55
Speaker
what's the How do you approach that? Because I think that you're hearing that would be really valuable for some people um because i think that's often where the gap is sometimes in AE that we we know we're adding value, but maybe communicating that particularly to a non-technical stakeholder is is is where the challenge comes in.
00:29:13
Speaker
For sure, yeah. I think and two two points on that. One is we are part of our usual quarterly planning process that every team goes through here. So we put together very in-depth retros on how did our KPIs move and why did they move. And then positively, hopefully, here are the things that we delivered that made them move. So that's kind of built into the arch of like we we do have that accountability at all times because we get a lot of empowerment and autonomy, we have the accountability to then show this is what that moved. And the easiest way to communicate to the rest of the business, we actually, it's worth this investment is to show, hey, this this number moved.
00:29:49
Speaker
And I can explain to you why it's important that that number moves. That's maybe the second point is like, especially to maybe people who are not deep in the data world and secret or whatever is going on.
00:30:00
Speaker
um I think tying it to their almost personal experiences of trying to do their job, right? Like if we do work a good job, their job becomes easier. um So I need to be able to figure out, okay, definitely people in leadership positions, they rely on data and monitoring of what's going on in the business a lot.
00:30:17
Speaker
But even other teams that are kind of around us, I need to be able to tell them a story of, hey, you know how you constantly try to figure out what our conversion rate in this specific segment is and the data keeps breaking or it loads really slowly or you don't, you biggest problem actually is usually you can't find the data, right?
Improving Data Accessibility and AI Integration
00:30:34
Speaker
Like we're very data democratized, means our warehouse is a mess, can't even really call it a warehouse. Like everything in there, are very, very difficult to find what specific data can I actually trust.
00:30:44
Speaker
put a lot of work into fixing that over time and we can show in our KPIs and over time like, hey, this is getting better. Like it is now easier for you um to find data. For example, we deployed this product called Table Finder. It's like a Slack channel. and You go to you like, hey, here's my problem. What's, do we have data on this?
00:31:01
Speaker
And we kind of do a search through all our metadata to say, oh, here's the most trusted, most relevant data set for that. And we can go back and say, here's how many users that Slack channel has has, here's how many upvotes versus downvotes the answers get, and at the end of the day, here's the effect that has on how many people say that it's easy or difficult for them to find data. um And that should be felt by your end users, right? You shouldn shouldn't have to like talk them into the idea of you doing a good job. They should just generally recognize that their work life got a little bit easier by the thing that you developed or deployed.
00:31:36
Speaker
Really, really interesting. do you know what, you've set me up for a lovely segue there as well with that new um project that you mentioned. i think one thing people would love to hear and I know we've've we've discussed is what you guys are doing with LLMs and AI in in real practice. um There's a lot of talk in the in the industry. i do think everyone in data, we are in a bubble a little bit more of a bubble compared to and when you look at organizations that that don't aren aren't thinking about data, we that we're very far ahead. but yeah What are you doing and in in in the space? How have have you actually implemented the use of LLN's AI-based projects? What are some of the projects that you're working on?
00:32:21
Speaker
sure I have to remind myself that we're in a bubble. It's a good reminder from you because my my head is full of all the things we can't get do and all the things that are bad about it. But it's it's sometimes cool to get a bit of first perspective of like, okay, in reality, some of this is actually pretty um magical already. um I think what we practically, what we have most impact already materialized is kind of on the data producer side.
00:32:42
Speaker
So one thing that our team specifically owns is kind of a package of different tools, MCPs mostly, that we call the analytics MCPs. So the idea here is like you get lots of like um kind of connectors. So hey, this can interact with DBT well now, this can interact with our BI tooling, with our data, but also with like, I don't know, Jira for project management and and Slack and all of this stuff.
00:33:06
Speaker
combined with this kind of firm skills marketplace where we have a few skill files like AI skill files that kind of automatically invoke whenever you try to do a data related job.
00:33:18
Speaker
And that means we enable people to um move fast, but within guard rates where we can kind of say, okay, this is how we would build a dbt pipeline advice. Very different from probably what an LLM would do if you just let it try because what what dbt recommends maybe what it was trained on on the internet is very different from how we use specific tooling um so that's where we have quite a lot of active users already weekly like this was probably our best product market fit story ever like the usage just completely exploded immediately um And we do see, I would say still more anecdotally, like really clear productivity wins from this. But to be honest, this is one of the areas where we're not yet that good at like really, here's the quantified exact like ROI calculation on here's how much money we spend on tokens versus what that game does actually and work for our customers. There's still a bit early days on that. Like, so we're still more in the, well, put it out there.
00:34:15
Speaker
People really seem to love it, but we're not yet in the quantification stage there yet. But I think that's why we have the clearest user user story and kind of impact story. The kind of table finder thing that I mentioned is dashboard finder as well, i has a similar story, very, very big update also outside of the data producer community. This is more aimed at people who actually want to make some decision based on data and they need almost like like a switchboard. Like, hey, what data should i actually be looking at? This consistently over the like since I started this even before that consistently biggest
00:34:47
Speaker
weakness we have when we look at this feedback we get from these surveys and talking to people. We we actually do a customer interview every Friday. We take a random BI tool user in the company and we sit them down and they just talk us through what their problems are with data.
00:35:01
Speaker
Problem that comes up again and again is like, I can't like, If I type in lifetime value, I get like 5,000 assets and I don't know which one actually is usable or is is like curated or anything like that. So that's why we see a lot of value as well. So it's like, okay, we can now use another one to point you directly to the data that you should be working in.
00:35:20
Speaker
I personally, you this is just a personal view, not validated yet, don't think that conversational analytics are that useful in the in the classic data retrieval use case. say i don't I think it's actually less comfortable than using a, let's say, exploring or something like that to, like, if I just want some charts, I just want to it explore some data, things actually more cumbersome to type like, hello, can I please have a bar chart with this kind of like like it's actually faster to just retrieve data with this other interface. So I'm actually more personally more excited about this kind of direct switchboard directing to the right data kind of use case.
00:35:54
Speaker
um But we'll, it remains to be seen what what the what the numbers say in the end of about this. um So that's kind of the the space where we've been using olms pretty extensively. So on the producer side, pretty deep into it already. Our organization, other teams in our organization have done an amazing job really early on, give a really strong platform to host all of this stuff, make it work and make it work in a safe way, right? Like we are a very, very regulated company. We hold a lot of very sensitive financial customer information and we take this incredibly seriously. So this is another thing at scale, like
00:36:29
Speaker
You really, really need to be incredibly careful and clear with What data do you expose to an agent? What data do you definitely not expose to them? How do you make this usable for users as well? Like right now, the LLMs are very, very good at this data producer use case where the people who use it would know how to do the job anyway, right? Like they would know how to write to these DBT models. they They can kind of look at the SQL queries that get generated and they can say, oh, this is complete nonsense or this actually makes sense. So we have this validation step in there.
00:37:02
Speaker
And for that, it's kind of magic. Like it's so much faster if I know what the outcome should look like. What we're really not good at yet is the use cases where it's like, well, this is someone who has no idea what SQL is, for example, or not really. And and they still have a very valid use case. At the end of the day, they want to make a decision. It's a reasonable question to go to the business to be like, hey, where's the data on this? Show me that.
00:37:24
Speaker
um But we're still very careful with this at the moment because you end up with obviously very very easily end up with hallucinations or completely like confidently wrong, confidently present that presented wrong information.
00:37:36
Speaker
And we're still very, very careful about that. Similar to what the rest of the industry has been doing, everyone's very hot on the semantic layer
Conversational Data Tools and Security in AI
00:37:43
Speaker
here as well. Like we're very, very strongly investing in this into this at the moment. Early results look pretty promising.
00:37:49
Speaker
and but that's probably the next step for us it's like how do we really get mass adoption for this use case of my my dream already four years ago before ai started like i always thought like oh can we get there it's like you have one user anyone in the business you have one interface you describe the natural language like here's my problem you kind of like a good a good analyst would coach you into asking the right question right they wouldn't just show you data they would be like no are youre asking the wrong like what are you trying to do actually um Basically have a little bit of coaching and then you go into, okay, here's the data asset that gives you that. That's that's all I want.
00:38:21
Speaker
But so many things need to come together for that, right? Like we've worked on it for four years. Now some of it is coming together a bit, but still not fully there. Like you need to have the data there. You need to have all the metadata, semantic information, and you need to have these interfaces and you need to connect all of it.
00:38:35
Speaker
And as I mentioned earlier, in a large organization, you need to make sure that's completely compliant. That's like you're very careful on security. You're very careful on data access. um That's kind of the problem space that we're operating in at the moment. It's very, very exciting to figure out.
00:38:49
Speaker
It sounds it. um One of the things that grabbed my attention there, you mentioned obviously like this sort of switchboard type sort of interface. And it got me thinking like some of the challenges you're trying to solve. Are there other other industries, other areas where they've solved these types of problems, just not in ah with with the data context? of The mind takes me to like the you know the bullet train and the but the the hummingbird and I suppose borrowing ideas and philosophies from from other areas. So really interesting to see you use that sort of ah analogy. Are there any other sort of use cases or where you've sort of maybe borrowed from other disciplines ah as a way of sort of solving these these new and new problems that the data is facing?
00:39:30
Speaker
yeah I think for us it's mostly from the kind of product management side. I kind of see every AE as a product manager to a degree. It's like, okay, what do you need to do? You need to sit down with your customers, internal stakeholders, figure out what the actual problems are, and then you need to build a plan around how to solve that, and you need to rally other people around that cause. So a lot of the methodology of how we work day to day, I think it's very much borrowed from good product management.
00:39:56
Speaker
I couldn't agree more. And I think, and yeah, being able to, with yeah with AI framing the problem and having the idea and the visions behind that and helping shape them visions, it's never been more important to to to to a creation of something new and pushing that that that that forward.
00:40:15
Speaker
Marich, you've been at WoW for for a fairly fairly long time and you've obviously shared some of the like the really cool and exciting and
Learning from Failures and Adapting to Needs
00:40:24
Speaker
impactful work. um think that's obviously great, but it's it's always good to hear some of the war stories. um yeah Well, some of the things that have gone wrong, i think that's obviously probably where you've done your biggest learning and most people do, right? So yeah, I hope you don't mind maybe sharing some of these some of these war wounds with the audience. But yeah, what would you say have been some of the biggest lessons you've learned from failures?
00:40:47
Speaker
Yeah, very, very happy to share that. I'm a big believer in this idea of almost publishing a CV of failures, where you say, okay, here's all the stuff that actually didn't work. I mean, here's what we hopefully learn from it. um I think one of, there are a lot of small examples where I was just completely wrong on my intuition of what people in the business actually wanted and would be useful to them. um One of them is maybe for example dbt exposures like this idea of like oh you have full end-to-end lineage like you can see all the way from the dashboard to the kind of the production service where that starts and the other way around and and in my mind is i well yeah of course I would want that if I'm not an analyst if I'm an engineer I want to see everything that happens downstream of me
00:41:28
Speaker
um I want to understand the usage of my my data assets and stuff. In reality, it never really got that much uptake. Like we spent quite a lot of time building this stuff and then you end up like, no one finds it that useful. Like you you do interviews with people, yeah, that's cool. But then you look at who's actually using it, no one. And dbtdocs kind of this documentation, layer It's been similar ah for us. I think this is more of an issue of how we present it and make it really accessible. um So I think my biggest learning has always been the earlier and earlier and earlier on validation, like never try to sit in a corner and dream of what people ought to need. It's like sit down with the actual people and you don't need to do this scientifically with like, oh, here's the sample size, i like speak to five
00:42:09
Speaker
people and you already have a really good idea of like, are you on track here or not? And do you see people really give up take here? Like, I think there's a big distance between what sounds cool to present at a conference to other data people versus what if you went to a random people person in the business, you ask them like, Hey, what's cool about data or something like that? ah There's a big gap between that, right? Like I think what people care about, like who actually need this data is all very different from what we in a data space find really cool and interesting.
00:42:36
Speaker
and So that's been a big learning for me is like talk even earlier, like never assume that you are the average user who represents everyone. um I think the other bit that I've really learned over time and I'm still learning to be honest, is to be more pragmatic and less dogmatic. um I am sorry.
00:43:00
Speaker
We um I went in often very hard line on things, right? I went like, OK, every model here now needs to be incremental because that's our cost problem needs to be fixed. That way you're like, we need to have this level of data quality everywhere at a minimum.
00:43:13
Speaker
And over time, I realized, no, no, that's actually not solving the problem. That's just me being like my my brain, one structure. at To solve the actual problem, you need to look at the downstream use cases and you need to match what's happening upstream of that to that. Like, oh, this is super important financial information. Yeah, we better have all the things that I mentioned. um Oh, this is something that will be used for one week for an experiment. Cool. Let people, you know, we can easily reverse the decision. Fine. Let people be a bit messy and quick.
00:43:39
Speaker
um So I'd rather now build systems that allow flexibility, but I just very, very clearly labeling what's happening like, oh, this is messy data. Cool. Very useful for a couple of people.
00:43:49
Speaker
If you just joined the company, you shouldn't see that. You shouldn't be able to like, that should be in your face. What should be in your face is the curated data. But I think I see this in the data space a lot. There's like dogmatism around. I know everything needs have data quality. Everything needs to follow these very script strict LinkedIn rules and stuff like that.
00:44:06
Speaker
um something that I i started in that in that camp, but I think over time I've learned more and more. most like You can really go overboard with this basically. Yeah, I think that links to, I suppose, the push by the vendors, right? Back in the day, big data, collect all the data, store it all in our warehouse, costs are cheap, you don' never know when it might be valuable. And actually, you you just get these swamps, it becomes yeah analysis paralysis, not knowing what to to look at. So I think that pragmatic approach really makes sense. I think from what you described, if I could contextualize it in one word would would probably be empathy. Empathy towards what yeah what what is the actual use case and and the stakeholders that are using it and putting yourself in in their shoes rather than looking at it through through through your data lens is is probably the the important bit there.
00:44:59
Speaker
That brings us on to, I suppose, yeah um probably, I suppose, one thing that would be good to touch on would be You've been on this incredible journey of WISE, you're you're leading the and scaling this AE
Evolving Roles and Skills in Data
00:45:13
Speaker
team. um war Looking back, um would you if you could change one thing what about this journey and and what you've you've you've been doing, what what what would it be and what do you think the impact of that would would would have?
00:45:27
Speaker
Yeah, i think um I would... move a bit quicker on um some of the pragmatism. So, for example, we were very, as I mentioned, a very centralized team for a long time and probably a bit longer than was actually useful. I think we we we hit this point eventually where, yeah, the incremental central improvements you can make to the platform are still cool, they're valuable, but how much value they add is nothing comparison to year one where you go from nothing to like there's something at least. um
00:45:59
Speaker
And then more and more you realize, hey, all the actual problems are in the domain, in the context. And then from over that, we kind of spun out more into, okay, we still have this come together centrally, as I mentioned, but generally we work in our domain context. um maybe that will go back like maybe one day we realize oh we actually have really really big investments to make on the central side and then we'll kind of shape should we shape the team a bit more i uh would have moved faster on that for sure like i think that's probably the one thing that i that would have had the biggest impact to change there are a lot of things that would change retroactively um but that's probably the one thing that would have saved a lot of unnecessary work and really gotten our stakeholders value much much much more quickly
00:46:42
Speaker
Excellent. Thank you so much for sharing that. And I suppose it'd be great to, I know you're hiring, you're still trying to try trying to scale the team. For for anyone listening, like what what is the right type of profile for you? yeah what who Who are the types of people and hires that you see really succeed? What what attributes and skills ah today do they have Yeah, I think i am I can start with a bit of a personal hot take on that one. Like I think one meme that I've seen in the community that in my opinion needs to die is that we treat people like video game characters that have like a certain pool of points to distribute into different strengths and weaknesses. And then okay, either you're good with tech stuff or you're good at talking to people or something like in a very um overly basic way to to look at that. And I don't think that's true. I don't think that's true. Yes, of course, we all have different strengths and weaknesses, definitely included in different areas that we focus on more, give us more intrinsic pleasure.
00:47:38
Speaker
But what I've seen, it's very difficult often to find profiles that are more well-rounded and kind of these unicorns like, oh, you're actually good at really being motivated by the end impact, but you also can technically deliver and you don't get blocked by actually being able to execute on projects is rare but that is who we're looking for right like at the end of the day it's what you need to have the understanding that we're not doing this job in in isolation like for the sake of it like we're trying to achieve an end goal here but you also need to really have the the skills at the end the day to deliver on it
00:48:08
Speaker
um And that combination is really who we're looking for. It's like people who get really motivated by the level of autonomy we give them, the level of empowerment. It's like, hey, instead of me telling you here are the five tickets, do those tickets and promotion, it's like, hey, here's a big problem, I think. Maybe it's a problem. Please go and look at that. And then you go and you're like, oh, wow, yeah, this is a problem. Or maybe not. and Hey, here's how we solve that problem. And let's build a team or let's get people in to do this, this and that deploy this code and then measure if it actually worked. Like if you're really motivated by that kind of cycle and flexing your tech skills and actually deploying stuff at scale again, that that is really cool. um I think that's a really good fit for our team.
00:48:51
Speaker
ah Excellent. If anyone's listening, um I'm sure feel free to to hit up Moritz or check out the um the careers page um as well. um So look, before we wrap up, Moritz, I suppose i um would like to get your perspective on looking forward. um I'm not going to hold you accountable to anything that we say here because the world's moving pretty quickly. um But yeah, i what where do you see the role of data and I suppose specifically analytics engineering heading over the the next few years? um Yeah, it'd be good to get your your hot takes.
00:49:27
Speaker
who Yeah, this is getting more and more difficult. to so Even more new predictions, it's pretty difficult. um I'm definitely born into this idea that AE will become more and more the librarian role and maybe less and less the plumber role, like less and less like, oh, I sit down and build specific models like super code all day and more and more towards almost this like philosophically concept of like what is knowledge like in in a business? Like what what does this represent and who could use this data? And like, can we between us agree what these numbers or whatever data type they are mean in the real world for our customers? Like, I think
00:50:04
Speaker
That is really exciting to me as a role. like I think that's going to be so at the core of decision making for how do businesses is run, right? Like how how do we drive value for our customers? It's peak going to be, i think, less and less like, oh, I sit down for hours and hours on an end and try to troubleshoot one annoying sequel bug.
00:50:21
Speaker
um And it's going to be more and more just like real tough decision of like, hey, there's all this data. What does it mean for our customers at the end of the day? Very interesting. i think one of the things I've been asking a lot of people as well recently is what what' what's your most controversial view on the on the data industry?
00:50:42
Speaker
Yeah, um I don't know how controversial this is, but I would say my hot take completely unsubstantiated is is that probably 90% of dashboards and data assets are useless and vanity.
00:50:53
Speaker
um i I had really, really good advice very early in my career. Kai, one of my former colleagues, like um the first dashboard ever built kind of destroyed us. Like, why do we have this tile? What what kind of decision do we make? What do we have this? What decision do we make? Every single, like, 15 charts on earth on the dashboard. I really learned a lot from that. And to this day, before I build anything, before hopefully anyone and on our team builds anything, we should kind of almost like they prototype, completely mock fake data and say, hey, let's say, okay, let's say we build this and this number goes up by that. What do you do?
00:51:26
Speaker
And in my experience, in most, most, most cases, the answer is nothing really. You probably hear the word interesting a lot. That's a very dangerous word. It would be very interesting to see that data. that usually that means happen Usually that means you're just wasting your time here.
00:51:42
Speaker
And so I think generally people in data realize this, but i think it's even worse than than the the general generally health opinion is. I think the vast, vast, vast majority of data projects I see are like vanity
Conclusion and Reflections
00:51:55
Speaker
internet. They're not not actually driving any impact.
00:52:00
Speaker
I think that's a very great place to and to leave. I think it's a really, really relevant point. And um yeah, I suppose is is AI analytics the the the silver bullet to to to that point?
00:52:15
Speaker
Probably not. I'm not looking forward to my inbox on the LinkedIn inbox on that. Well, look, Marit, thank you so much for such ah such an honest and candid conversation about what's going on at at WISE and and the role that you've played and and your team. So, yeah, it's been a pleasure to have you on the show. um Yeah, long time coming. Glad we got around to doing it.
00:52:39
Speaker
Yeah, thank you very much. It's been very, very interesting. I'm very happy to have the opportunity to think and talk about this stuff. As always, I ran into it bit more than necessary, but I'm very happy to be on the show and thank you No, no worries. Well, that's it for this week, folks. We'll see you in a couple of weeks.
00:52:55
Speaker
Thank you and goodbye. Thank you.
00:52:59
Speaker
if you've enjoyed today's episode i'd be so grateful if you could hit that follow button leave us a rating even better past the show on to a friend who might also get some insight from it it really helps us grow the community and continue to share amazing conversations I also wanted to take a minute to talk to you about Cognify.
00:53:17
Speaker
Those of you that don't know, Cognify is the leading recruitment partner for modern data teams. We help some of the world's best organizations scale data and drive real value from the hires that they make.
00:53:31
Speaker
you're thinking about building a team or making a hire and you're struggling with talent or just want some insights on the market, then I'd love to jump on a call with you and tell you a bit more. Equally, if you're looking for a job and want to find your next dream role, then reach out to myself or any of other Cognify team.
00:53:48
Speaker
We'd be happy to see if there's anything on our books that we can help you with and give you general advice on the industry. Finally, a big thank you to Omni, this season's sponsor. If you'd like to learn more about the AI analytics that Omni can deliver you, then check out the link in the show notes or come speak to me. i can happily point you in the right direction.
00:54:07
Speaker
Again, thanks for listening and look forward to seeing you a few weeks time.