Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
043 - Building Aurora: How Moneybox shipped an AI financial assistant inside a regulated fintech
1 Playsin 16 hours
Most AI projects in financial services stay in a pilot. Moneybox actually shipped one.
In this episode, we sit down with Marko Katavic, Director of AI and Decision Intelligence at Moneybox, to get the real story behind Aurora — the in-house AI guidance engine Moneybox built to help their 1.5 million customers better understand and act on their finances.
We go deep on what it actually takes to build an AI product inside a regulated environment: the architectural decisions, the trade-offs they made, the things that didn't work, and what the team is focused on next.
If you work in data, AI, or fintech — or you're trying to ship an AI product in a high-stakes environment — this is the episode for you.
We cover:
- What Aurora is, what problem it solves, and why Moneybox built it in-house
- The technical architecture behind a production AI product in a regulated context
- How they approached FCA compliance, safety, and human oversight
- The trade-offs they made — what they prioritised and what they deferred
- What good looks like when you're measuring an AI financial assistant
- What's next as the product and team continue to evolve
Recommended
Transcript
AI Hype vs. Skepticism Debate
00:00:00
Speaker
the problems of the current AI hype versus skepticism debate. I'm quite sort of miffed by the fact that people either either put themselves in the hype camp or in the skepticism camp and the nuance is kind of disappearing from the discussion. I think it's because people either see LLMs as things that can do everything or they see them as a thing that can't do the one thing that they wanted to do so they can't do anything. And so the nuance of where real product market fit for LLMs happens is kind of lost. What we believe is that one big benefit of LLMs is synthetic data. What we are seeing, especially in some of the more creative parts of the business, a lot of value that could happen. And so we're encouraging leaders in those parts of the business to work with me and my team and to work with ah sort of LLMs as consultants to figure out um ways by which we can bring more productivity in the organization.
00:00:47
Speaker
And we'll looking at that as one of the streams this this year. Today's episode is brought to you by Omni.
Omni's Role in AI Analytics
00:00:54
Speaker
Most companies I speak to want AI analytics but are failing to put projects into production. That's where Omni is different. It's the semantic brain that grounds AI in the heart of your business logic, giving you governed answers, whilst also the depth to identify root causes. It's intelligence everywhere, from your spreadsheets to their chat feature, even within your product.
00:01:13
Speaker
Omni moves you beyond the dashboard. Don't just take my word for it. Trust teams like Perplexity and Synthesia that are already using Omni to deliver intelligence that people trust. Check out them in the show notes or visit omni.co. That's O-M-N-I co.
00:01:29
Speaker
Now, back to the show. Hello everyone, welcome to another episode of the Stacked Data Podcast. Today is a really special one. We're diving into what it actually takes to build AI that people can trust at scale in one of the most regulated environments out there, financial services.
Marco's Transition to Fintech
00:01:50
Speaker
Today i'm joined by Marco who leads data science and AI at Moneybox. And in the episode, we're going to go deep and unpack his journey from big tech into fintech and then dive deep into Aurora, which is Moneybox's soon-to-launch AI platform chatbot.
00:02:09
Speaker
We'll talk about how the he ah you build financial AI when you don't have historical training data, how synthetic data and human annotation fits into the loop, and what the actual architecture really looks like under the hood, and the challenges that Marco and the team have have really experienced on on the journey.
Challenges in Building Financial AI
00:02:28
Speaker
So if you're a data scientist, ah ML engineer, analytics leader, or anyone thinking about deploying AI, then hopefully this episode is full of some good nuggets for for yourself. um Marco, welcome to the show. It's it's great to have you on. How are doing?
00:02:44
Speaker
Thank you, Harry. yeah' um I'm really well and really excited about talking about all the stuff that you just listed. Excellent. I think it's um yeah it's been a long time um building for you and your team. so um yeah I obviously set the scene a little bit, but it'd be good for the audience to to hear a bit more about yourself, Marco. um You've had a really varied career and done really different routes um into to where you are now. so Could you you help walk the audience through um yeah yourself?
00:03:14
Speaker
Yeah, absolutely. I guess where I like to start is even though I'm um'm leading AI teams these days and and machine learning and AI most of my life, I am a sociologist by education.
00:03:26
Speaker
um I like to lead with that even though i've got a degree in both sociology and information science. And post-uni, as you can imagine, as a sociologist, you don't really get to work as a sociologist, right? So I leaned into my other skills. which is coding. um And I started a startup where I worked as an iOS developer. We were building games for iOS at the heyday of iOS as a new platform.
00:03:49
Speaker
And we had some success with that, but that ultimately didn't pan out into ah into a career. And as the stars would align, um at that time back in Croatia where I'm from, um there was this startup building an analytical SaaS platform for um internet analytics for the audience measurement and marketing industries.
00:04:10
Speaker
And I kind of slotted very neatly into a product role there. So I started working as a product manager, helped build that product up. But you know how it is with startups. Everything is resource constrained and so you're you're not going to hire unless you really need to hire and everybody does whatever they can.
00:04:27
Speaker
And so because I had a statistical background and I like maths and I was in that kind of strategic role as a product manager, I dabbled in a lot of data science building the models and the probabilistic solutions that were needed for the for the product.
00:04:40
Speaker
That kind of got me interested, got me hooked. And that's how i transitioned into data science. Ultimately, that startup got bought up by a very large market research company that brought me to London and into building ah the data science department within that company here in London.
00:04:55
Speaker
um As that data science department grew, so did my ad appetites. And that's how I kind of transitioned more into data science leadership, which then got me hopping around various tech companies around London. I was in sports streaming and then I was in Meta and Instagram.
00:05:09
Speaker
um And then i went into FinTech, which is a very different flavor in consumer tech.
Meta vs. Moneybox Experience
00:05:15
Speaker
um We're going to talk a little bit about that, but it is it is it is different. Excellent. Yeah, I mean, um such a varied career and I think that's obviously made you clearly the leader you are now. um i would love to touch, I suppose, upon your time at Meta and I think you were specifically obviously at Instagram.
00:05:36
Speaker
During the peak of, I suppose, competition between TikTok in the relevant sort of problem space, it'd be great to just sort of hear about how you what what what that experience at actually such huge scale and personalization sort taught you.
00:05:52
Speaker
Yeah, um so TikTok brought to the scene as as as an app that is really, you know, people were wowed by the algorithm as they started using TikTok, right? But the same class of algorithms really did exist in meta before. It's just that it was employed much more heavily in the ad space rather than in the content space.
00:06:10
Speaker
And so as this fight was kind of ramping up, there was a big sort of push, how can you get the the all the learnings from the ads algorithm space applied into the feed space so that the feed can can become a little bit bit bit more discoverable and and things like that.
00:06:27
Speaker
But the first thing that hit me as I came into Meta up until Meta, Even though i did work at at reasonably scaled up companies, you know, 20 million users, whatnot, you don't really appreciate what it means to ah to have billions of users and millions of content.
00:06:46
Speaker
um Obviously, you can't at that scale, you can't infer the optimal, the best choice for each user, right? You have to layer your your relevance problem a little bit more. And, you know, looking at how um a lot of that is public about how meta layers the relevance problem, the recommendation problem and how they have multi-stage retrieval of candidates and then candidates into into a shorter list of candidates and how you increase model complexity as you get closer to the bone.
00:07:13
Speaker
that was That was very interesting and a lot a lot of those learnings actually are now applying in Moneybox as well, even though we're not at that scale. Because, you know, at Meta it's not just a question of how do you bind signal the best in order to deliver the best thing. It's that against the trade-off of cost and and inference latency.
00:07:31
Speaker
which for them is critical. You could get around that in a smaller company by just throwing money at it, right, which they can't really. But then that diminishes a lot of your ROI use cases. So, you know, I was in media before Instagram and there you can build recommender algorithms that are not necessarily as layered as Instagrams.
00:07:50
Speaker
but you you have a little bit more cost, you will still get the ah ROI because it's a media business. In finance, which is not a media business, you have to be ah very careful with your ROI, right? And so a lot of those learnings in how to be cost efficient from Meta, um I think we're applying them now as well in FinTech, in what we're doing right now.
00:08:09
Speaker
Fascinating. I and and really excited to unpack some of that. I suppose, what have you found as the big sort of difference from moving from big tech into to FinTech at Moneybox?
Transforming Data Science at Moneybox
00:08:23
Speaker
Yes. um
00:08:27
Speaker
the The first thing you miss when you leave a big tech ecosystem is the tools. um You know, Meta has built every tool internally and every tool is tailor-made for the use case that it needs to be, right? Their analytical platform, their dashboarding, experimentation platform, it's all heavily heavily hacked towards being what it needs to be for the the what it means to build in consumer tech.
00:08:52
Speaker
And that kind of the tool customizations you get there, you don't get it with off-the-shelf tools that are trying to solve multiple use cases, right? And so you really miss the tools. That's like the first thing you miss.
00:09:04
Speaker
um I think what has mapped very neatly is the the energy, the pace. um I was in one of the more pacer parts ah of of Meta and pace your Instagram is a pacer part and then within the area that I was using more pacer.
00:09:18
Speaker
I found that same pace operating within a fintech, within a scale of fintech and the energy was very, very equivalent. Now, what is very different is if you consider Meta's consumer products, right?
00:09:31
Speaker
Yes, there's a lot of complexity in them, they're obviously not easy to build and there's a lot of trade-offs that they consider, but... You know... in know not like When you go to the to the to the meat of it, it's a relatively simple product for a consumer to understand and it's a relatively simple product proposition. There's content, you consume content. right right um The algorithm is complex.
00:09:51
Speaker
How that algorithm finds you is complex. There's a lot of things that that need solving there, but it's there's not a lot of the sausage hidden um ah for the user except for the algorithm. right um In fintech, what might seem very simple to a user of the app is actually extremely complex on on the other side. And I wasn't even aware of that. This is my first foray into into finance. I wasn't aware of how the sausage is made.
00:10:16
Speaker
And in the background, there's a lot more happening. And making that complexity that happens in the background hidden from the user while still working and being flawless is quite an engineering challenge. So that was that was very interesting.
00:10:32
Speaker
A fascinating insight. Meta is, I suppose, taking a simple ah product and making it incredibly complex jobs, how they they generate revenue. and Then I suppose your fintechs way around gives off the aura of being simple, um ah but it's incredibly complex behind the scenes. la That's really interesting. um I suppose when you landed in Moneybox, what what was the state of data science? You obviously joined their data science ML and AI team. like What was the state at play?
00:11:06
Speaker
um There was one data scientist there, pretty good data scientist, but they were sort of haphazardly jumping from project to project and they were kind of trying to find ah roi I was hired to both lead the data science team, but also to set up a personalization strategy and find what is missing and within Moneybox to to really scale out a proper personalization and AI solution.
00:11:29
Speaker
and What was most challenging is I came in and I immediately wanted to change everything. right The app wasn't flexible enough, so we couldn't employ machine learning in enough places.
00:11:41
Speaker
um But obviously, that's not how you do organizational change. right So the the most challenging bit probably was rather than go after the obvious ROI, was try to find what is the what is the most natural way you can embed the thinking in Moneybox so that you get kind of compounding returns and start building incrementally.
00:12:01
Speaker
And i was very lucky that I found very good partners in in the engineering leaders in Moneybox um who who saw, you know, we jointly created what the vision could be and then um in executing on it, that partnership was was very valuable.
00:12:16
Speaker
um And then slowly we moved from haphazard projects that are plugged in here and there towards how do we get data gathering, data processing, signal clarity as the core part of the um of the product proposition. And then how do we slowly embed ML as not something that is transactional in some part of the app, but that is slowly permeating multiple parts of the app.
00:12:40
Speaker
That was journey. Amazing. and lot look We're going to unpack, I suppose, now um Aurora and and everything that that that you've been building. but um We've been talking about you working at Moneybox. It'd be good to maybe to take a minute and just yeah who who are Moneybox? i mean I'm i'm ah um' a customer of Moneybox. I used them for my help to buy ISAF when I bought my my first house. but yeah For the audience, I'd be good. yeah who Who is Moneybox as ah as a fintech?
00:13:09
Speaker
Right, we're ah ah we're a digital wealth provider, so we're an app where you can go on get a savings account or you can invest your money, you can get a pension, you can get good financial products that help you with buying a home or something like that. But really what we're about is we're on a mission to bring um financial freedom and financial opportunity at scale to everyone in Britain and bring sort of peace of mind to everyone who was always sort of jarringly running away from dealing with finances, right?
00:13:36
Speaker
um So we pride ourselves on being very easy to use, we pride ourselves on having excellent customer service and we pride ourselves in being there for the customer. We believe that wealth should be enabled for everyone. Wealth is not something that's only for the super rich, it's it's a state of mind and it's um there is a way to building wealth that we can bring at scale to everyone in Britain.
00:13:57
Speaker
So, great to hear. I know a few of the Moneybox team, a lot of them come to a lot of our our meetups at Cognify. And I think one thing that shines through, um everyone at Moneybox seems so bought in and mission-driven and really loves where they are. where they work and what they're trying to ah ah achieve. so um yeah I'm keen to now understand, suppose, how does AI fit into the picture,
Aurora's Mission and AI Ethics
00:14:21
Speaker
Marco? um I think you're soon to be launching, i don't know, been maybe already launched by the time this episode releases, but Moneybox Aurora. so um
00:14:31
Speaker
yeah What is it? What problem is is Moneybox trying to solve with this financial AI that you're building? Great question. um So if our mission is to bring peace of mind and freedom and opportunities to everyone at scale, then Aurora is basically the mechanism by which that will happen.
00:14:48
Speaker
And the way I think about it is there's a couple of problems we're trying to solve, right? Number one, we're not being, all of us, we're not being taught enough about finance as we're growing up through our kind of foundational education, right? Which makes it very hard to engage with finance, right?
00:15:03
Speaker
um millions are then left without confidence and without the knowledge to employ the money in the way that can best improve their circumstances, right? um If you are to employ your money that way, that unlocks freedom and opportunity. Freedom and opportunity unlocks you to live your but better life and to employ your skills really in the right manner as well, right? So it leads to happiness and all that. And that's the core thing we're trying to do.
00:15:27
Speaker
to do it, in a safe and and sort of relatable way, we believe that supporting people with those decisions can only be done with with AI. Right. So that's the the the first thing.
00:15:38
Speaker
The second thing is there's a lot of talk in the financial industry in the UK about the advice gap. Right. So for those of you who who might not be as glued into the financial services industry in the UK, the advice gap refers to the fact that um Because of some things that were happening in the early 2000s, the financial regulator in the UK is regulating heavily who can give financial advice. And that's a good thing because that means that consumers are protected, they're not getting bad advice.
00:16:07
Speaker
But it also means that the barrier to entry is relatively high. And so, you know, the costs are a little bit prohibitive and not everybody can go and get financial advice. You typically need a lot of money to get financial advice.
00:16:19
Speaker
Now we see that as a bit of a problem because the barrier to entry is too high. So we're trying to bring that barrier all the way down. We believe that everybody could benefit from some form of financial guidance or financial advice and that if we can employ i in the right manner, we can bring that at scale to customers in a way that they can trust it.
00:16:37
Speaker
Now, the third thing is If we have those two problems, right, that there's not enough confidence, there's not enough knowledge about how to make decisions, and it's too cost prohibitive to go and get help for those decisions, what is happening right now is that a lot of people are turning to generic AI tools and sort of generative AI tools to get some financial confidence. By a recent study, 60% of the UK public has used um a ChatGPT or a Gemini or a Claude to get some form of financial guidance, right?
00:17:06
Speaker
And we understand why that happens. In a vacuum, people are going to go after whatever they can go go after. Before um generative AI came about, people were going after advice that was given to them by financial influencers or they were Googling heavily, right?
00:17:20
Speaker
um But the challenge with generative AI is that it looks very plausible, right? However, we know that generative AI by the nature of how it is built is not foolproof, right? It's not ultimately safe, which is why they disclaim themselves at the beginning. So you can't, you can neither get the full utility from it, nor can you trust it completely to give you the right set of recommendations, right?
00:17:43
Speaker
So we kind of believe that financial services almost has a duty to build AI that you can trust and that it is reliable, right? So that users don't have to go to tools that were ultimately not built for that, that were built for more generic sort of knowledge exploration um and get advice ah that is that is good.
00:18:00
Speaker
So that's when you bring those three together, that is the vision behind what Aurora can be. Amazing. I've definitely used AI the for the being them exact use cases, so yeah personally can see see the benefit. um It sounds like um a big challenge ahead, especially in a ah regulated environment. um One of the things that we spoke about in preparing for the show is, I suppose, you guys had no historical data for training the your your models and on financial advice. so yeah like How did you bootstrap that from from zero and yeah if you could sort of hear that hear that story?
00:18:41
Speaker
yeah um I would argue we're not unique in that position. right so Even though we don't have historic advice data, even the companies that have been giving advice for a long time haven't necessarily gathered consent to use that data for AI training. so So it's a very, that's why the industry is a bit harder to crack is because you don't have great data to calibrate on.
00:19:00
Speaker
um And if you'll allow me a little detour, I think our kind of example shows, shines a great light on the problems of the current hype versus skepticism debate, right?
00:19:13
Speaker
um So i'm I'm quite sort of miffed by the fact that people either either put themselves in the hype camp or in the skepticism camp and the nuance is kind of disappearing from the discussion, right?
00:19:26
Speaker
and I think it's because people either see LLMs as things that can do everything or they see them as things that can't do the one thing that they wanted to do so they can't do anything, right? And so the nuance of where real product market fit for LLMs happens is kind of lost.
00:19:43
Speaker
And what we believe is that one big benefit of LLMs is synthetic data. right So what we found is, in a historic context, having human annotators create big conversation trails um that you can then train on, or big sets of circumstances from a certain distribution that you can train on, that all requires a lot of sort of data science efforts to create the right distribution synthesizers, and it creates a lot of human effort, a lot of labelers to create text, copywriters. So you have a machine of people who are needed to create data at a volume that you would need and then you have to get it right because it's super expensive to change that data at all. right
00:20:22
Speaker
Now what we're finding with LLMs is synthesizing a lot of data is super easy. Changing the distribution within the data that you've synthesized is relatively easy. What is hard is making sure that the data is valid.
00:20:34
Speaker
But that job for annotators is a much easier job than having to create the data from scratch. right And so you can create a sort of a training data loop relatively smoothly with LLMs that can give you a pretty solid um sort of data bank to train on. I know there's many companies that are actually building this into their proposition. So there's many companies that specialize in providing data for training AI models.
00:20:58
Speaker
And they're increasingly looking at how can they not only go to um sort of recruited human participants who can create data, but also how can they synthesize and shorten the sort of data creation loop.
00:21:09
Speaker
So that's something where we found very good fit for LLMs and where we're finding some success. I agree with what you said, obviously. i I definitely see the same people either in the hype camp or or in the skepticism and it's it's yeah with AI's, it's all a bubble.
00:21:26
Speaker
I think one of the biggest things that I i see yeah LLMs for is yeah know they are tools. um You can use tools in the right way, tools that have a specific purpose and in some some areas and they're stronger at that purpose than than others. and i think um ah When it first came out, a lot of people saw LLMs as ah you know ah magic and I think breaking it down that yeah know that they are just tooling and whilst I'm probably not smart enough to understand everything that goes on behind the scenes, they they are just a tool. so I suppose finding getting getting the most out of it is right. I suppose that's why you're building a tool that's specific, that you're building the hammer to hit the nail in that that financial um and education space.
00:22:07
Speaker
Obviously this training in the loop, um is this something you guys done in-house or did you work with external partners? i suppose How did you yeah find that balance of of synthesizing data, adding a human human annotation? and yeah was that Was that something that you've you've done completely by yourself? So did you lean on partners?
00:22:30
Speaker
um We do lean on some partners. you know Hiring human resources, especially in a spiky task such as labeling, is is very difficult. right so like Especially for sourcing the annotators, the labelers, we do work with partners.
00:22:44
Speaker
um There are some specialist software as well that we're looking at. but Really, it was a it was a real sort of bootstrap yourself, on figure it all out kind of process.
00:22:55
Speaker
um We built a lot of our labeling tooling initially, um essentially by vibe coding it, right? Because it's an internal tool. It's an internal tool that we have full control over.
00:23:05
Speaker
We're quite happy to vibe code it because it doesn't have to be super efficient. So we managed to boot bootstrap ourselves quite quickly to a position where we can start generating something something pretty cool. um On synthesizing data, we weren't using externals, so we had a a machine learning researcher in our team was was looking specifically at how we can synthesize this data for a little bit.
00:23:25
Speaker
And we figure out how we can get to the initial trunks we need to to sort of kick off the training process. um What we found was ah When we went into it, I did think were going to have some success. I didn't believe that we were going to have as much success as we did have, right?
00:23:40
Speaker
um Primarily because I was kind of underestimating how easily you can you can course correct, right? You create a lot of data. And then if the data is not good, you just course correct into slightly different data and it's fine. right And you don't need to use super powerful models for it, right because even lower parameter space LLMs can create believable sounding human conversations, right um filled with enough sort of salience.
00:24:07
Speaker
You don't need all the reasoning capability. You don't need all the sort of um deep thinking capability. So you can go in much lower parameter space, which really reduces your cost. So that's the fact that you can correct and the fact that you can have sort of agents evaluate the data meant that the loop was much easier to form than you initially anticipated.
00:24:25
Speaker
right um And that's pretty good. Now, what you can't do with this, obviously, is you can't create the kind of data you need to train a foundational model. So this is good for distilling. It's good for fine tuning models.
00:24:37
Speaker
It's good for if you need to train, God forbid. pretty deep learning models, right? And we do have a place for ah sort of classical machine learning models in in our stack. um So for all of that, it's pretty good, but obviously you wouldn't be able to do this to bootstrap yourself to a new foundational model.
00:24:53
Speaker
like That makes a lot of sense. I suppose, look, we'd love to go into the um the technical side of Aurora's architecture um and what that actually looks
Aurora's Architecture and Security
00:25:02
Speaker
like. I'm sure you can't share everything, Marco, but, um yeah, I suppose, how are you hosting your models? What does the your GPU and sort of infrastructure and environment look like? And, um yeah, how how do your models actually hit the hit the app?
00:25:17
Speaker
Yeah, um before I get into the infra, which I'm pretty sure part of the audience would be very interested in, I do want to talk a little bit about um about the sort of architecture of it. um In a financial setting, um mean, there are some people who are trying to crack this problem, the same problem that we're trying to crack using purely LLMs.
00:25:37
Speaker
we we don't have a We have done some tests, we don't have a strong belief that that is the right way to go. We believe LLMs within their internal states struggle to follow process, they struggle to remain within logical boundaries. So we're looking at neuro-symbolic approaches where we are solving some things with classical and ML, we're sort sorting sorting some things with software and with like deterministic human-written algorithms, and then there's a place for LLMs in the mix. right So because we have that and the LLMs that are in the mix are typically not the state of the art LLMs, right? Because they don't have to be.
00:26:13
Speaker
Because of that, on the infrastructure side, we need different infrastructures to serve different things, right? Crucially, we do not need the models that are of such size that we would need to go to the closed APIs of the AI providers.
00:26:29
Speaker
um This is important for us because we strongly believe that part of that trust that people need with a financial services institution that that gives you a service, including an AI service, is that the data remains with a financial institution.
00:26:41
Speaker
We know from our research that people are quite nervous in companies sending sensitive financial information over the ocean to a company somewhere that is an AI company and then God knows what happens with it. right Now, all those companies are adhering to data privacy practices to one degree or another, but um people are still nervous. right So we felt very strongly that we need to be able to self-host all the stuff.
00:27:06
Speaker
That's why we're single cloud tenant in Moneybox. We deploy everything to one one sort of private cloud that we have. And in that one, we deploy all our models on Kubernetes, um pretty standard deployment pattern. We have different clusters for different model classes.
00:27:22
Speaker
um We expose all of that as a service to the rest of the app. So the yes two the app can can hit those models that way. And everything, all of the interfaces and everything we have in the app are also built internally. So connecting to those services wasn't wasn't terribly difficult.
00:27:36
Speaker
um The reason why that is good is because the kind of GPUs we need to power these this and at the scale where we are doesn't make the serving problem that that difficult actually. We can fit most of the models that we need, we can fit them at most in an H100, one H100, and we can serve the scale of logins that we have, right?
00:27:57
Speaker
and so we can do everything at relatively low cost and you know benefit from the efficiency of Kubernetes. Now, if we were serving 100 million customers and if we really need a state-of-the-art performance, I think that that's where that you get the challenge that you just can't serve it using your own infrastructure because you can't achieve the sort of economies of scale and um you know when you load model weights in that you maximize weight utilization for the scale that you're at.
00:28:26
Speaker
um you know that's sort resource pooling comes very handy and that's where then you really need to go private API, I would say. But yeah, that's that's kind of how we've done it. So one cloud, Kubernetes clusters, we deploy directly into them. Research we don't do in that cloud, of research we do through through Databricks, which is the only company I'm going to plug here. We're very happy with Databricks.
00:28:47
Speaker
And so we research there, package it all up, deploy it there, and it's pretty standard pattern. great that's That's great to hear. I suppose you're saying if at the scale that you're at, then you can do this self-hosted, but you if you were to, ah what is Moneybox's customer base? Are you allowed to share that on on hit on here? Yes, I can. so i mean I think the 2025 report will not have come out by the time you hear this. and In any case, I don't want to complicate things, so I'm going to refer to the 2024 report.
00:29:19
Speaker
ah In the 2024 report, we had 1.5 million customers. um We still grew from then, right? We're UK exclusive, so we think that's ah that's a very reasonable customer base for for the UK. um but But yeah, that's that's kind of the scale we're talking about. You've got ah you got you got a little runway before you have to deal with and any of the other sort of infrastructure problems there. Yes.
00:29:44
Speaker
right Yes, definitely. um Marco, that's great to hear and in good to hear. I suppose your rationale is behind that and and why you've this up made some of those decisions. so i think that's really important. um One of the other things we discussed was um a back-end driven front-end. um Now, this is ah a term that I'd only heard a few times before. um and yeah i would love for you to, I suppose, break down what that means in in principle for the audience if they haven't heard of it because I found it fascinating.
00:30:15
Speaker
yeah so it's It's only really a problem because of our kind of deep-rooted belief that an AI that tries to do what our AI is trying to do is not going to be effective if it's a transactional box you have to access within the app, right?
00:30:29
Speaker
If you're an app and then you have to go somewhere and you access a box and then there's AI there that you talk to and then you go come out and you have the app again. For our use case, we don't believe that flies. We believe the AI needs to permeate everything and that your app should change as we know more and as we can serve you better and as we can guide you better.
00:30:46
Speaker
But if the app continuously needs to change, that means that from the back end, from the AI, we need to change how the app looks to you, right? Whilst making sure that the app is still 100 correct and good um around all of the numbers that is showing you all of your financial data, etc. Because you trust your financial um institution to never get your financial data wrong, right?
00:31:08
Speaker
um And that is harder than it sounds, right? It means that your your frontend has to be basically very non-intelligent. It needs to basically just be a renderer of whatever information is sent from the backend. And that's what backend-driven frontends are.
00:31:22
Speaker
It's when the backend is basically structuring all the information in the frontend and sending it as an informational packet. And then the frontend is just a design system that that knows how to re-render that in in various different ways.
00:31:35
Speaker
When I joined Moneybox, the front-end was very much, it was a very simple app. We we got the words for how simple the app is, but it was very much an app that is serving um a certain use case. It was built for that. It wasn't built for flexibility and modularity. And so we had to re-architect quite a bit to get it to a point where a lot of the elements now are driven from the back-end. So when we hit the stage where the AI can change parts of your interface to make it more understandable for you or um explain something in a different manner, that content can be injected in various different ways in the app
00:32:08
Speaker
Excellent. i That sounded like you know a whole re-architecture. That's definitely a big challenge. um I suppose I'd love to hear from you about what was the biggest engineering challenge that you didn't expect going into this project?
AI Integration in Moneybox's App
00:32:24
Speaker
and there There's always them unknowns from from what I understand when it comes to to data. so yeah like what What were the unknowns and I suppose yeah what would you do differently now knowing what you know?
00:32:37
Speaker
um The way you phrase that question at the end is probably ah the best way to phrase it. um I can say that I've probably lost six months to my naivety of coming from a consumer tech world and thinking, well, this is all simple. Why can't we just build the app differently? Let's just build the app differently. Completely underestimating all the rigmarole of what happens behind the scenes to money, right?
00:33:02
Speaker
So as the user of financial app, you deposit some money and it goes into an account and you believe that that's a simple That's a simple path. It's not a simple path. What happens in the background, how many different transactions that spawns, how many different financial institutions that transaction has to go through to get validated, checked, audited, logged, etc.
00:33:20
Speaker
is quite something. um I also underestimated how how clear we need to be to customers, right? um In most industries, um You don't have to be careful about how you present stuff to customers. In a financial industry, you have to be careful to make sure that you're not misled, penalties are much higher and much much quicker you can fall into the territory where you accidentally made something not understandable to customers. right So there's a lot more care as to what ends up i'm going in front of customers.
00:33:53
Speaker
And those are both elements that I underestimated heavily coming from consumer tech world. And I wanted to move at at the pace in consumer tech that I'm used to. But really what I found was that unlike under industries in finance,
00:34:07
Speaker
It's not desirable to always remove all friction from the consumer path. Sometimes you need to add friction to the consumer path because that's the test that the consumer is making the right decisions. right And a regulator will warrant that you need to have some friction. So for instance, you can't, you know, when you're opening an investment account, let's say you have to go through a couple of screens that explain to you what might happen, that your money might go down as well as well as up, that capital is at risk and all those things.
00:34:32
Speaker
um And I knew that those things need to be in, but I underestimated how how there's a lot of thought actually in the regulation as to what is the right amount of friction. So basically, when you build consumer tech, you need to build it where friction is one of the variables that you're flexing and optimizing for. right And that is not a pattern that I was used to. So that was the biggest thing that we had to think about ah around how we can build something scalable and flexible, but that can't scale that friction in a very auditable and safe way.
00:34:59
Speaker
I suppose that's a great segue into the the next section really, of which is, I suppose, understanding um how you've actually gone about building AI in an incredibly complex industry, but also really heavily regulated. and You mentioned obviously the peak penalties for getting stuff wrong. we we mentioned earlier about sort of influencers and yeah people going to their financial advice.
00:35:26
Speaker
they've also come under a lot of scrutiny and also big implications for for for misrepresenting information. so like How have you made sure that you can walk this tightrope of really adding significant impacts and benefits to your to your customers, but also making sure that um yeah you are appeasing regulators and customers can trust what you ultimately what what Aurora is going to be going to be spitting out?
00:35:52
Speaker
um Yeah, I think that I would answer that question slightly differently from ah from a customer side and from a regulator side. Now, what I will say for the regulation side, even though finance is heavily regulated and that can sound very scary, we're very lucky in the UK that our finance regulator, to the FCA, Financial Conduct Authority, is ah what I would call a very technologically switched on regulator.
00:36:13
Speaker
um And they've they've instituted this principle a couple years back called consumer duty, which basically means, you know, you should, in everything you do, you should make sure that you're doing right by the customer, right? Which sounds very obvious, every company should be doing that, but apparently it's right?
00:36:28
Speaker
So the regulator has instituted that principle and in how they're thinking about AI regulation, now that is kind of at the core of what they're thinking, right? So even though we haven't fully landed on what a set of regulations is going to be um um and how that's going to work, the kind of philosophy that's underpinned from my conversations with people in the in the regulator is that as long as you can demonstrate that you have a governable and safe process by which you are ensuring good customer outcomes, then they're basically okay with usage of AI, right?
AI Regulation and Customer Trust
00:36:58
Speaker
um Now, obviously, the devil's in the detail there, but because they're so guided by making sure that you're delivering good customer outcomes, that does give you a bit flex bit of flexibility in how you can move at pace. What they're also doing is they're enabling companies quite a bit with um the AI sandbox, which is They're sort of regulatory sandbox within which you can launch to some customers and test um with relaxed regulation in in a way, to simplify it, right? But also they create a lot of data sets that allow training and then they have teams and they're building solutions that can help you test your solutions against those data sets as well as sort of a check before it hits customers.
00:37:36
Speaker
which are all very sort of propulsive things that are going to fuel the industry moving forward, right? um So we're not that worried about the regulator and part of it is the customer side, right?
00:37:47
Speaker
So if you think about the customer side, we're building something that promises that can do finances, that can do help you build your wealth for you, right? Obviously, there's a lot of trust needed that you're going to go and like even with a human but the financial advisors, people people find it very difficult to go and trust, right?
00:38:05
Speaker
So we believe that a lot of this is underpinned by brand. Now, we've got a brand that is very trusted. It is a very um loved brand by the customers who who are using us. And we believe that because of that, we have high stakes playing in this game, right? We can very easily lose over all of that trust and that serves as a motivator for us to do something good.
00:38:23
Speaker
If we stand behind this solution once we launch it out, we stand behind it because we believe that it's good that it's been tested and that you as a user should use it. And so we're kind of riding on the trust that we have, but we have a lot to lose. And that's why we believe customers in a way are going to trust us. Right.
00:38:40
Speaker
We also don't believe that you can just have an AI system that does this. What we found both from user research and talking to our customers is they like to know that humans are in some way in the loop. Right. Now, that doesn't necessarily mean that the the model is going to be that there is an advisor somewhere that is just scaling out through technology. Right.
00:38:59
Speaker
knowing that there are some algorithms that have been checked by humans and that outputs are verifiable that are not prone to hallucination is something the users are going to want. So we believe that the fact that we're spending more to make sure that your data never goes anywhere, our brand, all of those are the pieces of the puzzle that are going to resonate with customers.
00:39:17
Speaker
And then finally, Our business model lends itself very well in terms of incentives towards doing right by the customer. we if the If the customer is having a good time and their money is growing, we have a good time and our revenue is growing. So we think that it's a good's good sort of fit from an incentive perspective there as well.
00:39:35
Speaker
Amazing. I think the thing that stood out for me there was that your customer-centric nature. and I think that can be, yeah that's kudos obviously to Moneybox and the brand, but like to you guys as a as a data to team and an AI team, and i think that can be applied to any any person in in data. Your customer isn't always your external customers, right? They're your internal customers and having that empathy and understanding how they're going to be using, what value you can add, and building that credibility, I think,
00:40:03
Speaker
really important to sort of distinguish yourself as a trusted data team, as a trusted source. um so I think yeah even though that bigger a picture than answer there at Moneybox, I think the lessons and the themes could be applied to to data teams, and yeah whether you're building an external facing product or or internal.
00:40:22
Speaker
um Thanks for sharing that, Marco. um I suppose what final question on, I suppose, the money box and the ah Aurora is what's really ai governance for for
AI Governance and Future Plans
00:40:36
Speaker
you guys? um It's a big tick box. um I suppose it's more than a tick box exercise, right? But ah yeah, it's obviously a real core thing that people are are looking at. So yeah, it'd be good to sort of hear see see how your you look at AI governance.
00:40:51
Speaker
Yeah, um we so our approach to governance is clarity, really. So any AI model that we develop, first thing we all agree between our compliance teams, between us, between engineering teams, legal teams, everyone, is what is the mandate for that AI model, what it should do and what it shouldn't do. And then we operationalize that quite clearly in the metrics that we're going to track.
00:41:14
Speaker
Now, that is then the mandate within which the team building the model needs to stay. So that gives them a lot of flexibility that they don't have to continuously come in and check in. Can we do this? Can we do that? You can do what you need to do, but you need to stay within the mandate, within need the guardrails and make sure that the metrics look right.
00:41:30
Speaker
um We then do monthly reviews against the mandate, but monthly reviews also of various sort of telemetry we're getting from the models. And we score all of our models against a continuous stream of data that is generated by humans, right? So we have humans who check what the model is saying, check what the decisions of the model are, um score them, and then we check that against our benchmarks and make sure that everything is fine.
00:41:53
Speaker
That is pretty standard as in any industry. um Where we take it a bit further is um we have what we call second and third lines of defense, right? So after that first human check, there is another check by very deep experts who do a smaller sample and check that the humans are doing right.
00:42:08
Speaker
And then eventually we're going to be looking at red steaming and external auditing as well as the third line of defense. um This is a pattern that the financial industry actually knows quite well.
00:42:19
Speaker
So I would argue that if any industry has good AI governance practices already, it is the financial services industry because even though LLM's came about a couple of years back, finance has been doing credit credit risk decisioning for decades and automated sort of anti-money laundering checks and financial crime checks have been happening for a while as well.
00:42:38
Speaker
Both of those things are heavily regulated and you need to have ah both a paper trail justification and you need to have comfort because people guarantee in financial services with their own reputation and their own um actually personal liberties as well because you know we can all end up being criminally prosecuted if we don't do right by our customers.
00:42:57
Speaker
So a lot of the playbooks on how you govern AI have already been developed and we're just sort of um employing that in a new context, if that makes sense.
00:43:09
Speaker
Excellent insight, I suppose. za yell Taking land processes and policies from elsewhere, just fine-tuning them with with with slight differences, very interesting. um Before we wrap up, Marco, it's been a pleasure to hear about Aurora um and what you guys have built. and really excited to see sort of how i see it in person and and how how it works um for for the customers. um Looking ahead for for your team at Moneybox, what's on your your roadmap? I know you mentioned about democratizing ML and and AI. It would be good to hear for anyone that's interested of of what's coming up from from you and your team.
00:43:52
Speaker
So Aurora is is really the main thing that you're going to be seeing in the consumer space and we you'll slowly be seeing more and more of it this year um if you end up being in the group that we're rolling out to ah before we before we big bang launch it to to the whole base.
00:44:07
Speaker
um And I spoke about Aurora quite a bit already. In terms of the rest of the stuff we're doing, something that I'm very excited about and where again I think LLMs have a pretty good product market fit is Before moneyboxing, before LLMs came about or chatbots at scale, there was always a problem in data science departments that you could see in every department a long list of things that could be done better with more automation, but there was just no ah ROI to put a very like expensive data scientist there to do something for three months. right
00:44:43
Speaker
um And that always hurt quite a bit because you have departments in that can't benefit from the fact that we have increasingly more and more technology that can help with automation and productivity. Where I think LLMs help now, and we're gonna, we had some success with this last year, and so now we're we're scaling it out to the entire organization, is you can almost use an LLM as a consultant, where departments can use LLMs to refine their internal AI strategy without me needing to do the AI strategy for them. um They can figure out what are the initiatives they want to do, they can even do prototypes for those initiatives.
00:45:18
Speaker
I think where it gets challenging is that Doing that just with LLMs requires a lot of trust and you know that LLMs hallucinate and so that kind of puts people off. You can find a very nice balance where you do all of that and then you do have an internal team that can spend a fraction of the time just making sure that everything is kosher, right?
00:45:37
Speaker
And I think you can deliver a lot of value that way. That a lot of people can self-enable themselves to do things that 15 years ago you would have employed a data scientist or an ML engineer to optimize a part of a business process, right?
00:45:51
Speaker
Now, and the reason why we weren't going after that headstrong in the years so far in Moneybox is that Moneybox is a relatively new company. So it was built from 2016 onward.
00:46:02
Speaker
A lot of the processes in Moneybox are automated by design from the get-go and they were built as technology already. So there isn't that much low-hanging fruit to be reaped by employing LLMs and automators.
00:46:14
Speaker
But we are seeing, especially in some of the more creative parts of the business, a lot of value that could happen. And so we're encouraging leaders in that part in those parts of the business to work with me and my team and to work with LLNs as consultants to figure out um ways by which we can bring more productivity in the organization.
00:46:34
Speaker
And we'll be looking at that as one of the streams this this year. Amazing. I have to say that's definitely been a theme that that that I've seen over the last 6, 8, 12 months of and the role of a data scientist. um yeah the Obviously, still very technical, but yeah what was the differentiator for me is this ability to be almost be entrepreneurial, to spot these opportunities to be to to build products and be a product engineer.
00:47:02
Speaker
um that has become more more relevant than ever and there's a ah lot of low hanging fruit in these organizations to go around and um sounds like Moneybox is right at the forefront and it's got some very sort of digital native and and very sort tech focused people but especially in more traditional organizations and some of the clients we work with there's for data scientists to go in and when they've got the right platform, they can add a load of value, build a load of products with LLNs, algorithms to automate a load tasks. I think that's definitely going to be a trend, I think, which is why I yeah don't see AI being just just just just hyphen and a bubble. There's real tangible impact. Yeah, great to see that at Moneybox.
00:47:42
Speaker
I suppose, I'm sure if anyone has liked anything they've they've heard, Marco, um they can reach out to you. um I'm sure you'll probably be hiring over the the course of the year. and um If anyone's keen to to hear more and and learn more about Moneybox, I'm sure Marco is more than happy to to have a conversation.
00:47:59
Speaker
yeah yeah Absolutely. Anybody who wants can reach out. Whether you want to work at Moneybox or not, I i tend to like to talk to smart people. So if you want to talk, and do reach out.
00:48:09
Speaker
But yeah, I mean, as you will know, in your in your practices, um none of this, none of the great stuff that we're planning is it is possible without data. So we're always hiding hiring people in all stages of the data journey. and No worries that is important as in data engineering and analytic analytical engineering.
00:48:28
Speaker
You know, garbage in garbage out still applies. LLMs haven't magically made that go away. So, you know, data foundations are are more important than ever, i would say. Yeah, I mean, it's um couldnt couldn't couldn't agree more. It's the foundations of Cognify is built and specializing in data platform analytics engineering because um understood that you need good good platform and foundations to be able to do even BI, let alone ah predictive analytics and now LLMs and AI. So um yeah, if anyone's interested to to hear more or to speak to Moneybox, then reach out. um I'm sure they'd love to speak. and
00:49:04
Speaker
Marco, um yeah thank you for being for such like an open and honest conversation. I suppose before I let you go, um what would be your your advice and that you give anyone trying to build build an AI product, whether that is in a regulated environment or not? what what What's your biggest lessons um and and advice for them?
00:49:27
Speaker
The way I would say it is, you know, people talk a lot about how regulation is a hindrance. I would not look at it that way. Regulation... whether it's built right or not, is there to protect the customer.
00:49:41
Speaker
So the more you can put the customer at the center of your business model, the less you're going to see regulation as a problem, and you're going to find that you can actually move with pace. So don't treat regulation as a as an external um sort of anchor around your leg. Treat just as a focusing tool towards doing right by the customer.
00:50:01
Speaker
Amazing. Customer-centric. um Excellent. Well, thanks for time today, Marco. Thank you, Harry. Always pleasure to chat. Brilliant. um That's it for this week, folks. um We'll be back in a couple of weeks with ah another episode. um Thank you and goodbye. hi everyone. Just a quick one from me.
00:50:20
Speaker
If you've enjoyed today's episode, I'd be so grateful if you could hit that follow button or leave us a rating. Even better, pass the show on to a friend who might also get some insight from it. It really helps us grow the community and continue to share amazing conversations.
00:50:35
Speaker
I also wanted to take a minute to talk to you about Cognify. For those of you that don't know, Cognify is the leading recruitment partner for modern data teams. We help some of the world's best organizations scale data and drive real value from the hires that they make. If you're thinking about building a team or making a hire and you're struggling with talent or just want some insights on the market, then I'd love to jump on a call with you and tell you a bit more.
00:51:02
Speaker
Equally, if you're looking for a job and want to find your next dream role, then reach out to myself or any other Cognify team. We'd be happy to see if there's anything on our books that we can help you with and give you general advice on the industry.
00:51:15
Speaker
Finally, big thank you to Omni, this season's sponsor. If you'd like to learn more about the AI analytics that Omni can deliver you, then check out the link in the show notes or come speak to me. i can happily point you in the right direction.
00:51:28
Speaker
Again, thanks for listening and look forward to seeing you a few weeks time.