Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Richard He: The Cost of Inaction for Data Teams and Data Platforms
66 Plays1 month ago
Richard He, Founder at Fundamenta – Data Consultancy, former Engineering Director at Virgin Media O2 and creator of Practical GCP YouTube channel, joined Yuliia to discuss the critical concept of "cost of inaction" in data infrastructure modernization. Based on his two decades of experience, including several successful migrations, Richard emphasized the importance of proactive platform evolution over reactive large-scale migrations. He shared valuable insights on measuring ROI for platform teams, and bridging the gap between technical execution and business strategy.
Richard's Linkedin - https://www.linkedin.com/in/shenghuahe/
Recommended
Transcript
Introduction
00:00:00
Speaker
Hi, everyone. It's Julia from Streety to Talk, and we are back um in 2025, I guess. So, yeah, I'm excited to host today Richard He, who is also, he prefers Richard, he.
00:00:17
Speaker
I'm trying to do my best, Richard.
Richard's YouTube Channel
00:00:21
Speaker
And most of you might know him from the YouTube channel Practical Distribute. So I'm happy to host you, Richard. Please introduce yourself ah for those who might not know you.
00:00:34
Speaker
Yeah, thanks a lot, Yulia, and yeah, Happy New Year, ah and Happy New Year, Yulia, Happy New Year, everyone, right? So, yeah, so, ah yeah, thanks a lot, and thanks for the for the for the quick intro. So, I'm Richard He, but really, like, that is easier, and that's everyone call me, right, or wrong. Yeah, so who cares?
00:00:49
Speaker
so Yeah, it's
Career Shift to Data Engineering
00:00:51
Speaker
a good point. So many of you might know me from my YouTube channel PracticalGCP, where I typically share practical tips for real world strategies for getting the most out of Google Cloud, specifically in the data space. So it's quite amazing to actually see the channel ah grew a lot over the last couple of years.
00:01:09
Speaker
So just a little bit about about me, so I started my career actually in software engineering um probably two decades ago, right? So, but I moved into data engineering about like eight, nine years ago. And since then, I've been primarily focused on helping, I think medium and large businesses, modernizing their data platforms. so um I mean,
00:01:30
Speaker
more specifically like combining data and software engineering. i but I think that's a really important point like data and software engineering to scale their data capabilities. right So now I thought you might as well just do this full time. So now I'm actually running my own business that i started my my own consultancy. So yeah, really excited to share my experience as an insight with you here.
00:01:52
Speaker
oh Yeah, thank you so much for joining. it And I guess it was lucky we get to know each other it personally and ah when you actually started to do your consultancy full-time, ah which is just so super recent a couple of months ago, um maybe straight, right?
First Meeting at Google Cloud Summit
00:02:11
Speaker
Yeah, actually, we met at the Google Cloud Summit right in London. So that was the first time. So yeah, thank you for attending my talk as well. That was ah super nervous to start with, but I think I'm glad it went well.
00:02:21
Speaker
Yeah, we're so glad to come meet in person and do a bit of collaboration in here as well. it's Yeah, I know, the person in-person works differently, despite the all you know how we get used to speak over the virtual, you know, Zoom, Google meets, still it works differently.
Client Inquiries on Scalability
00:02:39
Speaker
But listen, my point is that we talked a lot about your consultancy, what you're doing today, and what actually caught my attention is that you mentioned that lots of clients are talking with you about the migration projects.
00:02:53
Speaker
How do they scale their business? and it Very open up business, but data infrastructure and or sometimes involves modernization, migration. We are not even talking about the on-prem to cloud. Sometimes it involves pure cloud.
00:03:11
Speaker
And um yeah, you recently published this video about cost of inaction, which has actually touched the migration process ah process and and scalability issues. um Could you please go a little bit deeper and explain what is the concept of cost of inaction and how did you see it in your practice?
Cost of Inaction in Data Migration
00:03:34
Speaker
Yeah, actually, you touched on a great point because ah that's a topic I really like. The cost of inaction is actually a concept I only came across recently when I was reading, ah but it's actually something as ah as a thing that I follow to practice in all of these. I've probably been in five different companies for the last like eight years, or mostly helping them with modernization and transformation on data platforms. right so And one of the key things of cost of inaction, I really like this concept ah because people usually compare it with ROI. But is a big difference. ROI is less estimated, it may happen.
00:04:12
Speaker
no But if it doesn't. A lot of the time it doesn't. right That's just the estimate. so But cost of inaction always happens. Cost of inaction always happens. So if you think about the difference is cost of inaction is you're looking at what is happening in your business right now. So in other words, you look at the current problems. What's burning?
00:04:31
Speaker
So when you see what's burning, and you say, look, within three to six months time, if you do nothing, if you let it slip, slide, right? what's gonna What is gonna cost in your business? What is the cost? What is the damage? So I found that is a very effective way to think about problems, to allow your pete to be proactive, right? So it's a term of inaction, but what it actually means is ah you you look at, if you don't do this, what's gonna happen, then you do it.
00:04:57
Speaker
right So you do it early. and So this actually is the reason I want to talk about this and also associate this with migration is because so just just be clear right so migration isn't a something you just want to do a migration it doesn't work like
Complexities of Data Migration Projects
00:05:10
Speaker
that. Migration is ah ideally you want to kind of avoid a too obvious migrations that's going to but cost you a lot of money and effort and everything right so that's I think that's not what most businesses want to see because if you have to stop and do a huge migration that some businesses have thought the bigger ones took more than a year or two to actually build magnesium. That's very costly. You have to hire an external consultancy that spends a huge amount of time with lots of money, and lots of teams to actually do that. And in imagine view that's too late. right So what I'm trying to do, the reason the business taught to me is because what I've been focusing on has been how can you identify
00:05:46
Speaker
ah The weather is about scalability of technology or the the actual compute or the number of people, concurrency, all of these things. It's important to look at where you are. in that If you imagine this is like a line, right? You've got currently growing linearly.
00:06:03
Speaker
But then but if you if you need to do a migration that to even run it as a parallel project that takes you a year, that means you're on the exponential curve. but It's not like you can actually continue to grow your business at speed. This is a huge problem. so But instead, if you think about the cost of inaction constantly, right let's say every quarter you have OKRs. You keep thinking about it. and If you don't list this quarter, the next quarter was going to happen, or a little bit longer for ah high level, a little bit of Vega plan, but constantly in that iteration, which is something I actually did a lot over the years. um And actually, this is also why ah I'm quite proud of this. I would like to mention that in the Virgin Media O2, kind of when I was a director of engineering there, that was a key reason why we can actually scale really fast. Because if you just think about
00:06:48
Speaker
oh yeah, we're going to do a migration at some point, or we're going to do a modernization at some point. That's not the report, right? You have to constantly thinking about, okay, how do you get onto the next stage so you don't actually hit that exponential curve so you can continue to grow, right? That's to remain agile. That's really, really key. um Yeah, I think that's that's the point I really kind of want to focus on. That's why I released that video, because I can resonate that kind of, as soon as I came across this term, ah can you know you always learn something new every other day. yeah so But it is so important to keep thinking about that, to be proactive, which I do not um believe businesses do enough these days. And then they usually end up with a hit huge bill, large projects that take forever. You don't have the smooth transition between these different technologies, which is which we which is what we were able to achieve in VM O2. So that's, that' I think that's ah that's something I was really proud of.
00:07:38
Speaker
This is really interesting point, but I think also it requires strategic thinking. This not necessarily should be in all carriers. This is a job of somebody into the leading position um ah to think strategically about data infrastructure, understanding all the technology cavities they have today and map that on the business growth. So, I mean, I should say it's very much expected, but not done.
00:08:03
Speaker
like everywhere, you know, because we're just humans. um This is
Preparing for AI and Emerging Technologies
00:08:08
Speaker
interesting. Also, you mentioned the scalability of business, but how about, of course, I need to mention it. um Let's say new emerging technologies and innovation like AI, yeah agents, and things like that. The infrastructure has to be prepared for that as well. um This also Initiates is it the right word to see initiates yeah initis the modernization of technology just to make sure everything interacts and and then connects well, I guess.
00:08:45
Speaker
Indeed. Okay.
Comparing Redshift, Snowflake, and BigQuery
00:08:47
Speaker
um Listen, in a video, you mentioned that some companies were struggling with redshift. And yeah, you personally mentioned that you've been immigrating lots of companies to ah Snowflake and BigQuery. And I have this question. Are those warehouses already, in your personal opinion, to meet the yeah next challenges that businesses have today?
00:09:14
Speaker
Yeah, to honestly speaking, to be honest, when I bumped into these conversations recently, I was shocked. I was like, people still using Redshift? I mean, there's there's no, it's just because I've moved away from that ecosystem. I didn't actually bump back into that, you know, right away after, you know, I started running a business myself. I was more hoping, let's talk about how to scale in GCP, in Snowflake, but it's actually turned out back into you're still on Redshift. But I think there's a reason, right? So as, you know, especially senior leadership, we have to think about ah what is the most important thing to focus on in the business. So every time I see a company on certain technology stack, I try very hard to tell myself, hang on a second, there must be a reason why they're using it, right? And actually there there are, right? so
00:10:03
Speaker
you know I'm not criticizing Redshift because it is well integrated in the AWS ecosystem. right it's not like If you use anything else, let's say, if you're on AWS or you are on Google, you can't just go to another cloud and different technology at the early days when your business need to focus on your products. right so the The ones, the businesses I see, usually they don't have a big data team, or they don't have a big data function. And that's usually not how businesses grow, right? They do have data, but it's not like everything you just suddenly you have so many analysts, you have Looker dashboard, it doesn't happen overnight. And it typically takes years, right? So this is interesting because Redshift actually served the purpose for a quite a long period of time, right? It was fine, until it wasn't.
00:10:41
Speaker
Right? Redshift, other people might ask like, oh, this server is, there is a Redshift spectrum. and There is like Athena. Sure, I get that. But if you look at the complexity, people started saying, like, what why do I have to understand so many different things that are not connected together? um So I think from the the business perspective, ah your question is, are people ready for the new AI era or whatever we don't want to call it, this new world of engineering, a different way to engineering?
00:11:09
Speaker
um i mean Let's put it, the the all tuniist new opportunities for business. Yeah, all new opportunities. I think the technology stack does matter to to to a to a degree, um but it's also more like, ah I guess, whether how focused your your business model is. Let's say if you're very data focused, you have lots of teams, a lot of verticals.
00:11:31
Speaker
yeah I would say, yes, you need a modern data stack, right? So to to make sure you have the right tools. So it's not just about analytical databases. It's also about you know how do you do ingestion? How do you model your data with tools like dbt, for example, right? You need to make sure those things are solid as well before you actually get into um you know the the the more predictive analytics or the logistic, ah sorry, the the analytics to actually understand your business problem with reporting as well. So those you need a solid foundation to make sure you have single source of truth to have good governance, right? And also security. You can't definitely can't forget about that. Forget about that. Which sometimes business as I focus too much on let's just do AI, right? I think it's good to not forget about
00:12:12
Speaker
I would say the 90% of it ah before you actually get into that side. um I think the majority of the business is on a modern data stack. ah So I keep using examples like that some of the businesses I used to work with, but let's talk about the most recent one, for example, Virgin Media. to write Yeah, they definitely are because they have a solid foundation that we build. It took us a huge amount of effort.
00:12:31
Speaker
We have very scalable data platform. We have data sharing capabilities that has been scaling to 400 people now. That gives you a solid foundation to take on those opportunities. right But if you don't have those kind of things, governance is going to be your first problem. This is really dangerous if you don't have a well-governed data platform to allow you to scale. Because a lot of this information people are talking about using to take on the AI opportunities,
00:12:58
Speaker
ah have access to everything. right That's quite dangerous. So you kind of need a solid foundation to make sure the sharing, the platform for data sharing is solid. ah And then we're talking about scaling. right how How does it scale? How do you share data without copying data? that the The performance aspect of that But is is yeah, many businesses are ready for this. ah But I think the message needs to be clear that you know it's not just that you're on GCP or you're on AWS, you're on Redshift. It's just tomorrow is going to work. right is Think about the the data platform ecosystem and your use cases. If you're data focused, you have lots of analysts and data scientists and engineers working together. In order to take on that opportunity, it requires a solid foundation. So it's not something we shouldn't forget about.
00:13:41
Speaker
Listen, I mean, we can talk with you about the data issues, data team governance and everything, but there is also a party that involves and or involved in all of this is business. And when but of
Importance of Data Infrastructure to Leaders
00:13:56
Speaker
both of us know exactly if the business doesn't approve something, we for we cannot move. Like it requires investment wasn in teams and um some money investments, right? So how,
00:14:10
Speaker
How did you ah encourage a business to go in for the change in data stack or change in data infrastructure? like it's so lot A lot of work and business itself help have um a lot of misconceptions.
00:14:27
Speaker
about scaling their data infrastructure because they just assume that if you work in data, it works. But we're supposed to do that out of the box. Sometimes it's hard to communicate to business why we need to do something. Migration is a big project with lots of investments. How did you, in your previous roles, communicate that to business?
00:14:50
Speaker
yeah Yeah, I think that's a very good question. I think my experience on this was very mixed. ah Sometimes I got very lucky with, you know, the senior management, you know, managing, you know, I'm reporting, I've been reporting to really at the im understand this from a high level. So I can come in and help them. Some others requires a quite a lot of convincing. um And unfortunately, how could all they oh Yeah, but unfortunately, the the this is the struggle there, right? So um I think to start with, there's something I definitely would like to see more is the businesses do at least do some analysis based on facts they have right now, right? ah But unfortunately, what I've seen in majority of the business, they always think they can get away from it. So I'm sorry, but it's like is' like until they they they you know see your things crash and burn, they always think they have another six months.
00:15:42
Speaker
this This is a typical mindset that is in many places I've seen repeatedly, right? But ah let's talk about resolutions, right? So I found what's most effective is to basically use other businesses as examples. You you don't have to name anybody, right? So these are methodologies and facts on how things have happened. This is the point I released the video without mentioning names. But he's those are, ah so as soon as PPE will know what I'm talking about, right? They will say, oh, hang on a second, is this us?
00:16:09
Speaker
so ah here yes it's this this but simple But this is important. right i want to you know The thing I want to help businesses is ah how can you not leave things for too late? right Is there a way that I can ah resonate with you to make sure where you are right now you know where you are on that curve?
00:16:29
Speaker
Right, but it's still hard ah and because I think it's because of the fear of change. So this is this is especially like you mentioned ah rightly is the cost and it's not just the cost of the migration the time is the upscaling the learning, you know, how how do you even let's say if you need to migrate to Google Cloud or snowflake or your AWS ecosystem, right?
00:16:49
Speaker
That is a change that lots of people would be very careful with, right? Because this means you have to change a lot of things. so But what this means is um the the business will need to think about the shortest path to actually get there.
Proof of Concept Before New Technologies
00:17:01
Speaker
So that to do that, you can do things like a proof of concept.
00:17:04
Speaker
so yeah Usually, you you you can easily get like credit to basically from these cloud providers or hyperscalers or technologists to actually try out just to see what happens, how far it is, where your team is right at now at the moment to this thing. and Does it actually solve your problem?
00:17:19
Speaker
because sometimes these technologies where it all looks good, but it doesn't help you. So I think a proof of concept is essential to reduce the risk. And now if you actually do that, I found, um so this was actually one of my ah situations when, ah you know, a long time ago was in a company that does lead generation.
00:17:37
Speaker
So so that was a we werere looking were looking at AWS and then we got Snowflake as an option. Initially I was like, yeah, no no no one can solve this problem. right But the the proof of concept was something that really gives you this insight on either it doesn't work, then you write it off, you try something else. right Or you just realize this is the next thing.
00:17:56
Speaker
You know, that, that is, that is a game-tending moment. You only know, this is why I talk about, you know, in my channel as well is practical. The word practical or practitioner is really important because you're going to have to try it at a very small scale. But for some of your more complex use cases, not the two easy ones, because those dogs are not going to give you the proof of concept. There's relatively complex ones. And now you can see all of your problems on, Oh, excuse me. worry All of your problems currently on the left, uh, and comparing all of these opportunities on the right.
00:18:27
Speaker
Can that solve the problem or not? So that gives you the opportunity to spend a little bit of money to prove it before you go all-in because that all-in may be wrong, right? So that's that's a something I do a lot. I found it very effective, right? So even in the recent, ah um and in BMO2, when we scaled down this hub, that was exactly what we followed with the proof of concept, making sure everything's fine. Because sometimes there are gaps in there. You need to resolve this before you go all-in. What if the product is not ah ready for your production use cases because the one feature was missing or there's a bug?
00:18:57
Speaker
For the more mature ones, it's less likely, but
Assessing Business Impacts of Inaction
00:18:59
Speaker
for some of the newer ones, especially if people are thinking about using some of these more modern ones that I'm not going to name it, like really new-ish ones, I'm just going to go all in with it just because everyone is talking about it.
00:19:10
Speaker
I think that can be quite dangerous. right So doing a POC, but first of all, looking at what are the problems you have and trying to resolve those. If you don't have a problem, don't solve it. But it is important to look at the COI. I think that's ah back to that. So now I keep using that phrase because I didn't realize that's what it's called, but now I know. and it's It's always look at repetitively, I would say on a quarterly basis to say, look, if we don't do these things, what's going to happen? I know it feels a bit reactive, but it's actually proactive. No, it's 100% proactive for me. ah This is very interesting. So but let's, like, you know, ah let's play this game. So I am a vendor. Okay. I'm a vendor and basically you can act as a yeah director from the big reach.
00:20:00
Speaker
but fabulous company and how would I, how should I prove you that the cost, how should I prove you or and encourage you to do the POC and not you on this co cost of an action? Like what would trigger you immediately to jump and do the POC?
00:20:26
Speaker
I guess what ah what I know it will trigger me to make sure ah you know I don't want to do it is try to sell anything to me. right So i think I think that's usually the the worst thing to try to sell is to actually sell. I think the the ah ah from my experience, because I've actually rarely been on the consulting side, I've been kind of mostly within the businesses and doing stuff.
00:20:46
Speaker
um so this is this is pretty new to me but i think from my perspective it is very important that most of the business leaders and wanting to do one thing which is solving their problems i think is is the the thing i think is very useful uh then and really really important to follow is if they don't have a problem or if i don't think i can help them i'll back off right So I think that it is an important thing. um But if the business does have a problem, then I do have a solution. I think it's important to ah focus on how how that problem might get solved. So I think it's...
00:21:24
Speaker
is for businesses to try to, it's not a simple problem, right? So it's it's like it's not like ah we we go to a business ah or you can come to a business and say to me, hey, you you don't do this, everything's gonna crash and burn. It's important to have more facts and more details. So i think I think that's important to make it transparent, as transparent as possible. So it's not about this product comes in, we plug it in and everything works. It's about these are the key information that I'm saying, look,
00:21:51
Speaker
and you're not doing, but if you do this, this is potentially the game. Do you want to have a look? Right. So that I think that's more important and it has to be focused on the problem the business is actually having. It's really interesting. But um still, you know, even even the POC is a yeah time and investment by the team.
00:22:13
Speaker
Yeah, exactly. And lots of businesses, especially the ones i um I'm actually ah talking to, they do not have to talk. They're all very busy. I think that's that's probably one of the more difficult part as well, is is is time. um But I guess this is this is one of the things maybe ah is kind of a working cultural thing, a cultural thing that leads to everyone getting too busy, right? And then not really thinking about ah strategic one was yeah Strategic thinking was going to happen. So it's funny, like one of the things I ah encourage a lot, ah I actually had a quite big team at at Virgin Mido too, I had a team at the end I was 14, right? So it's like four different divisions. I think one of the things I actually try a lot, not everyone does this, but I actually as a manager to look after these guys, one thing I say a lot is,
00:23:02
Speaker
Spend some time every week, like use your Friday afternoon or something. Stop working. Think out of the box. Because um I think there's an old analogy from the software engineering world and say you spend 70% of time thinking at 30% of time coding. And that's so true. There's so many areas that's the same.
00:23:18
Speaker
If you just keep doing what you're doing, you're in a box, right? that that's That means you're not really thinking, solving it. You might be solving a problem, actually getting the company into a much worse situation, right? You might be using a solution that someone else can, or a different solution can just solve it in like five, 30 minutes. If you have an observability tool to look at your cost,
00:23:36
Speaker
you plug it in, you try it once, and now you realize you don't have to do three months with your own time. I think this is important. So I think as, but this is this is why I was saying sometimes I'm very lucky now working with very, ah you know, forward thinking ah yeah senior leaders. Although they might not be very technical, but they actually know, right? You you need to open to opportunities. You know, some of these migration that happened, um you know, even in the old days when snow, this is way before snowflake, when IPO,
00:24:05
Speaker
And my CTO at the time really kind of like, Richard, you need to think about, I was young, why is this like clueless
Visionary Leadership in Tech Adoption
00:24:11
Speaker
at the time? Well, not clueless, but I would still what like to say I would know i was not a senior leader at the time at all. ah But that was like eight years ago. And anyways, so he had this vision, look guys, let's look at this new thing. It was, Snowflake was like a hipster, right? ah In 2017. So it's so new. ah ah I think in the UK, they only had employee number six.
00:24:34
Speaker
sorry in yeah mma not just the UK, in the entire Europe, so it was like tiny. um And it's mostly sales, right? So it it requires an incredibly bold leader, ah which was my CTO at the time, to really see, okay, hang on a second, you guys are trying to bury your head and solve this problem with Redshift, and why not have a look at this new thing?
00:24:55
Speaker
so bear A lot of these times, it was senior leaders had the vision. I can't thank him enough. like that That was the one thing that changed everything. but you go You kind of have to have the employees to backing it up as well. right look and because I got quite excited with this thing. Initially, I was quite skeptical, but then what I did was I wrote a long list of questions like this. I said, it can't be true if you can unless you can ask all these questions, which it did. i was like And they tried it as well, right? And then we tried with the data was like, how can I say, and that's not possible. And then we ended up with, okay, explaining how the decoupling of computer storage work. You know, that's how the conversation got kicked off. But again, it was started with the business has actually had a problem. We have we have senior leaders actually understanding, oh, let's just explore something. There's nothing, you know, it's just to try something small, right? It's not gonna cost you a lot of time. It's a Friday afternoon thing, right? It's one of these Friday afternoon thing.
00:25:42
Speaker
But then it actually turned the company into, I think the second year because of that thing, they made 400,000, something like, it's public figure, ah just not gonna name the company, but they actually made a 400,000 per month uplift.
00:25:58
Speaker
it was It was massive in terms of percent percentage of revenue new gain, right? So but what if you what if the senior leaders did not think, did not look around, did not look at the different options in that, even when they have a problem, right? That's, I think that's key. But if you ask me, how do ah how do I make that happen when the senior leaders don't do this? Your options are limited, I'm sorry. Right? So this different company was different. Do you know the company?
00:26:24
Speaker
I think it's worth like a you know the the the teams and the mid-level managers and you know the more people, the principal engineers, you know you can all try to pitch to your senior leaders in a way that is more structured. right Don't speak the tech language, but speak something that is more easier to resonate based on, I call this kind of switch thinking, basically look at, okay, what why does my ah my boss don't want to even look at this?
00:26:50
Speaker
is maybe he's got some other or he she or she he or she has other things that they want to focus on. right Maybe this is not a priority. Maybe we're a product-based company. Data is irrelevant. But at least I think having that kind of conversations can open up things a little bit more, rather than ah rationally just say, oh, this shit doesn't scale. right So it's it's better to just put a bit more structure when communicating with the senior leaders. I think that's when ah you know that the opposite can also be true and things can change.
00:27:18
Speaker
the Yeah, very, very interesting topic. You triggered me or something in there. That's good. That's good. Listen, I have another question to you. Well, we talked about all the, you know, nice stuff and nice experience you had in your career, but how about mistakes that this is made?
00:27:38
Speaker
when scaling their data infrastructure. like Do you remember any new experience? like I don't know, the investment in Snowflake back Brilliant. It's just a brilliant decision, right? But how about this takes?
00:27:55
Speaker
So personally, I would say since 2017, I stopped making mistakes because of the mistakes I made i made in
Preventing Mistakes in Tech Adoption
00:28:01
Speaker
previous places. I mean, the you know, ever I make mistakes, right? The things I try to avoid is try not to make huge mistakes and those can be prevented. It's not actually that difficult, right? The proven concept with a relatively complex use cases you have to prove that new thing is gonna work or not is key. So I do not pick any software, any solutions to scale it for a team unless I've tried that first. Now I have to personally review it myself to see I have a checklist to see you exactly you know what are the things the tick box is taking, what is not, and then I'll ask the team and ask the vendor. If the vendor can't answer my questions, you're talking shit, right? So that's that's really important as well. You need to challenge the vendor and say, look, this is my problem. I saw this issue when I did the POC. Explain it to me.
00:28:40
Speaker
why it doesn't work. I think that's that's that's important as well. so And then luckily, I think it's a bit of a luck and also kind of maybe this is business I was in, but I think more importantly is this CO cost of inaction style thinking, but this concept, which keeps ah me in this zone that I won't go ah BAU, if you don't do anything at all, it's BAU, BAU until big bound. And then if you think too far forward, right if you try to do everything at the same time, that also causes failures.
00:29:13
Speaker
um because this is something mistakes are made in very early stage of my career. I was trying to bring different frameworks together. So when you try to introduce something, don't do like five things at once. Do one thing at at a time, right? So that is another thing that can cause failures. um I think one of the large scale ones, are when I ah first I was with a company where you're trying to solve our scaling problems with very high traffic. So Black Friday traffic was 30,000 messages a second in 2014.
00:29:38
Speaker
right That was still on-prem time. So the the I think that's when I learned this kind of ah importance of proof of concept, but you can't cover everything, right? Because sometimes the time would not allow you to try everything. ah So once you feel good enough, I think it's good to have a mindset and just expect things to fail.
00:29:57
Speaker
and then just use your team and work together to get through it. I'll give you an example of what it was, right? So I was using HBase in 2015, 2016. That was absolutely really crazy technology that yeah unless you understand distributed computing, that was on cloud error, right? Cloud error was actually the pioneer back in the days. ah So, but it got too complicated, and you know, this is why the kind of things got split quite a bit.
00:30:21
Speaker
but Back in the days, HBase's combination with Kafka was like the golden standard in the Hadoop ecosystem. um Because it used commodity software to allow it to scale. However, it wasn't on the cloud. so The biggest difference was if it's not on the cloud, the amount of knowledge you need to have to run a distributed system was extreme. so For example, if you don't understand how minor compaction or major compaction works on HBase, which is what screwed me,
00:30:47
Speaker
It actually has a scheduled ah major compaction run every 30 days. You know, how do you know that? Nobody, even a lot of people didn't know. So we ran the thing for a month. I always thought everything was great, right? And then everything went to when, when, when, when, when, when bunkers, when bunkers on a weekend, I was like, every alert is just basically flying over all over the place. I was like, what the, what the heck is going on? But you can't literally see this at all.
00:31:09
Speaker
So I spent like 12 hours without sleeping ah ah you know overnight. was like And now I realized, and you know what I was doing? I was reading documentation. It's crazy because you do not know this. None of the knowledge monitoring is showing this is happening, right? So my point being, sometimes I think this mistake's a lot harder to make these days, but it's not a silver bullet, right? If you run these technologies on the cloud these days, you still have to understand, let's say, how do you distribute data? How do you partition your data? It is so much easier, but I think one, don't expect not to make mistakes because most people will, they still will.
00:31:46
Speaker
but it's to reduce the chances of making big mistakes. right this is This is why cloud is better in a way because you don't really yeah you can't really make those kind of mistakes anymore. Who cares about complexion these days um um on or any of these technologies that's run by a cloud provider? You don't, right? They deal with that complexity themselves. It's good to know But it's much less likely to make mistakes. But and but the mistakes kind of spectrum has shifted into a different area. right If you adopt a new technology, the way you use it, and the amount of money you you have to spend, the time you wasted for the migration, those can all be different kind of mistakes. right So because things are a lot faster moving, you probably end up doing a lot more things, adopting a lot more technology, a lot more software.
00:32:26
Speaker
So the more important thing is to really think about the the problem that at hand that you're trying to solve and when bringing technology in always do a problem concept. So that that's the the key thing I would say and that would cover I would say 80% of the problems. The rest 20% is actually not that difficult to solve. It will be sleeping nights. That's always the case just running any business.
00:32:45
Speaker
It happened on every single one of my jobs. But it doesn't happen very often when you get these things right and ah started thinking that every quarter, what are the burning things going to happen in the next quarter or in the sixth time? And you start prioritizing your engineering teams to start putting um fixing these things in the proper way to find the root cause, fix them. If you don't understand something, learn it properly. but Then you understand, OK, this is why this is happening. And now you it will no longer happen. Then things don't fail.
00:33:13
Speaker
I think that's an important um mindset to have in you know running any teams, adopting any technologies, in my view. That's very interesting because you just pitch in the old fashion and it comes across to me as there is no way to cut corners.
AI's Role in Productivity
00:33:30
Speaker
You just have to have to do things properly and then it will work properly. Like you just cannot ask chat GPT or ah quote. eat um i wish I wish it doesn't solve all these problems I have, but they so far the
00:33:45
Speaker
the things only solved was when I tried to write an introduction of myself is very helpful. And honestly, you know, coding is very helpful. I think there is, I think a lot of the opportunity you touched on that point, because a lot of the opportunities on the, on the part I really believe is assist. Assisted. Yeah. And if more assist on within the company, not necessarily outside of the company. So I think I call this lower hanging fruits, right? Because let's say if you're within the company, you can actually do something based on your own information, structurally a little bit more and generate code at scale with better quality, better prompt.
00:34:18
Speaker
I think that the productivity gain can be very significant in there. But I think a lot of businesses is just say, yeah, let's try this. There's no structure in there. I think that's probably my view, the the thing, especially in the engineering world. That's one thing, because I focus a lot of this stuff on myself, but obviously now I'm running my own business. But it's one of the key folks. I think the business should tackle those low hanging fruits to ah help improve developer product productivity.
00:34:41
Speaker
It's efficiency efficiency of employees' time, basically, I see it as that. I don't want to get sidetracked. AI, agents, all the internet is very interesting. But if we focus on something more traditional, um you talked a lot about modern data stack, onboarding the tackling the BOC, but also how do you personally you measures the return of and of an investment on these new onboarded solutions. like How do you understand if you... Yeah, return of investment. What it helps you? like how Do you have to pitch it normally to your business stakeholders? How do you justify it to yourself? like What is return of investment? Yeah.
00:35:28
Speaker
Yeah, it's interesting because ah in the date because I specialize in data platform. In the data platform space, it's actually much harder to try to prove this kind of thing, so it's actually more challenging. ah Put it this way, if you're in a vertical, let's say you run pricing engine, the the algorithm that you tweak would have a direct impact on the conversion, right?
00:35:47
Speaker
So then you can easily measure your ROI. In a data platform team, your stakeholders are not necessarily anything to do with your customers. It's your peers, your other teams, right? So ah so the things I usually do, initially, but to be honest, this took me so many years to figure out exactly what to do to measure this ROI because many times, especially the data engineering teams, the business don't put mass investment in the data engineering team because they don't see the outcome. They think everything is,
00:36:14
Speaker
It just works like this, right? It just works, right? so And I think this is where storytelling is really important and to base down stakeholder feedback because stakeholders, even the internal stakeholders, they're still stakeholders. So I think the key number one thing for me ah is about um making sure you measure the productivity gain.
00:36:38
Speaker
I think that's, in my view, the most effective way in data platform teams. So I'll give you an example.
ROI from Data Platforms
00:36:43
Speaker
so And this is probably the the one I would say the most direct way that the business really see the impact is that analytics hub scaling thing that I talked about in the Google Cloud Summit. right So the reason I want to talk about it is and if you look at the productivity gain, I actually put ah the hours in there. It was like something like 20 hours wasted like per week. ah and You can measure that based on um the amount of time people spend in the old solution, like pull requests, reviews, communications, all of that, you can you can have a good idea to measure it, right? So we came up with like, it's about 20 hours. That's, if you add all this to together, all these teams, you got a ton of time. This is the amount of time the data engineering spend on the product, sorry, platform team, or you're talking about business stakeholders who are consuming the data.
00:37:31
Speaker
is is is everything combined. okay So it's it's um it's all of the data consumers needed to get access to that data in a way that was confusing. And there there was a damage that it with the old solution, which was roughly 20 hours right per week. But then, um with the new solution, you can look at ah the productivity gain. Because if you ask, let's say, 20 people, how bad it was, you get a good idea.
00:38:01
Speaker
If you ask tiny people those 20 people again after it's done, you know after the the rollout was finished, then you get a good idea. Then that's where the 20 hours versus 30 minutes came from.
00:38:12
Speaker
Then you tell the business, look, if you do not have this, ah that 20 hours could easily multiply because it's an exponential problem. It's not a linear problem. So that 20 hours is just what you see at the platform tab. Best of an action. Yeah, but then if because they have to keep ah asking questions, doing pull requests, doing reviews and fixing their problem,
00:38:34
Speaker
which means they are always getting distracted and they're annoyed. With those things, you can't easily measure, but it's like a happiness score, right? It is a convenience score or a happiness score. So if people are so annoyed on using a solution that a data platform team has provided, you're gonna have to do something about it, right? So, and once the feedback turns into, this was the most amazing thing I've ever used, right? If you you interviewed 20 people before and after, they give you this kind of feedback.
00:39:00
Speaker
then you know you're you're successful. And they will vouch for you to tell the stakeholders that this is why we can move a lot faster now. so And that is a multi is a multiplier in the business, right? this is This is why I think it's hard to explain why a data platform team is useful, because you you don't usually talk about it. People just expect, oh, it's gonna work, right? Oh, yeah, and this is how amazing. You just share data, now you have this and that. But what people don't understand is exactly what happened behind the scenes.
00:39:26
Speaker
is how do you have to set up this governance model, the security model, the how people work together, who owns what? right What if something goes wrong? Where is the monitoring? That's the hard work people don't really talk about. right But then when when you make these things transparent, and then people are actually more likely to, oh, actually, you guys actually did something amazing. this is how So you know in a way that a business so can actually understand when it's measuring the ROIs, I think it's interesting. Many of the businesses don't really care about measuring ROI for platform teams.
00:39:56
Speaker
ah But like I said, that's probably the direct result in the platform team doesn't get budget. um yeah you very sure how how they measure So again, how do you measure it? It's super difficult to measure because ah platform team serves horizontally to the entire organization. They don't get they don't have these direct benefits in a way that they can show to business. Like, okay, we generate this data set for sales team, so they want this amount of deals, but it's super indirect.
00:40:27
Speaker
They think of it in a slightly a different way as a collaboration model. so ah First of all, what you're asking is a very hard problem. Not many business have figured it out yet. um the but the the The things that I find most effective with collaboration. so If you bound with your vertical teams, right? Platform is horizontal. If you bound with your vertical and say, look, this is what we're providing for you. This is why you're getting married. Can you actually tell your stakeholders we're in this together?
00:40:52
Speaker
It's about a collaborative effort. it's not like you i'm cut but The traditional problem with the engineering teams in the platform or data engineers or analytics engineers is the business comes to you and says, hey, I want this report. Oh, I want this thing done. I wonder to ah ah thread model um ah sorry i want to a detection model, ah but I don't care how you do it. Just go give me this stuff. It's most like throwing this thing to the other side of the team. or Oh, I just need this data.
00:41:15
Speaker
But it's not thinking of the data platform team, not the engineering team as as part of the team. I think that's that's the biggest problem out there. But integrat with the teams I see work a lot better, with the platform team getting a lot more attention and support, are the ones that actually blend together.
00:41:30
Speaker
ah So that's that's how you I think it's much easier to, I used to ah ah we used to kind of play a joke about this kind of stuff. It's not actually a joke because my peer was the director of data science. So we are like, ah if you are screwed, I'm screwed as well. as right It's a is like a is like a ah mutually affecting each other and also benefiting each other's situation. If the way we will win together, if we fail, we also fail together. But how do we fail? On a platform team, okay, your data ingestion is very slow, and it breaks all the time, and then you don't actually have single source of truth, there's no catwalk. All of these kind of things, people think, is oh you you just should have it, but when they look at the data lake with hundreds of thousands of tables... um and
00:42:10
Speaker
just tell Try to put an AI model in there and solve it. now i'm I'm really looking forward to see how that's going to work ah yeah so should you think come in Yeah, I think is the collaboration in there is really key um to actually solve the problem, but it's a very hard problem to solve. I don't think I nailed it exactly, but I definitely had ah you know good direction to how this can work and how the different teams can really benefit. Is the energy yellow level as well? You don't really want to just come to work and just does a work job. You want to say, hey, if you do this together, this is the benefit is going to grow. And then the platform team, usually they don't care about the business value at all.
00:42:46
Speaker
just I'm just going to give you some data. But then it encourages the platform team to also ask more questions. Hey, what is this for? Why do you need it? And that helps with prioritization too. So which means the platform team, that your team come the data engineering team specifically can focus on more high value ah data sets. So I think that's that's that's ah yeah and that's my view of how to tackle this.
00:43:08
Speaker
and that this I mean, i it's very much resonates with me because what you're saying is like the humans talk, get qualitative feedback, not just what should the metric like watch metrics you know in some dashboard, just get the feedback from the people and they can suggest you and you know you can enhance your collaboration this way. but This is, I love it. I love this approach. My, one of the last questions is going to be, Richard, when you come into organization today, you don't have this much power as you used to have at your previous organization, right? They more see you as a consultant and you need to act.
00:43:48
Speaker
fast and you need to provide actionable insights or whatever. I'm very curious what you do today. What is the more like the biggest request you have from organizations and how do you help them? like Where do you see the biggest value and who are your basically ah target audience who can benefit the most from you today? like How is it structured? What do you see from the cases so far?
00:44:17
Speaker
I'll let you know when I figure it out. But yeah, it's very early stage, right? You keep in mind I only quit like ah less than three months ago. So it's not actually that long at all. But I know that you have a bunch of customers so far. And actually, but because they see that you can come in and act. And yeah, tell me what what is the best way to to deploy your basic?
00:44:36
Speaker
I think I could like to position myself as a trusted advisor or collaborator or partner, whatever you want to call it, to basically focus on the problem they have, right? I think I i might be running a consulting company, but it's the things you can see from all of the content I share, you know, this thing about Redshift and then the scaling issues, I mean know it is still the problem. Now, as I start seeing a pattern of what what condition I immediately create something i like, let's just This is ah what I wanted to say. This is very detailed. I give examples. I tell exactly how to look at this. right So I actually really deeply care about what I do. Otherwise, I would not be you know quitting a job. I wouldn't use the word power. you know You don't really have power in those organizations. But it's definitely a lot more influence and you know in a larger organization when you're a mostly successful, you know building teams and large scale transportation programs. So right now, yeah, definitely. they But but you know if starting a business is that easy, everyone would have done it. right
00:45:31
Speaker
so But this is the next stage of my challenge is I didn't do it because I want to make a lot of money. is is i I want to do it because I want to prove something that the things I did you know with and for these businesses for the last decade has value.
00:45:47
Speaker
right so i i so mean If I go to one business at a time, if you think about the if I didn't create the practical GCP channel, which many say look richard you really help me with this and this this thousands of people benefited from that kind of content. right That's not a business, but it's something I deeply enjoy doing to share.
00:46:04
Speaker
um But obviously I need to figure out how does that work in the in in business context more, right? And I personally, I don't like selling, like like I said previously, I want to focus on problem solv solving, but I believe, ila you know what I'm figuring out, but art at this stage, yeah, I do have customers, but it's... is I'm still trying to figure out how do I achieve my big goal in, you know, I i don't like people, it's five years plan, I actually have a 10 year plan. I try to help, let's say, ah thousand ah sorry a hundred businesses at the same time, right, in this status space. So it's not like helping one business at a time. I really wanted to share this kind of a knowledge. So this podcast, ah and thanks for inviting me you doing this, I think this is valuable as well as well to
00:46:45
Speaker
you know get other businesses to understand at different stage of their business, it's not just technology, it's not just data, it's when to act, it's when to act to avoid a much bigger costly problem. And this is something I've been really, really good at over the last decade in these different businesses. I think i think that's that the real you and the most important thing I want to focus on, however kind of shape and form it takes. I don't actually mind.
00:47:13
Speaker
I think running a business is is a venture, right? So he's so it ah it's ah is something I deeply care about doing. I'm not gonna give up easily. I think that's what I would say. You know what, um I do think one of your strongest, like you have also strong, um how do you say, strong skills or the one of the most strongest thing about you, you can go deep into details, like with practical GCP. I don't think there are so many people as you that can go this deep and also explain this simple to the audience. But also you can switch from this detailed context to a strategic task, like in a flip.
00:47:52
Speaker
And you can map business and technical ah issues together, which is unfortunately not available to also people out there, ah because mentally it's difficult to change the context this fast as you do it.
00:48:09
Speaker
I would say, and I think you're like this is one of the biggest assets of yours,
Bridging Technical and Strategic Issues
00:48:14
Speaker
being able to connect those and and being able to adjust as well in different business ah circumstances, I guess. I see one of the strongest sides of yours, this this thing.
00:48:29
Speaker
yeah Yeah, thanks for that. I think it's important, right? So it's a gap-filling exercise. I believe the real consulting world is about is filling the gap, right? So there is a proper term in consulting, which I read a random book the other day, it was like, which I didn't finish, but it's called a value distance.
00:48:46
Speaker
You know and the thing you described is about because a lot of the companies they have this issues like the senior leaders know what what do they want to do but it don't really actually know what they want to do or how to how to exercise it the technical guys can't explains in in in their languages so the gap is actually getting bigger and bigger from what I see right so and because people think this house is easy right no it's not so it's hard to explain in a way that when people say this is easy how to give a bit more context of why it's not that easy. So you bring things down a little bit. But when people say it's too hard, which is usually the engineer says, it is probably not that hard. You kind of want to bridge that gap. And then finally, that solution in the middle middle is to close the gap, which is the value of distance. So if you can close that distance, and that's that's what I call proper consulting, ah I think it's
00:49:31
Speaker
is His analogy has been created like a decade ago. I can't remember who wrote the book, but it's very important. I think it's the key thing. This is why um you know to turn that into a real language is solving businesses' problems. It's always about solving problems. ah If there's no problem to solve, which some business is actually working well, well and that they don't need a modern data platform, and you try to give them a modern data platform, that's a disaster.
00:49:55
Speaker
right So it's about what the business is actually experiencing, you know, back to that COI thing, which I think is a really good way to spend less cost in a continuous way so the business can keep growing, right? So the key that we can't ignore is no one wants the business growth to stop. If you have a very strong um Let's say I want to do something, this this is whatever. Then you realize your data platform doesn't scale at all. When you're just about to scale, that is going to be your worst nightmare. But if what if you can start preparing your team? By the time you want to do that, you already have something that can support you. This is what we did in VMworld. So that's why we didn't have to stop at all, which kept on going. right That's really, really important. But the process, the the details in the execution. So this is where you are saying the technical skills and ah the the data strategies. Those two things are not that different.
00:50:47
Speaker
is to think about the technical school, how can you turn that into a business context? But the reason, i I guess the reason I can do is because I've done both side of things for a extended period of time with failures and success. I typically, you know, I fail too, but I don't leave a business until that problem is a resolved.
00:51:04
Speaker
So that's a very important thing. You don't just, oh, it's a problem that you move on, right? that's thats That's failures and after failures. But if something fails, fix it, make sure the businesses turn the curve. They started going up. but And then if you have no challenges, you move on. That's fine. But don't move when the business is not in a good shape. um I think that's the important thing because because I've always been doing that for a long period of time. You kind of see it, right? So whenever you have a problem, I can say, okay, yeah, that's that's because you're on this bit.
00:51:30
Speaker
or the other business when they're bigger. Because I also work in different businesses, I work in startups, ah scale-ups, mid-size companies, enterprises, corporate, you know, it's a very different many industries. So it allowed me to see um how the different business at different stages will need. right I think that's quite unique. I don't, yeah, I think you're right. I personally, I've not seen many people can do what I do as well. ah But yeah, i'm I'm in a different stage of challenge and I don't know how to scale, how to kind of ah help multiple businesses, how to turn this into a business. I think that's that's the fun part, right? It's it's draining, it's emotional, but it's it's ah also at the same time like so so so much fun. It gives me so much energy. The the there other a point that I wanted to highlight right now is that you never stop learning.
00:52:13
Speaker
Like even this powerful GCP, I cannot imagine how much content you need to digest and reprocess to be able to expand that in a simple human language, how to deal with it. Exactly. And this is what gets you going this far with execution part and being able to map that on the yeah business and strategic level. ah This is very interesting. I mean, this takes a lot of time and and resilience as well.
00:52:40
Speaker
I have to say, if you don't like this kind of stuff, you can't do it. So this also is why I'm saying I'm not doing this, you know, just, I can get rich, right? So, but it's, it's more, there's very rich people actually said this repetitively. Like if you don't have something that really moves you in life, you can't, you can't run a business. That's absolutely true. I feel it's so, so true now because, you know, I can, I can definitely say the journey to do entrepreneurs, whether you start with yourself or your co-founders,
00:53:08
Speaker
The first, like, I would say, you know, I've been doing this for full time for three three months, almost three months now, right? I can say confidently, this is the most a difficult thing I've done in my life.
00:53:20
Speaker
So, but I get better. But it is also like true that if you have lots of resilience and you like this kind of stuff, which I do, and this is something I like a lot. I like funny. So I really like talking to people and then understanding their problems. I'm not like one of these engineers just sit in the back office, right? I really like to go out there, meet face-to-face. This is why the practical TCP London group, I really want to, you know, let's get everyone together. Let's talk. yeah The face-to-face stuff is is something I really like doing.
00:53:49
Speaker
but But also, you know, I also realized as we are um getting to the end about wrap up the podcast, I guess yeah it's my first podcast where we didn't mention data mesh.
00:54:07
Speaker
You are so much into the practical side of things. Do you want to mention data mesh? It's okay to talk about that.
00:54:16
Speaker
Yeah, I don't even know, like, it's so confusing sometimes. if if so Which definition would you like to have about data stuff? Yeah. um No, i I do appreciate you following the time to chat with me, because I have a lot of respect for what you're doing as a professional, as a leader.
Conclusion
00:54:38
Speaker
um So yeah, um see you in London.
00:54:42
Speaker
at the... your practical discipline. Yeah, practical discipline. Yeah, 30th of January, actually. Yes, of course. If any of the virtual meet you or two guys listening to this, I want to thank you for providing the office. It's going to be in their brand new office in Pannonon, so it's amazing.
00:54:57
Speaker
It's amazing. That's true. Also, if you'd like to ah you know to get a max meetup, follow Richard for more updates on that. and um yeah Thank you so much for listening. Richard, it was such a pleasure. Thank you so much for stopping by. No worries. Thank you very much for inviting me.