Transcript
Speaker: I realized that there was a need of like ah changing the operating model and change the culture. So the thinking. So DBT was the tool that enabled us, was not just the cheap transformation. The transformation was in the thinking and how we work and now we do what we deliver. It's our duty, it's our job as a data professional, as a data leader, to make sure that they understand that we cannot go time and time faster be wrong. The context is what's gonna change everything because ah without the context, AI is gonna return a confident answer without giving you the the real business context, the real metrics. Like, you know, like a confident lie is still a lie.
Speaker: Today's episode is brought to you by Omni. Most companies I speak to want AI analytics but are failing to put projects into production. That's where Omni is different. It's the semantic brain that grounds AI in the heart of your business logic, giving you governed answers, whilst also the depth to identify root causes. It's intelligence everywhere, from your spreadsheets to their chat feature, even within your product. Omni moves you beyond the dashboard. Don't just take my word for it. Trust teams like Plexity and Synthesia that are already using Omni to deliver intelligence that people trust.
Speaker: Check out them in the show notes or visit omni.co. That's O-M-N-I dot co. Now, back to the show. What happens when a data team stops asking what dashboards do we need and starts asking what data products are we building?
Speaker: That's the exact question at the heart of Emanuela Cote's talk this year at the DBT Summit. Emanuela is the head of enterprise analytics at Compare the Market and today she's joining me on the Stacked Data podcast to talk about her experience at the DBT Summit, key takeaways, and sharing some of the insights from her talk.
Speaker: Emanuela, it is great to have have you on the show. Thanks for joining me. are you doing today? Thank you, Harry. I'm very good. Thanks. so Thanks for inviting me. No, no worries at all. And yeah, as I said in the intro, how how was the the Summit? It's your first time in Vegas at the DBT Summit. um How was it? Yes, it was a great experience. And I went previously at what was called Coles in London, but the Vegas experience was a different experience as a whole. It was really interesting. And I think this year, ah every conference in the data space, we all know, like has this center on technology.
Speaker: Our AI is changing our job, our AI is changing the world and what we do. So was really interesting to see how DBT and Fivetron, because there's been a big merger this year, they are ah facing this and how they're like merging their two missions to help data professionals be on top of the game, basically. Amazing. Yeah, I mean, AI's been the talk of the industry for the war seems like the last forever now. um But yeah, what was the conversations of the of the summit? You mentioned AI, but specifically, be great to get some of the, I suppose, just a feel for what what what you were talking about with people at the conference and some of the specifics. Yeah. I mean, like the the thing that I noticed, there was like a common pattern between all the data professionals. And it's just like realizing that the data foundation are even more important now that they are centering the stack. So anything about semantic layer, context layer, there's been a big, big topic about is analytics engineering becoming context engineering? what ah How does it change the job and the responsibility of what we do? Everything was ah mostly about that. And then of course, there were all the new DBT releases. But I'd say i'd say that the biggest topic is like, how do we make sure that our data foundation, our bulletproof for AI, that we can trust agents to be plugged into our data and return to us a confident answer that's actually the true answer of what we need. So it's more like ensuring and enabling AI to be reliable. Interesting. I think that's so definitely what I've been hearing across the industry is everyone's and everyone I speak to is trying to unlock AI analytics, conversational analytics. We'll see what terminology the the industry lands on.
Speaker: What was some of the announcements that caught your attention from DBT them themselves? Obviously, you mentioned of the the merger with 5Tran. What are some of the announcements that they made specifically around dbt that really caught your attention and you're excited to hear more about? Yeah, there were many. like Let's see if I remember all of them. dbtv2, the new engine. I think instead of, like for for me, telling you all the announcements that everybody can find on dbt website, dbtdocs, it's important to see what I think is really important for us.
Speaker: on the job. So definitely I'd say the open data infrastructure and the fact that Fivetran is ah sort of creating a ah layer for context, so what we call now context layer. And um so together with DBT, now there is like semantic layer, context layer. And in the YAML, something that caught my attention was DBT chart. So it's still in beta by something that gives like end-to-end a platform where data professionals can actually like visualize and create sort of a sort of HTML dashboard like on the on spot.
Speaker: On top of that, of course, the partnership and integration both Anthropic and Chuck GPT, so make it identify as well DBT. There's there's been an announcement dbtwizard. I still have to figure it out how i feel about the desktop version because for a core, former core user, dbtwizard is already available in the CLE, so in the terminal. But now for everyone that is got platform, ah They renamed it many times, dbt cloud, dbt platform, now it's dbt For them, there's going to be actually an agent and embedded in the editor so that you can ask anything. I went to a demo for it and then you can ask anything about your data. If there is any problem on the lineage, any problem on the source, freshness, you can ask anything and it can help you improve like your workflow. But I'd say ah probably the core engine is more powerful. The open data platform, the merge with Fiverr and the give select all the exposure for everything. So we can build semantic layer, but also context layer in dbt and dbt charts.
Speaker: The problem more release, but ah this is what caught my attention. Absolutely. Interesting. So, dbtcharts, reading between the lines, it looks like a they're making a play into the BI space. it Is that reading the the announcements correctly? and I don't know what's going to be. Now it's in beta, so basically you can ah use the YAML file.
Speaker: to create visualization and you can actually like, you know, you run them and you see them there. So i think, I don't know if they want to take over also the BI space, but I think that the main point after all the release and the announcement is that they want to definitely be the leader in the data space, the the company that provides everything from ingestion, streaming, transformation, context, visualization. So it's actually a big big announcement, my opinion. But we'll see because it's still in beta. So I'm really curious to see what we can do with it and if it actually takes over the
Speaker: The BI i tools, because now the BI tools are evolving, no? Like if you think about Omni, Hotspot, all of them, they're all AI powered. So I don't know. They're in a different level of maturity compared to what DBT chart is, but it's definitely going to be very useful for um companies and for data professionals. Yeah, i i think it's a really interesting space because um yeah when Looker first came out, LookML was pretty groundbreaking, right? um Yes. You could do your modeling um within your BI tool, but I think there's been this shift over the last or six years before AI even came onto to the scene, but taking your your your semantic modeling outside of LookML and outside of your BI tool. Just one, to avoid locking from someone like a big corp like Google, but I think also to give you that that flexibility as well. So interesting to see now DBT making potentially a play on that space where they everyone has been building their their new semantic models.
Speaker: Yeah, it's interesting. We'll see. We'll see. We'll see. what Would you recommend the going to the the summit in Vegas? It's obviously a relatively large expense and commute over across the pond, but you've been to Coalesce in London, now that they're at the DVT Summit in Vegas. Why would you recommend it and what would be your advice for anyone heading there in the future? Yeah, I'd say, well, the whole experience, experience as a whole, ah speaking with other data professionals, but also understanding how the space is ah moving as it's working ah so outside of London. Because there were like people from everywhere. I met so many people from Brazil or different places in the US. s so it was interesting to see how like many different industries operate in a different way, but we all have the same problem, if it makes sense. Something I've noticed probably ah my stage of my career was really useful to see how ah the team operates, but I would definitely recommend it to individual contributors. It was ah a great place to be hands-on. There were so many labs.
Speaker: and you can get your own certification including the prize. So if I said the same thing I've told my team, if I could give an advice to anyone that is in analytics engineering engineering space and want to make a case to go, first of all, try to go as a speaker. Like they always open the the call for speakers. i know that a lot of people apply and not everybody gets selected, but if you have like ah generally a nice problem, a problem that you want to talk about, that can be the other professional can relate to and you frame it well,
Speaker: your chances to be selected and speak there are high. So if you go as a speaker, the conference is offered to you. So you just have tried to make the case for expenses to go to Vegas. So it's easier. And yeah, even if not, I think definitely the experience and so on, laboratory certification, meeting other other data professionals, the networking. For leaders, probably, to see what's going on in the space, there are so many vendors. You can do demo different tools, ah different BI tools, but also like all the spaces like the store from DBT that show you If you want to, for example, think it for think about investing movies from core to cloud or to platform, B2, when you want to pay DBT, you are going to see on the spot what actually is going to change. In practice, they're going to show you everything. So I found it a really useful conference, honestly. it wasn't I wouldn't be here and lie to you like it was like mostly sales speeches. It was actually very practical and ah very technical. So like really useful for an analytics engineer, for sure. Amazing. So, yeah, definitely. I imagined a few sales pitches, but it sounds like the networking and the just learning from others. um It sounds like one of our meetups, but on a a death yes several so several magnitudes are larger. So, yeah, it seems like a great place to to to visit, to go, to network.
Speaker: Emmanuel, I'd love to touch a bit on your talk. Obviously, I think your talk will be released on YouTube, so we don't have to go into, to to I suppose, the nuts and bolts of of what you you actually spoke about. But your talk was about moving dashboards to data products. Yes.
Speaker: what What made you actually want to tell that story and share your insights on that journey? Yeah. um So basically, like, you know me well. So I've been working with DBT extensively in different organizations. And what I've noticed that, like, every time was a different...
Speaker: experience, there was a different level of maturity of my role, but also the company. So when I joined Compare the Market to where I work now, i e ah realized that there was a need of like ah changing the operating model and change the culture. So the thinking. So DBT was the tool that enabled us, was not just the transformation. The transformation was in the thinking and how we work and now we what we deliver. But thought maybe this is going to be useful for other people because what I've done, i was like, let me take my experience. What did I learn back in the day at Farfetch when I first adopted DBT? What did I learn then at Dojo? How did we use DBT to, I don't know, apply data mesh, for example, for the magnitude of our data space?
Speaker: And what I'm doing now is actually using the experience to do this shift and at the same time to use this transformation to make us ready for ai So I was like, probably this is going to be really useful. I'm pretty sure then many organizations eventually are going to be in one of these stages, whether it's stage one, stage two, stage three. So if I can help add someone, um then it's something good. ah Also something I realized that, you know, like I read a lot of like books or like a recent podcast, read articles. And sometimes it feels like what you read about how to do things well in the data space, it feels, wow, I'm so behind, you know, like I'm not even close to be there. So I was like, let me do something different. Let me sort of create my my playbook, my guideline, but from somewhere that actually leave the pain point. So there is nothing fancy, not nothing polished. It's my experience. Like,
Speaker: transferred in a sort of like, this is the lesson I've learned. Those are the pain points, I believe. If I could give you a sort of an advice, a tip, that's what I would tell you to do. So no textbook, but someone that actually was there, leave the pain point, broke the dog, broke the he the the main branch and and lived all all through that. And um I think that people really like that, the fact that I wasn't lying and I wasn't telling, a like, you know, a Polish story.
Speaker: was very transparent at the stage where we are now, that the transformation is in flight and nothing is perfect. We're still, like, evaluating how to get better. And I think a lot of people related to that. I think that's, um I mean, it's so relatable, right? I think everyone feels, especially in my position, I talk to so many people and they feel like they're behind. And then actually you when you connect people, they realize they're not so far behind the curve and everyone is still trying to find their own ways. I think that's also, you know, every data environment is different. Every bit of institutional knowledge is is different. So yeah, I think I'm on a big believer, especially in this AI era that you You probably have to build a bit more than ah than that that than before to find the right solution compared to some of the out-of-the-box things that we are seeing. Emmanuel, on this journey that you said, you said you've experienced some of the pain points. um What has been the biggest, I suppose, blocker or part where you've got stuck in this transition that you've experienced? And maybe just a brief insight into maybe how you've overcome that. I think probably it's um the legacy war you we inherited, okay? So like when you want to do a transformation, it's never an ideal situation where everything stops and then you transform, like, you know, you trust you migrate your data platform, for example, or you rebuild your data state. So it's ah the trade-off and like making sure that you can achieve what you want to achieve whilst the business is still up and running. So the business deal like needs report will the business still needs information. you know, like to have that inside. That's one of the pain points that I'm experiencing, but I'd say as well the the buy-in.
Speaker: So I do understand that now every company and every senior leadership wants to be 10 times more faster, like ah more productive, effective. But um it's our duty, it's our job as a data professional, as a data leader, to make sure that they understand that we cannot go 10 times faster, be wrong. you know So it's very important that what we do in the foundation is is done properly, scalable, is reliable. There's so much work that we have to do to ensure that then we can build agents and ah enable conversational analytics. analytics so It's more like getting the right buy-in and make sure that the senior leadership understand that you're there to make sure that everything is done properly and not because you want to stop them. We all want to enable AI or we all want to build analytics agents. I am actually one of those leader that is excited for what's coming. You know, like, of course, it's like, you know, the uncertainty is a little bit scary, but I want to, I'll just say, roll with the punch and like, you know, I want to be part of it.
Speaker: Yes. In 11 years of ah of career, like everything, every now and then thing was changing, like you know from a SQL server to MySQL and then like BigQuery, Snowflake, and then like SQL based BI tool. And then Looker came in. So everything changed along the year. Now it's changing faster. So I want to be on top of that. I want to be part of this transformation. I realized that to be able to do it properly, we need to put in place some guardrails. and build a foundation at scale. So that's why ah what's the most difficult thing probably for for us, like as a head of senior manager, try to make sure that we do think at scale reliably, with the semantic context layer, and the senior leadership understands why we have to do that. And would you say it's communication that is key to um ah that alignment with making sure leadership understands that you, why you're building these foundations and why they're important for the future? Like, how do you deal with that buy-in conversation? Because I think that's something that leaders and teams always, always struggle with. but I don't think there's a golden bullet answer, but yeah, what's your approach? Yeah, don't think there is like one one answer that like ah fits all. Probably it's keeping transparent. So like what I found useful for us is to give regular updates, so where are we, or if there is any blocker, there are any risks, so they see that the work is moving forward. about at the same time speak the language because you know like um when you present to like a C-suite you rarely want to go there with word like pipelines or like you know like you want to make sure that they understand why this is important in a language that i can you know relate and also like probably the secret is like get them to understand that you are on board so you want to do as much as they want But you just need to do those things to enable that. So I don't know, like for me, it's communication, regular updates and trade-off. So if something needs to be delivered quickly, I know that is no ideal. If I think about the work we are doing, we're building all our data states. Ideally, you wouldn't want to build, ah you know, single use cases that going to create sort of so some sort of tech debt. But that's a trade-off that you do with the senior leadership. There is something like, you know, a really urgent, like deal breaker, put your resources in place, switch priorities and deliver that so that that they see that you're you are your player, you know, like you're
Speaker: yeah Yeah, I think it is so I was thinking essentially what you're getting at is that just balancing you have to consistently continue to show so ah value but make them understand that, you know, to build a skyscraper, you need foundations first. So, yes um yeah, really interesting and I think really relevant points, Emanuela. I suppose wrapping up further into sort of a final section, thinking back to the the summit and AI and everything that you you've heard.
Speaker: how How do you think the roles of analytics engineers, I think specifically being synonymous with with DBT, how do you think the role is changing and what positives and negatives are you seeing for the role of an analytics engineer from the summit? Well, definitely I think that there's going to be much higher expectation for the role. So, you know, if I think about the evolution and how the role is born, and it's born because like the old former BI, ah BI analyst or BI engineer, wasn't required to build just report, but was required to build data models. And that's why like the concept of data as a product, like I mean, andkiki DBT enabled that.
Speaker: If I think about it now, it's not just that. It's just like, yes, so as I mentioned, semantic layer, but the context layer. So the analytics engineer, ah it must have as well the sort of analytical thinking and understand the business context is required to to do way more. few years back, I was thinking that like, you know, the analytics engineering space, you could find very different type of analytics engineer professionals, if it makes sense. Someone that was more front-end and more like a business phase, someone more back-end. They wanted to have to be close to the pipelines and to the data ingestion. There is not this distinction anymore. An analytics engineer must sit with the business stakeholder, must understand why we are doing something. The context is what's going to change everything because ah the without the context, AI is going to return a confident answer without giving you the the real business context, the real metrics. Like, you know, like a confident lie is still a lie. That's the point. So um i think the the expectation is higher. Definitely with AI in the game, with all this agent, I am a big believer that you must
Speaker: have code, you must be able to code yourself, otherwise you don't understand. and You know, like if something is wrong or like ah AI is useful to debug the code, but not to write it from scratch. But it's also so true that there is also this trend of like junior, more junior professional that rely a lot on AI for the code. So I'd say when high test, for example, or interview ah new candidates, what I try to assess is the critical thinking. So do you understand why there is a problem in this data? How would you go and investigate that? Do you think about asking the stakeholder what happened there? You know, like it's more business partner that can code and that can write models. So like the role is changing, in my opinion, is going in that direction and it's going to be much more, they're going to have much more responsibility. It's also to be more rewarding, if it makes sense, because instead of being on the on the shade, building the foundation for other data professional surface, like, you know, if you think about the analysts or the data scientists and they they go and present to the business stakeholder, now the analytics engineer is going to have like a lot of more exposure with the business. Amazing. I think that's only a good thing as as well. I think the industry's been going that way for a long time. i think the people that we place are not the people that answer tickets. It's the people that want to be, get into the weeds of the business, be true, as you said, business stakeholders, partners to the business stakeholders, should I should i say. So i suppose to to top it off, Emmanuela, I think
Speaker: For, I suppose, analytics engineers, but i you could wrap wider data professionals into this as well, what would you spend time learning over the next year if you want to become better a data professional? Well, definitely sounds cliche, but ai I don't think that we use it as as its full potential. I try to like explore time-permitting everything, like you know building your own brain, but building your skills, so like you know sort of those things. But the connection between the data and the IA. So definitely like I'm thinking about context layer, no?
Speaker: Context layer is now something that is um everybody talk about it. How you give context to your data. What's the best way to do it Is it an empty file? Is it an YAML file? Shall we write something that automates? I would never say stop learning how to write good code.
Speaker: I think that's the secret of it because like otherwise you don't have the knowledge, the critical thinking to judge what AI is returning you, but the context around it um and go beyond. Go beyond. Before it was just like learning SQL, learning Python. Python learning data modeling technique. Now it's like how this applies to agents, how this applies to wider business context, how this make me be more productive at work and not explore new things. So I would explore a little bit beyond that, try to be bold and think about if AI keeps growing, what my role is going to be in two to three three years time. So how can I be indispensable to the business? You know, not um to make sure that AI is not going to replace what I do. So I need to to bring something on the table, something to the table that is like beyond that. So either the connection between the tools or either and the business context, either I'm the best coder and a spot what AI it does wrong, like something on on this pace.
Speaker: And um probably for this is not specific for analytics engineers, somebody I would tell anyone that is working as an analyst, go beyond that. ah We need to, you need to like power that with statistics knowledge, machine learning has to be, before was like a specific skill just for data scientists, a machine learning engineer. And now we need to be able to do that, everyone in the data space, because Descriptive analytics is the one thing that AI can potentially do. But if you can ask yourself why, if you can do diagnostic analytics, if you can bring something more to the table, then you are indispensable analyst. I couldn't agree more. I think it's um it's about thinking bigger than the one task that you've got and thinking in systems, how to and how do we connect all of these systems with the institutional brain of our of our own own business? And if if you're thinking in that way and enabling that, then you're going to become the one that's indispensable rather than yeah rather than being a ah GPT answered away from your...
Speaker: ah from your job amazing insights emmanuela thank you so much for joining us on the pods and um we'll put a link to the youtube videos for for your talks when they yeah come out as well in the episode and if you want to speak to emmanuela i'm sure she wouldn't mind anyone dropping her reconnection request on on linkedin as well amazing thank you so much for having me harry excellent that's it for another week um everyone thank you and we'll see you again soon Hi everyone, just a quick one from me.
Speaker: If you've enjoyed today's episode, I'd be so grateful if you could hit that follow button or leave us a rating. Even better, pass the show on to a friend who might also get some insight from it. It really helps us grow the community and continue to share amazing conversations.
Speaker: I also wanted to take a minute to talk to you about Cognify. Those of you that don't know, Cognify is the leading recruitment partner for modern data teams. We help some of the world's best organizations scale data and drive real value from the players that they make.
Speaker: you're thinking about building a team or making a hire and you're struggling with talent or just want some insights on the market, then I'd love to jump on a call with you and tell you a bit more. Equally, if you're looking for a job and want to find your next dream role, then reach out to myself or any other Cognitive team. We'd be happy to see if there's anything on our books that we can help you with and give you general advice on the industry.
Speaker: Finally, big thank you to Omni, this season's sponsor. If you'd like to learn more about the AI analytics that Omni can deliver you, then check out the link in the show notes or come speak to me. i can happily point you in the right direction.
Speaker: Again, thanks for listening and look forward seeing you a few weeks time.
