Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
016 Omni - The Next Gen of BI
201 Plays1 year ago
Self-serve BI was one of the trailblazers for the modern data stack. The ability to allow business stakeholders to not only see reports but also start exploring their own data was truly a seismic shift in data.
What is next for BI?
Join me in the latest episode of the Stacked Data podcast as I sit down with Collin Zima, the Co-founder and CEO of Omni, for an insightful discussion on the future of Business Intelligence.
🚀 About Colin:
Collin Zima's journey from an early Google employee to leading data teams at Hoteltonight before eventually becoming the Chief Analytics Officer and VP of Product at Looker gives him a unique perspective on the evolution of BI. Now, as the driving force behind Omni, he's shaping the next generation of BI solutions.
🔍 Dive into Next GenBI with Us:
In this episode, we explore:
The primary purpose and value of BI
Challenges facing the BI sector and strategies to overcome them
Omni's innovative approach and key focus points
The importance of a customer-centric approach in building BI solutions
Future trends and advancements shaping the BI landscape
Tune in as Collin shares his experiences, insights, and advice for organisations navigating the complex world of BI in today's competitive landscape.
Colin's message was clear to me, he and Omni doesn't seek to rewrite the BI script but to refine it!
Recommended
Transcript
Introduction to the Stacked Podcast
00:00:02
Speaker
Hello and welcome to the Stacked podcast brought to you by Cognify, the recruitment partner for modern data teams hosted by me, Harry Golub. Stacked with incredible content from the most influential and successful data teams, interviewing industry experts who share their invaluable journeys, groundbreaking projects, and most importantly, their key learnings. So get ready to join us as we uncover the dynamic world of modern data.
Colin Zimath's Journey in BI
00:00:34
Speaker
Hello, everyone, and welcome to another episode of the Stacked Data Podcast. Today, I'm joined by a very special guest, Colin Zimath, the co-founder and CEO of Omni. Colin was an early Google employee before moving in-house to lead data teams, where he was an early customer of Looker, before jumping into the Looker team itself at very early stages, where he went on to become the chief analytics officer and VP of product.
00:01:01
Speaker
going on an incredible journey working directly with Looker's biggest customers to understand what they wanted from BI. Two years ago, Colin left Looker to start what he sees as the next generation of BI in Omni. Today, we're going to take a deep dive into what's the purpose of BI is, how it drives value, what are some of the biggest challenges facing the sector,
00:01:24
Speaker
And what's next for the next generation of BI? Colin, welcome to the show. Lovely to have you on. How are you doing today? Great. Thanks for having me. No worries at all. So look, Colin, I've given a bit of a brief overview to your career, but it'd be great to get a bit of background from yourself. I'm sure I guess would particularly be interested in what it was like being a part of Google back in 2007. Yeah. So I mean, the brief life story is I came out of college and I went into structured finance.
00:01:50
Speaker
So I was like treating instruction credit instruments for a year, switched over to Google actually at the sort of poking of my co-founder at Omni Jamie. We were roommates together in college. I was a statistician there for four years. Then we started a previous company, ended up selling it to a company called Hotel Tonight.
00:02:06
Speaker
That is where I really got connected on the BI side of the house in earnest, where Hotel Tonight was Looker's fourth customer. And then kind of ever since then, I've been building BI tools. So that's sort of the life story. Google in 2007 was a very different place. I think when I joined, it was a little under 10,000 people. And when Looker got acquired, I believe that Cloud was about 50,000 people.
00:02:28
Speaker
So just the scale of the business has grown so significantly in the last like 15, 20 years. It's been kind of crazy. I mean, still what I would say is I was coming from UBS, which is, you know, like kind of old school finance. And it was very eye opening to come to a place like Google, where it actually felt like you could almost do anything. I mean, like a lot of bigger companies are very hierarchical, which is like
00:02:54
Speaker
your boss is in charge of a thing, the boss's boss is in charge of a thing. And of course Google works like that way too. But it really, it felt actually like a startup, even at 10,000 people. Like it felt like you could go talk to anyone in the company and you could have a really strong idea and you could sort of go start executing on it. And again, like it was 10,000 people. So that wasn't completely true, but at scale, it was kind of unbelievable to see how well Google was operated. And I think that
00:03:20
Speaker
is partially the benefit of just having an amazing business, but also I think that they were very thoughtful about how they built kind of like the organizational structures. And I don't know if it's completely sustained itself to 2024 and how they operate now, but it was an amazing place to sort of learn what tech is like. And obviously I haven't been back to finance, so it was pretty good.
00:03:41
Speaker
I suppose that culture of shipping value quickly, failing fast, that was really built by the sort of original tech companies. So you mentioned obviously there about how you got into BI. I suppose really keen to understand how you first started to focus on BI and what drew you to
Colin's Work at Hotel Tonight
00:03:58
Speaker
the space.
00:03:58
Speaker
Yeah. I mean, this is probably a typical story, but I sort of learned what I would consider BI in Excel. Like obviously in finance, you live in Excel and that was really the first data tool that I ever used. Like in college, I actually work at the same asset manager for all four summers of college and I literally built spreadsheet automation inside Excel. I was like a BBA programmer slash reporting builder in Excel. That was everything I did.
00:04:23
Speaker
And then sort of as I leaned into hotel tonight, I was always sort of like an applied data guy. So at Google, I was a statistician, which meant that I helped sort of do evaluation on the ranking algorithm. It was a lot of kind of like R and data science before it was data science, I guess. And then I got to hotel tonight and it really felt like
00:04:39
Speaker
the thing the business needed the most from me was this combination of being able to take all the data that we had and try to make more intelligent decisions with it. So like a great example of that was that hotels made actually used to in every city rank their hotels by hand.
00:04:55
Speaker
They had a market manager in every city who literally set the order of the hotels in every single city. And it was amazing. And it was like very much the startup, like we will do whatever it takes to get to the next level type thing because the business was already working when we got acquired by them. But it was this really cool opportunity to say like,
00:05:13
Speaker
interesting. We're doing it this way. It turns out that having some background in search and ads, which is where Jamie was, we were able to look at the data that they had and said, I think that we could probably automate this. I think that we could build a ranking algorithm. And almost the concern with ranking was
00:05:30
Speaker
We were concerned about almost losing the spirit of the app. We wanted to be able to merchandise nice hotels and have a diversity of properties. And these are things that conversion rate optimization is not good at. Conversion rate optimization is about showing the cheapest hotels, because generally those are the ones that book the most. So it was this really fun project of getting all the data in order and essentially building this conversion rate optimization engine. But we had to tune it in a way that it expressed the sort of
00:05:59
Speaker
the vibe that we were going for with Hotel Tonight, it had to shove in luxury hotels and things like that. So we had to almost do that combining of these very mathy concepts with product and the business.
The Importance of BI in Decision-Making
00:06:14
Speaker
The human element. Exactly. And I feel like that's where I really started appreciating a lot of the data things, which was
00:06:20
Speaker
Let's just do this kind of stuff. I mean, and even the level one of this project was literally just getting real time data in front of the market managers so they could see what hotels we have, who is listed, who is not listed, who has sold out. It was almost like a.
00:06:36
Speaker
create a little bit of actionability about what the next step is in your market. So like, who do I pick up the phone and call today? It's like, oh, the hotel that gave me rooms the last five days, but didn't give me rooms today. And I think a lot of people on the analytic side jump to these ideas of like, I need to super optimize the business. I need to go build that conversion rate optimization thing.
00:06:56
Speaker
Often the first problem is getting someone like a really reliable slice and dice of a table of data that they can go make decisions on. And that was a lot of sort of what I got to start experiencing at Hotel Tonight. And it was also my first time kind of using the BI tool and implementing a BI tool and all those things. So there's like the tech side of it too. But I sort of think about it as like data is sort of applied business in some sense. It's figure out what everyone's doing and get them a little bit more information so they can make better decisions.
00:07:25
Speaker
Yeah, I mean, that BI is the bridge between where data actually is then conveyed to the business and actioned upon. And it sounds like in that first stages, it was that first steps towards actually translating, not just proper statistical analysis, but adding that element of business product and strategy.
00:07:47
Speaker
Colin, what in your view is the primary purpose of business intelligence? I think it's just to make the company smarter. I think there's, again, there's, I think, tears of all these things.
00:08:01
Speaker
But the first tier of data is making it so that everyone in the business can understand what is going on. It's just be able to see what happened yesterday, what happened last week, what has happened over the last three months. And I think a lot of people are somewhat dismissive of almost the simple things in data.
00:08:18
Speaker
Again, get someone a table of data or be able to look up all of my customers and their closed date and all of my customers and their renewal dates. These are, I think, almost trivialized concepts in BI because it's not interesting or fun, but those are the building blocks of making bigger, better decisions for the company.
00:08:40
Speaker
The most tangible thing a CS person needs to do is renew their customers. So they want to be able to look by region and by date at when those renewals are. Are those customers happy? Yes or no. It's these basics, I think, that are sort of the first level of data. And then from there, it's about unlocking value for the business over time.
00:08:58
Speaker
And some of that is backward spacing value reporting and things like that. A lot of it can be more powerful. It's predictive in the future. And it can point at next steps. But I think of almost the data team as, in some ways, the brain of the business, which is we're trying to figure out what's happening. And we're trying to action people to go do the next thing. And I think that's what all effective data teams are doing.
00:09:22
Speaker
So using the data to force an action, to force a change in behavior, essentially. Yes. And I think to the change point, the changes can be very small. Again, the CF person looking up the list of renewals by region and by happiness level, it's not about
00:09:42
Speaker
conversion rate optimization of who's renewing or anything like that. It's just about either saving them a few hours of their day because they can go pull up those contracts by hand, but also it's about giving them a little bit more context when they go into the conversation. Who is using the product at this company?
00:09:59
Speaker
And kind of have they not logged in for a period of time or are they using it really happily? Like how broadly is it adopted? Are they actually consuming their contract? And these are all sort of like the SAS version of these problems. But a lot of it is just about giving everyone a little bit more context in how they make a decision. Like on the product side, you could
00:10:18
Speaker
entirely build on instinct and say, if people need this, we'll go deliver it to them. Or you can build and watch how people use it, and then tune those decisions. So we just built out this Excel layer, and we have tried to copy a lot of Excel functionality in the app.
00:10:36
Speaker
we haven't copied everything because we have limited bandwidth. We can go see the functions that people try to put in that are unsupported Excel functions. And now we trap them. It is literally just a list of every single function that someone attempts to put in, but we can go see the next five functions that we should go build because we can watch how people are using it and what they're trying to do with it. And
00:10:56
Speaker
Again, these are not crazy, complex concepts. They're a combination of slightly more effective tracking and really simple pipelines that get data in front of users. And I think that's, honestly, that is what data teams need to be doing.
00:11:13
Speaker
Yeah, I think simplicity is always key. I think there's a typical sort of pain point that I do see within the industries when you have these stakeholders that tend to, maybe they have their idea, their instincts, like you said, and then they go and look to find data to back it up rather than the other way around, which is obviously a challenge.
00:11:33
Speaker
I suppose, how would you approach changing that approach? And that's when BI, I think, isn't used in the best possible
Challenges in Building Data-Driven Companies
00:11:40
Speaker
way. Yeah. I mean, the company has to care, first and foremost. You're not going to become a data-driven company if the management team doesn't really care.
00:11:48
Speaker
That's an important sort of overarching theme. I mean, honestly, I think the big thing is just go solve problems that people have. And so, like at Looker, and you know, we've sort of mirrored a lot of the sodomny, it's not about trying to build a data environment for every single user in the company. You're not going to be able to do that effectively for every single org and, you know, unlock enormous value. It's usually, let's go find a person
00:12:12
Speaker
and solve a really tangible problem for them. So again, like the hotel tonight analogy was this was the market management team. They effectively ran the business. It was like, let's go get them the data they need to go make better decisions. And when you do that, then people start sort of demanding data to go make decisions. And you almost like invert the relationship, which is, you know, I was going to doing this by myself. Now it's sort of fed to me. And now I'm sort of looking for it at all times. And so I think a lot of it again is like, get
00:12:41
Speaker
basics in order in certain places and then sort of slowly expand those things out and make sure that it's really well operationalized for those users. So, you know, if the support team needs data to make a better decision, it's probably just some log of their users. And well, you know, what are the last 10 support tickets that they had?
00:13:00
Speaker
Again, that doesn't sound like data analysis, but a support person is going to have a very different conversation with a customer if they can look at the last 10 tickets versus if they can't. They're still going to support that user either way. They're still going to try their best to go help them. They're going to know a lot more if that problem has occurred 10 times or if this person is a frequent person that connects with them. Those are the little data hooks that will start making people more effective. And it's not about starting with a really overbuilt
00:13:29
Speaker
overly architected data system. Like you want to have those things in the back of your mind, but it's really like simple pipeline, get someone data, refine it over time. So I'm a big believer in almost like the lean startup for a data team, which is, you know, do something, iterate on it, do something, iterate on it.
00:13:45
Speaker
Nice. Simplicity is coming through as a clear factor, but BI has been around for decades. Why do you think BI hasn't maybe solved this simplicity problem yet? It's a human problem also. I think all of these things are hard because I think in data especially, you're managing a bunch of different people all at the same time.
00:14:10
Speaker
So I think the hard part about building a data culture or a data product line for a business is the marketing team and the sales team are not necessarily incented to use the same platform. Marketing has data in a vacuum, and the sales team has data in a vacuum. And it can create these contention between orgs. And the data team is one of these teams that actually needs to span the entire business. The sales team has an interface with marketing, but effectively operates on their own.
00:14:38
Speaker
Same with the product team, same with the support team. Again, there's cross-functionality anywhere. But the data team is a service org for the entire business. And that creates these big challenges about actually coordinating all of these different people. The exec team wants things that look really pretty. The support team wants really simple tools. The finance team wants heavy permissions or something like that. I'm making stuff up here. But there's different incentives across all of these users that need to get balanced and coordinated.
00:15:08
Speaker
And the challenge for the data team is that you aren't always as close to the users in those different areas as you could be or you should be. And it's not reasonable to be as close to all of them either. So in many ways, I don't think it's a tools problem really at all. You know, like good tools can always make these problems better.
00:15:28
Speaker
But I think it's this challenge of coordination and incentives. Oftentimes, it feels like when you're on the data team, you work for the executive building reporting when you probably should feel like you're working for all these different orgs, helping them make more money or operate more effectively.
00:15:44
Speaker
And again, balancing across all of those different types of users and things like that is very challenging and kind of like trailing thought on the user side. It's also a type of product that actually every single person in the company uses. And I think it is weird to sort of state that because there aren't that many pieces of software that actually everyone in the company uses. It's things like email and docs, but it's probably the most technical piece of software that every single employee in the company uses.
00:16:11
Speaker
And the challenge there is that you've got lots of different people with lots of different capabilities. You've got Excel people that can munch data in their own little environment, but maybe don't have SQL skills or something like that. You've got data people that are very technical, and you've got engineering folks that maybe don't like tooling but want to write SQL on their own. And then you've got these non-technical users that really care about the presentation side. So you're balancing across all these very different incentives.
00:16:36
Speaker
And that can be very challenging. And then the last part that I would make is there's a risk aversion built into data people. Like we like this sort of sense of complete control over everything. And I think you see this when people put out reporting as a data team, like they really want things to be perfect. And often the business needs to move a little bit faster than the data team is comfortable with.
00:16:58
Speaker
And that also creates this interesting tension where if you put out something that's wrong, it doesn't feel good. But if you wait too long, maybe the business goes another way. So it's really just this push-pull of incentives, I think, across a bunch of different people that makes it so hard. And the data team has to figure out how to really balance across all these different types of users.
00:17:19
Speaker
Yeah, I think that sort of you need that constant feedback to be between the data team and the stakeholders. As you said, if you leave something too long, then they might just not use it. And by then there you get mission drift, things have changed and it becomes irrelevant. But I know many data professionals strive for perfection and sometimes you don't need perfection. It just needs to be 90% there. And that's where the value can come from. And as you said, constant iteration is always the road to improvement.
00:17:49
Speaker
So what are some of the other biggest challenges that you see that are facing the BI sector?
Trends in Business Intelligence
00:17:54
Speaker
And why do you think that is? I mean, for the BI sector in particular, I mean, I think the data always goes through these sort of oscillations between sort of different points of view on how things get done. So like at Looker, we talked about the three waves of BI and it was like highly centralized in this object's Cognos micro strategy. And then you had like the decentralization push from Tableau and Power BI. And then sort of like Looker was a throwback to centralization.
00:18:18
Speaker
And I would say DBT has actually continued that path of centralizing and now pushing the semantic layer and things like that. So I think there is this tension between how much do you let people do things and how centralized do you want things?
00:18:33
Speaker
that has certainly created this balance in the org of trying to figure out what the right point on that is. I think people have gotten more technical over time as well. So I do think the technical tools have improved really significantly underneath everyone.
00:18:49
Speaker
But I think the hard part is that it's gotten so easy to build companies and so many data people, obviously myself included, have started building things that there's a lot more noise in the market about just like the number of products that exist. I remember when Looker was growing up.
00:19:05
Speaker
the landscape was just a lot less crowded. And I'm speaking in the BI layer, but now we also have eight or nine different product verticals that are equally crowded, like data observability and cataloging. There are so many different categories right now that I think trying to sift through what you need in all of those different places can become very difficult as well.
00:19:28
Speaker
And so I think the challenge is it's gotten very easy to build these stacks of a bunch of different great tools. But now I need to coordinate across all these different products and services and my users potentially need to understand them. And I think it's going back to almost like that pragmatism idea. I think people have moved away from more pragmatic
00:19:48
Speaker
stacks towards very heavily built up, best of breed types of things that need to coordinate very heavily. And I think that there will start being backlash, again, back towards centralizing or fewer tools that do no one thing quite as well, but more things for more users so that it's a little bit simpler to operate and use.
00:20:13
Speaker
But I mean, I think these themes of like centralization and decentralization will just exist forever because there's good reasons for both, you know, data team control. Yeah, it's the balancing act, right? Exactly. From one side to the other. And, you know, I think it also comes down to your own use case. You know, when you've got highly sensitive data, you don't want everyone to be able to access that. And then when you're on the flip side, you know, if you want your team to be more explorative in other areas, then, you know, you can bring down some of them guardrails.
00:20:43
Speaker
I really agree with what you said about the growing of the modern data stack. Tristan, the founder of DBT, I posted just the other day actually, didn't he, about the death of the modern day stack or how that's playing out with just this increase in tooling that's on the market.
00:20:58
Speaker
I know from an in-house perspective, it can be extremely time consuming just dealing with renewals. And if you've got five or six different tools on them, you can spend the whole year in negotiations with vendors. So I think it's needed, the specialization in tools. But yeah, I think as we continue, I think there will be that consolidation as well.
00:21:20
Speaker
How do you see these challenges evolving in the near future, Colin? And what strategies could you pass on to organizations to help them address these challenges and equally avoid some of the common pitfalls?
00:21:34
Speaker
Yeah, I mean, obviously one of our bets is on consolidation. Like I think in the, like part of the Omni vision is like, let's actually make something that can be decentralized and centralized at the same time. So give someone a Tableau, Excel mode experience and give them a Looker experience all at the same time. So I think that's part of it. I do think that we're continuing to like, if you look at the big underlying themes around sort of everything that's happening in data,
00:21:59
Speaker
The thing that I would remind people, even though the modern data stack is dead stuff, is that, again, 10 years ago, there was no Snowflake Redshift BigQuery. Setting up the foundation of the data stack was very, very difficult. At Hotel Tonight, we were connected for a little while to our production MySQL. I took down the app a few times actually doing data analytics. So obviously, that's not recommended. But now the idea that a two-person company can spin up a five-trans Snowflake
00:22:29
Speaker
dbt omni stack literally on their own in, you know, a day that is actually pretty unbelievable. So I think that the ability to actually get data into a structured place in a very straightforward way and set up sort of reliable pipelines has been sort of incredible. So I do think that
00:22:51
Speaker
like the things that were sort of best in class have become completely normal now. And I think that's a really important thing. I think the thing that I am most curious about is that one theme that I have seen is that with the sort of rise of DBT, we have, I think in some ways moved backwards to an idea of much more OLAPI style reporting.
00:23:12
Speaker
which is people building these very unidirectional data pipelines, which is I dump raw data, I clean the raw data, I make reporting tables, I stick a BI tool on top of it. And again, I think it starts to merge with this idea of data analysts that want heavy control over the pipeline from end to end, and you don't touch a table unless it's a reporting table. But I think that actually creates really bad incentives for the business about how fast we can move with data.
00:23:42
Speaker
because we need the data team to prepare the thing every single time they want to do something. And I do think the ideal structure for this sort of thing is you will have tables that look like that, where the data team has to prepare everything before someone can consume something. And it needs to be heavily tested and version controlled and data as software or whatever.
00:24:04
Speaker
I think for every one of those assets, there probably need to be many assets that are not treated that way because the speed and the time to get value out of that pipeline is so much slower than just giving people data, letting them do things, and then optimizing underneath it.
00:24:23
Speaker
I think that we've sort of, we've oscillated very far towards the side of centralization in now the dbt layer, like even below looker. And I think that there's going to be frustration from businesses about sort of how quickly they can move in this world. And so I think that's probably the backlash that I'm expecting to happen here. Like semantic layers or even the idea of a semantic layer has gotten
00:24:47
Speaker
I think in some ways a little bit overplayed. And I say this as someone who's building a semantic layer to their tool. So we're in this game also. But I think that people have lost the underlying reasons that you do these sorts of things for the elegance of this stack that every single piece of data is self-explanatory and things like that. The mitigating thing I would say is that all the LLMs and AI tools are going to need some level of semantics to do things.
00:25:16
Speaker
So I do think that there will be things that explain the business in data terms. We can call it a semantic layer for AI, but I think that we need to move away from this sort of OLAP style. I build a reporting table for every single use case. And if someone wants a new column, I'm back in the ETL cycle, adding a new column to the reporting table. I think people are going to slowly consolidate tools and be a little bit more flexible in terms of how they build.
00:25:43
Speaker
Yeah, I think especially given the last year that we've had in the economic situation, there's been a huge amount of scrutiny put on data teams as to what value are they driving. And I think it comes back to, as you've said, that ability to shift value quickly is so important. Time to insight is an important metric for many data teams and insights. So yeah, adding that layer of complexity is a trade-off that I think you have to do for each use case, but it has to be one that's carefully considered.
00:26:11
Speaker
Colin, look, we've spoken a lot, I suppose, about the industry as a whole, but I'm really keen to understand a bit more about Omni's approach. What sets Omni apart from traditional BI and how does it address some of the challenges that we've mentioned?
00:26:26
Speaker
I always talk about sort of like our cheat codes and superpowers building a company. And I think the backdrop to Omni is obviously, you know, I was at Looker for eight years. Jamie was there for five. Our first seven employees. So Chris was the CTO at Stitch. And then we had four folks that were at Looker for, I think combined 20 or 25 years or something like that.
00:26:45
Speaker
And right now we are, I think, 30 or 31 people. I believe it's 21 or 22 from Looker. Five folks came from the stitch side of the house. So it's a lot of us that have been doing it for a very long time. And so I think our embedded advantage is we've actually touched a lot of these problems. And it's enabled us to have a much stronger opinion about what a slightly better way, like the things that we did really well and the things that we wish we did a little bit differently. And I like to think that we're just building a tool
00:27:13
Speaker
that tried to borrow the very best ideas that we felt like we experienced, but combined them with the things that we just felt like we were doing completely wrong. And so to the point that I was making earlier, I started using Looker when it was really a command line tool for building a data model. You really actually had to code the semantic model in the command line, which was terrible. But it was so good at what it did in terms of building out this model that people could
00:27:41
Speaker
not reproduce the same work over and over again. It started with these light foundational pieces where you were not over building a data model, but you weren't rewriting joints every single time. You weren't rewriting metrics every single time. It had business encapsulation. Personally, I love that side of the house of centralized analytics. I think most things should operate in a very centralized way.
00:28:04
Speaker
The flip side of that is I sort of mentioned this already, but I was a big Excel guy and I found myself literally just writing a ton of SQL. And the reason is there's this sort of decision that you need to make every single time that you start analysis, which is like, am I coming back to this thing?
00:28:20
Speaker
And if I'm coming back, I want to go model it and do it right. And then I can go do my analysis. And if I'm not, I just want to go get an answer and move on. So in that Excel example where we have to go look up functions that people are using, this is not like a complex data model. It's like we've got logs somewhere. I literally just want to see a list of the functions that are getting written. I don't care if they're perfectly right or if I'm missing some or if I'm including internal data and not
00:28:47
Speaker
not filtering it out. I just want to go see some functions that are getting used and make a slightly better product decision. And it truly felt like there were no tools that were actually trying to bridge these things. Everything was built as either DIY, do it yourself. And maybe it's a very technical version of that, like a SQL runner or a notebook. Maybe it's a very non-technical version of that, like in Excel or a Tableau point and click visualization. But there was nothing that actually wanted to do both of these things.
00:29:17
Speaker
And kind of at our core, the thing that we're trying to do the most tangibly is make it so that people can move really quickly and the tool can sort of float into the background and let you just operate. And so, like, that's Excel stuff. That's drag and drop visualization. It's SQL. But
00:29:33
Speaker
you can reuse those pieces. And you can actually put, you know, reuse into an organization. And so, like, we are trying to be looker and tableau and mode all at the same time. That is the sort of, like, big ambitious thing that we're doing. And the nice thing is that we
00:29:49
Speaker
we decided, this is literally how I pitched the company one day after I left Google, was we're going to do all these things at the same time. And it freed us to think about, OK, what does an environment look like where end users can write an Excel-style column that is first name plus last name with a space in between? And then a data team can pick that up and say, that's modelable. I'm going to push it down into a database and maybe even push it all the way back down into DBT or something like that.
00:30:18
Speaker
And so, like, I think that we have this very unique view over what data modeling actually looks like that sort of appreciates, you know, modern data modeling. Again, don't like can overuse the word modeling. Sorry, modern. But like this idea of what data modeling actually looks like in sort of like a tech forward environment, but gets out of your way when you need to move really, really quickly. And so that's again, like,
00:30:43
Speaker
We just are trying to be a tool where someone can actually comfortably, I say cut corners, but I mean that in a very positive sense. Save time, give them their own agility and it's getting that time to insight.
00:30:59
Speaker
Exactly. Throwaway reporting is good sometimes. And I know that almost sounds weird for a looker or a data person to say. But it keeps the core stronger, actually, when you are throwing away work. And you're doing it anyway. So it's really just embracing this fluidity between moving fast and being really robust and solid.
00:31:22
Speaker
Nice. And I like obviously what you meant about the centralization as well, where it clearly obviously gives them this agility, but also then saves them time and not having to rebuild what's already been done. Because that is one of the biggest wastes of time. Yeah. So Colin, obviously, there's been this big influence from Looker. You've got a lot of the Looker team. What are your key differentiators, would you say, between Omni and Looker and some of the other tools on the market? What makes you special?
00:31:52
Speaker
So I mean, again, I do really think that we're the only tool that actually allows for both of these things. We're the only tool that embraces a user needs to write SQL into a one-off thing, and a data model is good. And so I think that is at our core what the biggest difference is. I think underneath that, there's a few different takes that we have on how people do data. So a simple example is,
00:32:18
Speaker
Again, in this decentralized, centralized world, you have the idea of in-database analytics versus in-memory analytics. To oversimplify it like Tableau, at least historically, suck data out of your database, put it in the BI tool, and then you slice and dice it. Advantage, it's really fast. It's very interactive. The BI tool knows exactly what everything looks like. Disadvantage is that it's not real-time. It's not taking advantage of your data warehouse.
00:32:42
Speaker
Looker was built around in database analytics. Everything is pushed down to SQL. The database does everything. It's magic how it happens, but it's all in database.
00:32:52
Speaker
And again, when we reflected on what users actually want, what they want is the performance of in-memory when they're slicing and dicing things, but they want to always have the entire database there. And so this is another example where we've rebuilt it such that if I write a query and I don't have the answer, we're in a database. If I already have the data set that I need to give you that answer, we're an in-memory tool. So if I ask for the users table and I say select star from users and I get the whole table,
00:33:21
Speaker
Now we're an in-memory tool for the users table. If a new row comes in, I can go bust the cache and go back down to the users table. And so you can fluidly float in between a real-time slice and dice dashboard or a filter that is instantaneous. And I can write a 1 trillion row query or whatever down into the database and let BigQuery and Snowflake and ClickHouse and Databricks or whatever figure it out.
00:33:45
Speaker
And so I think there's all these pieces that we've rebuilt to enable these sorts of concepts where we sort of said the extremes on both sides are actually problematic and what users actually want are the best of these different sides. And so I think that's sort of the
00:34:03
Speaker
the vision that we're carrying for the product is, like, don't pick sides. Just do both and make them actually fit well together. So, like, the other, obviously, examples, I've talked about the Excel stuff a lot, but sometimes you want to write SQL to a database to pull data out. But, like, I've never wanted to write SQL to do a period over period thing. Like, no one wants to write window functions to do those things.
00:34:25
Speaker
And other tools do that way better. In Excel, when you type in equals and then you click and then you click on the row below and then you press enter, that is the way to write a period over period change. And so again, we'll let you write SQL and then we'll let you write column one, cell one divided by column one, cell two. And I think the idea is that if you are actually thinking cohesively about how all these things fit together,
00:34:48
Speaker
we can give the Excel person an amazing experience, and the SQL person an amazing experience, and the data modeler an amazing experience, and we can actually create some fluidity between these things.
00:34:58
Speaker
As you said before, it seems like it's the best of each of these different areas and trying to keep as many people within the organization happy and giving them the ability to utilizing data. Colin, you're obviously a big part of building the customer support Looker, and Looker was very well known for its customer support arm.
00:35:20
Speaker
What did you learn from working so closely with some of Looker's biggest customers, and how are you bringing that forward into Omni?
Omni's Customer Support and BI Approaches
00:35:29
Speaker
Yeah, I mean, I probably should have put that in the last section around differentiation and what we're doing differently. I mean, I think you sort of nailed it, which is, I think what we realized, and actually I learned this before at Hotel Tonight, because Hotel Tonight also had just incredible customer support.
00:35:45
Speaker
which is I think people think about customer support as a cogs, as an expense of doing business. And I think if you invert it and think about it as marketing or customer experience, everyone, the company benefits and the user benefits really significantly.
00:36:03
Speaker
And so this idea of just super, super high touch customer support is almost like a selfish one in that we can make sure that people have an incredible experience as they're touching the tool. And by doing that, we can continue to make the tool better, but we can also create advocates in our customer base that love working with us. And I've talked about this before, but I think the big difference is the time of the feedback loop.
00:36:28
Speaker
When we are talking right now, the reason that we can have a good conversation is because the latency is close to zero.
00:36:35
Speaker
We are, you talk, I give you an answer, I give you an answer, you can respond to it and talk back. That creates this sort of interactivity that lets us problem solve in real time on a problem. And if the user is forced to wait, they actually end up out of sort of like the flow state of what they are doing and often just kind of can't even do the task or it might take an order of magnitude longer to do the thing. So if we have to wait even like 15 seconds or 15 minutes in between our conversation,
00:37:03
Speaker
It can't be interactive in the same way. I need to almost find different work that I am doing. And so I think one of the things that we discovered is this idea of answering things in truly real time with your customer can get them unblocked and effective in a way that is an order of magnitude better for the user than a lower touch customer support model. And so I think that's the support side of it is just maintain really high touch and just obsess over the experience. I think the other side of it that's really interesting
00:37:33
Speaker
And I would say this to any SaaS company out there is when you make sure that your customers are successful, it helps the business in a very significant way. And I think even at Looker, we often would pause and we would say, we're not going to help you with SQL. That's your problem. Those are your business problems. We'll help you with tool problems. I think we've even gone a step further, which is
00:37:56
Speaker
If I could help get you coffee and that would make you more effective, we're going to do things like that. And the reason is because it enables everything around the tool being effective. So I just logged into 5tran today and I realized that I have a couple of customers' credentials to 5tran because I was helping them set up ETL pipelines when we got started.
00:38:18
Speaker
And I think these are the kinds of things that if you can do it as a company, and obviously like these things need to be economically scalable to operate, but you can create such a better customer experience. Like where in that example, if we couldn't have gotten their data into a really effective place into a database, it wouldn't have enabled Omni to help them.
00:38:39
Speaker
And so just kind of finding ways to be effective for customers. And I think this is true for every type of employee in every context, which is like you have to go well beyond what your actual job is to make people successful. And when you do that is when people like love working with you, but also like
00:38:58
Speaker
it creates opportunity for you. So we've had a couple of trial customers that before they purchased were recommending our product. And that's an amazing place to be. And honestly, it's probably less about the product than it is about the customer support and experience.
00:39:13
Speaker
I love that. I think it's something that, as you said, everyone can take into their lives. I always speak to people who sometimes, you know, they want to get to the next level in their career and when asked, you know, well, what are you doing that's already at that next level? And they're not able to articulate that. You need to be going above and beyond to be able to get that behind, to be able to prove that you are able of helping and supporting. So yeah, and obviously,
00:39:39
Speaker
make such a closer tie. And that goes back to your piece at the beginning. Your customers are the ones, they're going to be feeding you information about how to improve and how they're utilizing the tool as well. So it helps, I suppose, you guys as well for your product roadmap as well, indirectly. So, Colin, where do you see the future of business intelligence heading in the next five, 10 years? We're a race that always seems to be looking ahead. So yeah, where do you see us going?
00:40:07
Speaker
It's a really good question because, I mean, obviously we see a lot of omni in that world, but it's like, I mean, I think the next things that people are at least evaluating are really obvious actually, which is like, you know, every day there's a new AI for SQL company and it feels like natural language is sort of like becoming a popular concept for people to consider.
00:40:27
Speaker
But I think like putting that aside for a second, I still think the constant tension that we're going to have as sort of an industry is this sort of bounce back and forth between centralization and decentralization. Like you've got now DBT, but all these BI tools and semantic layers and things like that.
00:40:45
Speaker
are trying to build this really rigid foundational layer for people. And then you have, at the same time, probably more Excel in the cloud tools than I've ever seen before. More people trying to directly compete with Excel in the BI layer. And the reason this keeps happening is because both problems will exist forever.
00:41:06
Speaker
people want to move really quickly and cut corners and have the tool get out of the way and just answer their questions. And then we need data observability and these foundational concepts and software engineers for data people. And I think the battle that will continue to play out
00:41:25
Speaker
indefinitely is sort of like which train of thought is winning at a given point in time and how does it link up to the sort of other side of the world. And so like it certainly feels like we're in a place where more and more people are centralizing more and more things downward into like SQL transformation layers and things like that.
00:41:43
Speaker
But there's going to come a time where the business is like, this is too slow. And shadow IT is going to build on top of it. And people are going to build these turbo flexible tools that completely escape the data schemas. And then we're going to be back on the other side of the house. So it's a complete non-answer. But again, the AI for SQL stuff, if you just hook up an LLM to your database,
00:42:05
Speaker
maybe if you feed it through a data model, you're centralized, but that thing is going to do what it does. And so again, I think we're going to have a new battle for what is ground truth and who owns it and how fast can it change. And hopefully we can settle in some middle place, but I think otherwise there's going to be this tug of war. Keep going.
00:42:24
Speaker
Yeah, I think the AI in that natural language is something that everyone's speaking about. Obviously, it hopefully puts the power into stakeholders, but I think there's many problems and challenges that are again going to come with that, which I think we'll only see unfold as we move into the future. But are there any other specific technologies, advancements, or trends that you believe will significantly
00:42:47
Speaker
impact the future of BI. AI obviously being a big one, but there's I suppose the headless BI concepts and yeah, can you get your thoughts on what could influence BI in the future?
00:42:58
Speaker
Yeah, I mean, I think the headless stuff is still a ways out. I mean, obviously I've sort of said this, but I think it's like a little too overbuilt in terms of like actually setting that up reasonably. I mean, the thing that we've dreamt about since the beginning of Looker that I'm still hopeful will come to pass is kind of the idea of more and more sort of things in the data warehouse.
00:43:19
Speaker
So this idea that suddenly all of your SaaS tools could be perfectly mirrored in Snowflake, BigQuery, Redshift, Databricks, whatever, just like you snap your fingers and all of your vendor's data is just in the data warehouse. It creates this very interesting opportunity for Read-Write in the BI layer. So the challenge is always, if I'm a BI tool on Salesforce's data, if I can update a record, I need to update it in Snowflake and in Salesforce.
00:43:47
Speaker
And obviously, you've got all these reverse ETL tools that are doing things that are very similar to this. But for a BI context, it's just impossible to create a reasonable app with two sources of truth. In a world where Salesforce and Snowflake are the same product, have the same backend, now every BI use case can become rewrite.
00:44:07
Speaker
And again, this is not happening yet. But in a world where you could actually bring down the barriers for data movement to near zero, I think we have some very, very interesting opportunities to turn BI apps from just like pure read only to actual applications where people can make
00:44:25
Speaker
changes and things like that. So I think that's one of the big things. I think the other thing that I'm seeing that has kind of been surprising a little bit is just embedded analytics everywhere. It seems like every tool is putting analytics into it now, like B2C tools and B2B tools. And where at Looker, that was still like maybe 30% or 40% of our business. Now almost every single customer that we talk to when they're buying internal is also thinking about like a lightweight reporting app.
00:44:50
Speaker
And I think that's going to continue where you'll have your analytics tools omnipresent in every single piece of software that you're using. I think that we're going to continue down that path as well.
00:45:00
Speaker
Yeah, I would definitely agree with that from what we're seeing. In fact, there are a couple of the upcoming episodes on the podcast around sort of that data as a product and monetizing data and how you go about doing that is the conversations that I'm having increasingly more because now I think there's much more maturity in data internally. It's now how can we put that power in the hands of either external stakeholders or customers who can start leveraging that. But yeah, that's a whole other conversation.
00:45:30
Speaker
Colin, it's been an absolute pleasure to have you on. You've been really insightful as to what are some of the biggest challenges facing BI and it's great to hear about some of the work you guys are doing to address that on me. But before you go, have you got any final thoughts or advice for organisations looking to leverage BI effectively in today's landscape?
00:45:51
Speaker
I think my biggest one is really try to make yourself uncomfortable as the data team to move as fast as you possibly can. I think every single person that is a full-time data person's instinct is probably to be a lot more conservative than
00:46:10
Speaker
they could be to be as effective as they possibly could. And so one of the things that we think about when we build Omni is just like speed is one of the most important features in the app. And you're not doing anything specific other than just making sure that you care enormously about speed. And so that's probably the biggest piece of advice that I'd give data teams is like,
00:46:31
Speaker
make sure that in trying to make things perfect, you're not going so slow that the organization is just moving past you and that speed is probably the most important thing on par with quality.
00:46:44
Speaker
Amazing. Don't become an afterthought. Be there, delivering value as quickly as possible. That's great advice. Well, Colin, it's been an absolute pleasure. Thank you for coming on the pod and sharing your insights. Yeah. Best of luck with Omni and what's to come. If anyone's enjoyed hearing Colin, give him a connection request on LinkedIn and jump onto the Omni page. There's lots to uncover and one of their team will be happy to, happy to speak with yourself. Cheers, Colin. Thank you for having me.
00:47:15
Speaker
Well, that's it for this week. Thank you so, so much for tuning in. I really hope you've learned something. I know I have. The Stack podcast aims to share real journeys and lessons that empower you and the entire community. Together, we aim to unlock new perspectives and overcome challenges in the ever evolving landscape of modern data.
00:47:36
Speaker
Today's episode was brought to you by Cognify, the recruitment partner for modern data teams. If you've enjoyed today's episode, hit that follow button to stay updated with our latest releases. More importantly, if you believe this episode could benefit someone you know, please share it with them. We're always on the lookout for new guests who have inspiring stories and valuable lessons to share with our community.
00:47:59
Speaker
If you or someone you know fits that pill, please don't hesitate to reach out. I've been Harry Gollop from Cognify, your host and guide on this data-driven journey. Until next time, over and out.