Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
How Apache Pinot Achieves 200,000 Queries per Second (with Tim Berglund)
2k Plays9 months ago
The likes of LinkedIn and Uber use Pinot to power some astonishingly high-scale queries against realtime data. The numbers alone would make an impressive case-study. But behind the headline lies a fascinating set of architectural decisions and constraints to get there. So how does Pinot work? How does it process queries? How are the various roles split across a cluster? And equally important - what does it *not* try to achieve.
Joining me to go through the nuts and bolts of how Pinot handles SQL queries is Tim Berglund, veteran technology explainer of the realtime-data world. He takes us through Pinot step-by-step, covering the roles of brokers, servers, controllers and minions as we build up the picture of a query engine that's interesting in theory and massively performant in practice.
–
Apache Pinot: https://pinot.apache.org/
Apache Pinot Docs: https://docs.pinot.apache.org/
StarTree: https://startree.ai/
Event Driven Design episode with Bobby Calderwood: https://youtu.be/V7vhSHqMxus
Tim on Twitter: https://twitter.com/tlberglund
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Kris on Twitter: https://twitter.com/krisajenkins
–
#podcast #softwaredevelopment #apachepinot #database #dataengineering #sql
Recommended
Transcript
Scaling Databases with Concurrency Issues
00:00:00
Speaker
Orders of magnitude are a real testing ground for architectures. Every time your demands on a system go up by a power of 10, I think the game changes and gets more interesting. So this week we're going to think this through in the realms of databases. One query a second. That's easy mode. Even an analogue system like a podcast host can manage one question per second.
00:00:25
Speaker
Ten concurrent queries a second. We've just lost Microsoft Access. Last time I checked, I think it had a single global lock system. So no concurrency, farewell access, not going to miss you too much, I must admit. But pretty much any other database should be comfortable in the tens to hundreds range for concurrent queries. By the time we get up to a thousand,
00:00:50
Speaker
the game begins to change. Some relational databases will be happy with that. Some are going to start to push you towards connection pools, caching, read-only replicas. And that's completely fair enough. They have a lot of work to do, which isn't just to do with querying. But I think above the thousands range, we are probably leaving a lot of pre-internet architectures behind.
00:01:15
Speaker
Okay, next level up, 10,000 queries a second. That one might even tax your operating system. Depends, of course, but you're probably in the realm of tweaking kernel parameters, definitely load balancer parameters. At 10,000, we're maybe even thinking about clustering multiple machines, and that is a game changer.
00:01:37
Speaker
But even with that, I reckon, experienced but naive me, I reckon I could build a system that handles 10,000 concurrent queries a second, provided I get to choose the kinds of queries we support. And I know that's a big caveat.
00:01:54
Speaker
Next level up, I have got to call it quits. I'm lost at 100,000 queries a second. I don't know how you'd architect that. I really don't. So it's time to call in an expert, because I know it's possible. I know they manage that kind of scale at places like LinkedIn and Uber with a little something called Apache Pino.
Introduction to Apache Pino
00:02:15
Speaker
So who do I know who's a Pino expert? Tim Berglund. And he's about to do a wonderful job of taking us through the architecture of Pino and the constraints that go into building a query platform that gets a good balance of flexible arbitrary queries and a really high performance sweet spot.
00:02:35
Speaker
I'm intrigued by that, I want to know how that's built, and Tim is not afraid to go down into the guts and explain it. And as a bonus this week, Tim has a wonderful voice for radio. So this week, the name Developer Voices lands particularly sweetly. Let's hear from him. I'm your host, Chris Jenkins, this is Developer Voices, and today's voice is Tim Berglund.
00:03:10
Speaker
Joining me today is Tim Berglund. Tim, how are you? Chris, doing great. As my grandpa used to say, any better and I'd be twins. It's good to see you. I've not heard that one before. I've been dreaming of you. No, no, your grandfather must have been a creative man, run to the family. Definitely was. So the reason I've got you in and I'm going to back up. I've almost literally just got off a call with a friend of mine who's kind of a junior developer looking to get into the larger world.
00:03:39
Speaker
And he asked me why there are different kinds of database, and particularly why there are transactional analytics databases. I gave him my best answer, but it wasn't recorded. I'm not on record. You see where this is going? Go on record. Why? Why do we have different trans databases? I haven't really thought about that one. It is an excellent foundational question, and it's good to remind ourselves of some of the whys.
00:04:09
Speaker
I've just been doing a lot of history of tech reading and interviewing and assembling and kind of looking through that from a historical perspective, which we needn't entirely get into. But computers, digital computers started out as batch analytics machines. So at first, you had a bunch of stuff on punch cards. You fed it in and you processed the data. They were doing analysis.
00:04:37
Speaker
you hook up a terminal, they get a little better, a little faster, a little cheaper, you hook up a terminal and you can actually interact. You can write an application where now some knowledge worker can do things and as it were, input transactions, do crud over entities. So now there's this idea of a database that's changing and yet you still need to find the story in the data
00:05:06
Speaker
Fast forward a few decades, it's just kind of allied that for purposes of brevity. I think it comes down to this. When you're in transactional mode, generally speaking, as a first order approximation, you're usually worried about a thing. You're making a thing, you're trying to select a thing by an ID, maybe to read it, maybe to read it and change it and put it back. But there's usually the one thing.
00:05:34
Speaker
that you're optimized for. And in analytics, you're usually looking at many things. You want to find, you know, I get into this, ideally not read through the whole database, but select in some intelligent way the things you're interested in, get some measurement, and run a reducing function over that measurement.
Transactional vs Analytical Databases
00:05:55
Speaker
Basically, you know that kind of filter aggregate or filter filter filter aggregate thing is sort of the bread and butter of analytics and so you're concerned with these things so transactional world you have this thing analytics world you have these things and The the physical world is being what it is
00:06:14
Speaker
with the limitations of just the way mass works and electricity works. You can only do things so fast. It's probably impossible to build one system that's optimized for both things. If you look, just think about caching.
00:06:34
Speaker
Okay. Just think about caching for a minute. Caching is going to be your friend because you're doing IO and there's various tiers of ways of storing things and you're going to want to cache stuff. In the transactional case, you're reading and writing a certain number of IDs, these things, and there's probably a small set of your overall number of
00:06:57
Speaker
objects, documents in your database that you care about, that is amenable to caching, right? That works well. You've got this power law distribution probably of your stuff, and you're doing this. Well, now layer on analytic queries on top of that, and that distribution is at least going to have a fatter tail. You're now scanning all these things, maybe all the things sometimes, and while that's not
00:07:23
Speaker
amenable to caching. You don't get to do both of those at the same time. So you're not going to be able to build one system that does both well. That's how I think about it. I think there's a lot of ways you could tackle this. But this is the joy of running this podcast. I get to see different people's ways of thinking. I can hear someone in my head arguing that they have got a system already that's optimized for both
00:07:49
Speaker
But your definition of optimized is a lot more focused, right? I mean, I can take an Oracle database, I think, and happily get 200 parallel connections. Yes. And I believe you wouldn't be impressed by the number 200. Well, it depends what you're doing. There's the old departmental database of the 90s, that paradigm.
00:08:18
Speaker
moved into the world of the web at some scale, you know, that might be great. If you're looking at a broadly scaled smartphone app or a website with, you know, user population in the hundreds of thousands or millions and a lot of concurrent users and serving queries, you know, live powering features in that application, then well, that's a horse of a different color.
00:08:45
Speaker
still really achievable in the transactional world with kind of the received toolset. Like we know how to do that. In the analytic world, that's a very different thing, right? Nobody until recent, very recent years has thought, oh yeah, let's do that analytic filter and aggregate group and aggregate kind of thing in the user interaction loop. That's a little bit of a leading edge kind of thing.
00:09:09
Speaker
Yeah, yeah, at that kind of scale. So coming at this from a solid background of relational databases and how they query, how do you solve that problem? How do you go from the system that will happily do 200 parallel queries happily enough to hundreds of thousands? Yeah. Well, if it's OK, I'll take the
00:09:39
Speaker
the analytic end of that. I mean, there's a lot of answers to how to do this on both sides of that divide. And there's lots of folks who kind of have solutions for the transactional side of things.
Architectures for Real-Time Analytics
00:09:56
Speaker
You've got the Cassandra kind of approach. You've got various ways of scaling, a Postgres kind of thing. You've got CockroachDB.
00:10:04
Speaker
you know, all those on the transactional side. On the analytics side, you kind of have to engineer a system with those requirements in mind. You have to say, you know what, what we want to do, and by the way, we need to back up a step and say, you said coming at this from a relational database background, you know, the tradition of analytics
00:10:30
Speaker
sort of pre-big data in the span of careers of people of a certain age, you know, kind of going back to the 90s. All that stuff happened on relational databases, traditionally, right? It was fine. That was the technology. I mean, you built schemas that didn't look anything like the third normal form you learned in college, but relational databases can do these things.
00:10:51
Speaker
But if you're going to go to this kind of scale, it's a different toolset. If you're going to say, the architecture has to change because you've got these non-functional requirements that present themselves of you want orders of magnitude, literally maybe 100, 1,000 times
00:11:11
Speaker
more than those two concurrent queries to get kind of radically scaled concurrency, a freshness requirement, a sort of a streaming era, real time data era freshness, where an event happens and it takes a couple of three seconds to make it through a streaming pipeline. And then it needs to show up in results, right? It doesn't get to wait around. There's no batch job.
00:11:38
Speaker
And latency, again, if this is going to power not, again, the origin of all this is kind of printed reports, right?
00:11:47
Speaker
originally printed on green bar, then printed on laser printers. That all turned into dashboards in a browser tab. And the typical dashboard is just a faster report. And if the data is 15 minutes old, it's probably fine. That's not a big deal. If it takes a minute to refresh something, that's great. Nobody's going to get excited. But now if it's a feature in a user interface, I tap on a thing and a query happens. I'm not going to wait.
00:12:14
Speaker
Right. That needs. That's got 100 milliseconds. Users don't wait that long. I certainly don't. It's not a positive character trait, but notorious indication for things like that. Even people like in marketing who want those reports or analysts in a company, they will wait 15 minutes because they're being paid to. They'd rather.
00:12:34
Speaker
Exactly. But users making choices of things, people like me. Yeah, that's make or break. That's make or break. So yeah, you get this concurrency latency freshness triad of nonfunctional requirements. And so you have to start from the beginning.
00:12:55
Speaker
I think the fundamental kind of rate limiting thing that goes on inside an analytic database, an all app database is scanning,
Pino's Architecture and Query Optimization
00:13:07
Speaker
right? You're getting to where there is some metric that you're trying to aggregate.
00:13:14
Speaker
Yeah, you know, you filtered and maybe you're grouping by some other thing, but still you got to you got to scan values of some metric. That's the hot spot. And so you want to. There are just two things you can do. One is scan less and the other is scan faster. And I hope that that brilliant insight. Oh, Tim, thank you for explaining.
00:13:42
Speaker
Yeah, I'll go with that so far. Today's episode of Developer Voices is brought to you by... Right, so you scan less, you scan faster. I'll give you a third one. You can split the scan over multiple machines. Exactly, and that is a corollary to scanning faster. It shouldn't be a given, it should be stated, but any system
00:14:10
Speaker
like that these days, if it's not DuckDB is going to be a distributed system. There is that interesting case of like, hey, what can I do if I don't want to be distributed? What are cool things I can do in an analytics database? And a system like DuckDB is
00:14:25
Speaker
is all that, but yeah, you're going to split the scanner for multiple machines so that you can, in effect, scan faster. Yeah. Okay. So Apache Pino, I mean, there are a few options out here and they, they all take slightly different strategies to get to that glorious scan faster, scan less.
00:14:47
Speaker
scan over multiple machines thing. And I'm interested in the design choices that Pino makes to get. Yeah. Shall we just kind of walk through it? Let's go right down into the, give me some data structures. Yeah, right. As I said to my wife. Give me some data structures. Oh, yeah. We talk about that all the time. That one might not work for me in the same way. No.
00:15:29
Speaker
Fundamentally, in Apache Pino, you've got tables. There's nothing weird in terms of data model, in terms of query language. You've got tables. They have columns. Columns have data types. You query the tables with SQL. Nobody gets hurt.
00:15:44
Speaker
of note, skipping around to the side a little bit, you don't create tables with SQL. You create tables with JSON. And that's an interesting design choice. The kind of classical way is like, OK, I'm going to create a table. I'm going to give you the schema. And then I'm going to tell you a little bit about some indexes that I'd like. And we'll just sort of build this thing up over time and mutate the thing.
00:16:07
Speaker
Yeah. And typical vendor extensions to create tables, like, well, here's the schema, and here's this custom bundle of key value pairs I'm going to put in there. I think the approach that Pino takes is that this is going to be a complex thing. Let's just kind of back up, and here's a JSON spec, and you'll create a table, define a table that way. OK. So you get a table.
00:16:31
Speaker
And that's not essential. Obviously there could be a future change where more DML is implemented and you can kind of do it. But like, syntactic sugar over the top of that. Exactly, exactly. Yeah, yeah. But see, you get tables. Tables are broken up into pieces called segments. And this is the fundamental
00:16:54
Speaker
Structure that that leads to the distribution that is the distribution model in Pino, right? So as a table is created There's two different ways to create a table one is out of batch data. The other is out of streaming data however, you're doing it you're creating segments and so a segment is an approximately time ordered time time delimited chunk of rows and
00:17:20
Speaker
Okay, why time as your like segmenting strategy? And it's not strictly, I mean, you wouldn't call Pino a time series database, but I say approximately because, say the batch ingestion process, right? Maybe there's
00:17:41
Speaker
collection of parquet files in a S3 bucket. And you're ingesting those. You're iterating over those. And you'll iterate over them and say, well, I've got enough. I'm going to call that. That's a segment. I've built a segment. Let me load that into the cluster. And so during that process. And in the streaming case, when you're ingesting, say, from Kafka, that's a bit more explicit. You have messages that you're consuming, and then you cut that off.
00:18:09
Speaker
OK, it's probably shouldn't say time, because that gets us thinking of a bunch of questions, and maybe we can dig into those. But we still have to think about the difference between the logical table, which you're saying feels very familiar, and the underlying disk model, which is where things get tasty. It gets tasty. Yes, you have these segments. Segments, and it would be worthwhile to
00:18:36
Speaker
Let's just call a segment a chunk of a table for now. It'd be good to dig into what one looks like in a little bit. But here's this chunk of rows, and we need to put them somewhere. So now let's get into the physical architecture of PINA, the components of the cluster. There is a component called the server, or the servers.
00:19:00
Speaker
I don't know if I'm the only one in the Pino world who maybe regrets that naming a little bit, because it seems like kind of aren't they all servers? But hey, look, there they are. They're really the true heroes of the cluster. They store segment data, and they do when it comes time to compute queries, they do most of the compute work. So the segments table is composed of segments. Segments are stored on servers.
00:19:26
Speaker
And let's see, where shall we go next? Would you like to talk about how segments are built or do you want to try to do a query?
00:19:36
Speaker
Let's go through a query, and we can dig in on the way. OK. So ingest, and more to say about that, the ingest process creates segments. Segments are distributed among servers through some process, which is interesting in itself. And now, here they are. You've got your table. There are these segments. Maybe there's one server in a Docker container on your laptop. Maybe there's 300 in your cluster in production.
00:20:06
Speaker
To make a query happen, we have another element we introduced. These are called brokers. There are usually fewer brokers than servers, but it is, again, a scalable element. And this is where a client process will actually connect to an API endpoint on a broker, submit a little
00:20:28
Speaker
JSON query document that basically just says, hey, here's the SQL I'd like you to execute. And that broker now will take that query and say, all right, we've got some table is implicated there. I see some predicates. And here's maybe the
00:20:53
Speaker
aggregation operation, not going to be of so much concern to the broker. But here's a table and here's some predicates. Let me try to figure out which servers ought to or need to do the work of executing this query.
00:21:08
Speaker
because it'll based on
Managing Pino's Cluster and Real-Time Data
00:21:10
Speaker
metadata that the broker knows should be able to make some good guesses about where the necessary segments live. Depending on the predicates, depending on the table metadata, it's going to try to route that query in a smart way. It's not just blast it to all servers that have any segment of that table. Sometimes that's what you have to do, but you don't want to do that.
00:21:36
Speaker
The word guess is interesting there. Why is it guessing? Yeah, you said it makes some guesses about which servers to talk to. Okay. That's, I think, me being a little anthropomorphizing a little too much. It's actually not probabilistic. It's deterministic. No, you do know.
00:21:53
Speaker
Okay, good. You know, either we can prune certain segments from even needing to be consulted, or in the worst case, you can't, and you're literally going to scatter to everything and do a lot of gathering. So it'll do that. It'll figure out which servers should get the query, and there are various
00:22:16
Speaker
levels of configuration and optimization and things that you can introduce to help that work well. But in the basic case, you've got, say, maybe there's a time column. And you're going to know, well, this segment definitely only, I'm aware of that time column. I was aware of it at ingest time. It really only has things from time t1 to t2. And your query has a predicate outside that range. I'm not going to bother with this segment. Right. Yeah. Yeah.
00:22:47
Speaker
What if, um, I may be derailing your architecture diagram, but what if like, I can see how, let's say you've got all the food, all the data coming in from Uber Eats, right? And I can see how that'd be spread over 300 machines, whatever. Now I come along and I am hankering after barbecue food as an analyst. So I want to know like how much money was spent in Texas since the start of the year. Okay.
00:23:17
Speaker
So that feels like the start of the year isn't a very useful filter, and I'm going to really have to hit a lot of the cluster to find all the places with Texas in.
00:23:28
Speaker
And that's okay. I mean, that's fine. And you as an analyst who likes barbecue food, I definitely love that journey for you and want to help you with that query. But the main place Pinot is pointing itself is more towards the person who says, I want some barbecue. I'm in San Antonio. It's six o'clock.
00:23:58
Speaker
What's going on right now? What's the delivery time now based on the recent history and things happening in the last few minutes, even if it's 6.10, delivery time might be different than 5.57. OK. So you are explicitly optimizing for queries that have a sensible date range.
00:24:24
Speaker
Not so much that, no. I mean, that large date range is fine. That's just going to be a lot of segments. That's maybe going to be a query that takes longer. There's more scanning happening. But the user facing analytics use case is more what Pino is thinking than the internal business analyst facing thing.
00:24:44
Speaker
Now that's not exclusive. There are interesting real time internal facing business analyst cases, but I think fewer of them than once analytics, once you take that same set of data and turn it into a feature in your application and deliver it to a user, then that latency concurrency freshness triad of nonfunctional requirements impose themselves on you.
00:25:12
Speaker
and you need all this stuff that Pino does. If you're just saying, you know, which zip code sold the most barbecue at dinnertime in January or since January, it's okay if that takes a minute. I mean, who do you think you are, right? Go ask Snowflake, it's going to be fine. That's interesting because, as I say, design choices lead to different sweet spots.
00:25:41
Speaker
and how it plays out in the world. It's interesting that you're optimizing specifically for a kind of interactive user. And that's how PINA was born. And there's a whole bunch more architecture to get back to. And I mean, hey, you're asking the questions here.
00:25:59
Speaker
We're going to go there. But do you want to do the origin story quickly?
Real-World Impact of Apache Pino
00:26:03
Speaker
It's important, I think. Yeah. OK. Well, we benchmarking your origin story against Madam Webb in cinemas for about three more minutes. If you can do better than that, you're doing. Actually, if you can only do better than that, that's not good enough. That's not a lot. Yeah. Ouch. Well, OK, it was born at it was born at LinkedIn just after Kafka. So as the Kafkafication of LinkedIn was happening,
00:26:29
Speaker
And that was really the framing of that story as we get it really has to do with data pipelines and not so much reactive microservices and that whole angle of Kafka.
00:26:43
Speaker
Links in the show notes. You've got some great episodes that talk about both of those things. Thank you. The recent Bobby Calderwood interview, I think, for reactive microservices. I mean, that's my description of it. I don't know if that's what you guys, you or he would call it, but I thought it was great. So you've got Kafka.
00:27:03
Speaker
creating these real-time pipelines and sort of having its impact at LinkedIn. And now there's all this real-time streaming data, but LinkedIn was still a resume and Rolodex site. It wasn't the interactive social media site with a feed that we know today that I visit often. And the first thing that they, you know, somebody conceived of this idea and I need to find the PM, interview them.
00:27:34
Speaker
the who viewed my profile part of the site. And that was the beginning of real-time data. And they wanted it to be real-time. I mean, it strikes me that you could make that a batch thing and it would work, but that's not what they wanted to do. And so they built this and it was either Elasticsearch or something like Elasticsearch. They had a search tool and they built it and did the
00:27:57
Speaker
the what was essentially an analytics use case on top of that. And they used like a thousand nodes of whatever this search and search kind of thing. Back then, that is probably how you would have done it. Yes. Yes. This is like 2013. Well, that's bad. And it's kind of funny to hear Kishore Gopalakrishna, my boss, co-founder of Star Tree, co-creator of Apache Pinot.
00:28:21
Speaker
He didn't want to write a new database. And this is really the story of, I think, like a relatively healthy person. What's the one thing you don't want to do? Well, you definitely don't want to write a new database. That's a terrible idea, right? Something has to be wrong. And he tried not to, but the team ended up building sort of primordial Pino.
00:28:42
Speaker
But in the origin story, they took that 1,000 nodes down to 70 with increased traffic and then decreased latency and just all these wins. They're like, okay, this might be something. And so it was born of a user-facing analytics requirement. And then at LinkedIn, they started using it for lots more things like that. Analytics on posts, the feed is built on Pino queries, so you're doing lots and lots of Pino. Spread there from LinkedIn,
00:29:10
Speaker
And this was the early days of gig economy meal delivery. So DoorDash was relatively new. And Uber was responding to DoorDash by introducing Uber Eats. And they adopted Pino to drive some Uber Eats functionality. OK. Let's get back to the architecture. You've brought me nicely back to thinking about food.
00:29:31
Speaker
So if I've got that, what's going on in San Antonio right now that I can order? Query. Take me through how the architecture of that is processed. Got it. OK. So it comes to a broker. We're going to look at the where clause and see, can I? Well, OK. Sorry. We're a broker, right? Yeah. We're in possession of fairly comprehensive cluster metadata. What tables there are of those tables
00:30:00
Speaker
what segments there are, what servers they're hosted on, how they're replicated.
00:30:08
Speaker
And perhaps certain other things one might know about the data in a segment like time ranges. There is a notion of partitioning, which we'll come back to when we talk about the Kafka integration. So a broker knows all those things. And I'd like to just take a brief detour to introduce another component. The broker knows those things because there's this element called the controller that really is the
00:30:37
Speaker
metadata clearinghouse. It just makes sure metadata changes happen to the controller, and it's responsible for pushing them to brokers and making sure that brokers are always up to date with the current state of the cluster. OK, so it's not the server telling the broker, like, I've changed. The server tells the, sorry, what did you just call it? Controller. And actually, what happens is when you
00:31:05
Speaker
load a new segment that is insert data, you've ingested some new data, you tell the controller. And the controller then makes sure that data gets to the right servers and then pushes that information, those metadata changes to the brokers. So it's the place where metadata changes happen in the cluster.
00:31:28
Speaker
Needless to say, there's a little zookeeper hanging off to the side actually remembering all these things. Inevitably. Thinking of itself as the true and perpetually undervalued hero in every distributed system. Yeah, absolutely. Anyway, basically, we might get to minions if there's time, but that's the pieces on the map. We can do a query now. Query comes in.
00:31:53
Speaker
Broker, being in possession of all the metadata because the controller is helpful and does his job, says, all right, based on my predicates, I see that there are these seven segments. Let's go with eight segments. Otherwise, we think we need seven segments. Are you trying for a power of two here? No, no, no, actually. We'll go five. There's five. There's five segments that are implemented or implicated in this query.
00:32:22
Speaker
So now the broker will say.
00:32:26
Speaker
to how many servers can I scatter this query? Because it would like to parallelize that work, right? Now, I'm going to avoid going much down this rabbit hole. There's a page in the docs that does a pretty good job explaining this. But you, on the one hand, want to scatter so that you parallelize that work. That was one of the ways that we could scan faster, right? On the other hand,
00:32:55
Speaker
You know maybe not necessarily all the way all the time because that will increase tail latency that will reduce reliability you know components that can fail you might have a gc somewhere so there is this little bit of tension between i want lots of people doing this work and i don't necessarily want. Everything doing this work.
00:33:15
Speaker
Yeah, parallelization isn't free, so you build in that cost. Exactly, exactly. Query routing is a deeply complex subject with lots of little knobs in it. But in the basic case, if you don't want to touch the knobs, then this thing happens. The segments are distributed to servers. The broker will figure out which ones. It will then scatter the query to those servers, and one server might
00:33:42
Speaker
have two segments and need to run this query across two segments, might have five of them, you know, it's however that works out. But let's just say it's nice, like we were building a slide for a presentation and there are five segments and there's one each on five servers and it's a beautiful world and gets scattered. Those servers now
00:34:03
Speaker
do the filtering and scanning. They do the IO on the segment and do the compute, whatever that might be. I've scanned this metric, it's delivery time or it's temperature or it's whatever, and now I'll average it or do whatever it is I do, create that result set and those servers then send that back to the broker. The broker collects all of the results it has scattered
00:34:33
Speaker
and does that final reducing operation, whatever it is, and then you've got a result. Okay, so there are shades of MapReduce inside this. MapReduce always...
00:34:45
Speaker
always pop it's down in the corner winking at you and you thought it's what 2024 you thought you know 11 years ago you you were too good for it and it was a pain and it's just there and saying yeah no I'm I'm still I'm here and you can't get away from me it gets better
00:35:05
Speaker
It gets a little more MapReduceier. The thing about MapReduce is it really was a quite general paradigm that just simply is the case.
00:35:19
Speaker
Yeah, if it wasn't invented, someone else would have... It's kind of discovered. I was discovered. It was invented. Yeah, it was discovered and it keeps getting rediscovered. So yeah, that's the single stage query engine I just described. I can, by configuration...
00:35:38
Speaker
or by an option on each query, say, I would like the multi-stage engine, please. Because that single-stage engine works real well for that filter and aggregate thing I just described, right? But what if it's a join? What if I have two large fact tables that I have to join and then do some computation on the result? Well, that would cause gigabytes or more of results to stream back to the broker in
00:36:06
Speaker
a really ugly case and brokers aren't built for that. So the way that works is in the multi-stage case, the broker will compute this multi-stage plan. It'll figure out, well, yeah, okay, to begin with, the kind of root scanning that we have to do to get started. Sure, these are the servers that are gonna do that work, but they will then,
00:36:33
Speaker
stream and shuffle. Again, Matt produced down there, like I never left.
Efficient Query Processing in Pino
00:36:41
Speaker
I'm thinking of the two astronauts looking back at the Earth.
00:36:46
Speaker
It's all mountainous, isn't it? Always was. So anyway, that first tier of servers will stream potentially lots of data, because now this is built for this, to another tier of servers, not brokers, who might say, and to make this a little more explicit, say there's a join, an inner join, and then an aggregation on the results. Well, you'll do the left join, maybe a filter if you can.
00:37:16
Speaker
The select for the left table may be a filter if possible. The select for the right table may be a filter if possible. Servers are doing this work. They'll stream the left and the right results to another tier of servers who will actually do the join. Right.
00:37:33
Speaker
stream to another tier that actually does the reduce the whatever the aggregation is and then back to the broker back to the broker the brokers. So the initial the broker at first is doing this query plan and then almost drawing a map.
00:37:50
Speaker
for how the data is going to travel eventually back to me. Yes, yes. And the little shuffle step, which has to happen there, is because if you're doing, say, an inner join, at some point, you've got a left-hand table, and you've got to go look at a thing that's your right-hand table. And what is that thing? Well, it's going to be a key value map in your memory, probably, right? So you've got to get those keys in the right place.
00:38:20
Speaker
As it's shuffling that, is it re-sharding it so you're trying to deal with a subset of the keys on each later stage? That's a good question. I imagine the answer is yes. That gets to a part of the engine, a level of detail in the engine that I don't know. But there can be many stages there. And so that is likely that that happens. But I don't want to commit to that answer so much.
00:38:43
Speaker
Okay, that's fair. This isn't quite a job interview or an exam. You don't call them exams. You call them tests. Very different. Maybe it feels like an exam. We call them tests over here. Well, I know what this podcast is about. This is not a podcast for fluff, so I don't object at all.
00:39:01
Speaker
Good. Okay. So I can see how that works, but since we've mentioned MapReduce, let's mention it one more time, my first association of MapReduce was Hadoop, as it was for many of us poor souls. And I associate that with parallelization. I do not associate it with speed. No, no, you don't. Convince me that this process will be quick.
00:39:28
Speaker
Yeah, yeah. Well, I mean, the proof is in the pudding, right? There are folks who publish results of how this stuff works. It always depends on how big the data is, what the cluster is like. I mean, you know, there are so many variables that we almost need
00:39:46
Speaker
Like nascar you know there's like one car that you can build and then the driver is the one that that that really competes i can't believe she's a nascar analogy with you i just this is none of this is lost on me but it's podcast british host let's talk about nascar.
00:40:07
Speaker
I do actually know what NASCAR is, so we're on good ground. There you go. And no disrespect for the whole ecosystem there. I'm just not a part of it. But the idea is that the car is standard and the driver is what differs. And if you had a standard cluster and a standard data set, then
00:40:24
Speaker
then you could start to make some comparisons. But this is all, there's just everything that happens in oranges in this world. But yeah, no, again, the design of this thing, of Pinot and the controlling kind of set of non-functional requirements as constraints, concurrency, freshness, latency,
00:40:49
Speaker
that first kind of query, the filter and aggregate thing, bread and butter, single stage engine, there are documented results, users not StarTree of like 10, 12, 15 milliseconds P95 latency on meaningful production data sets.
00:41:14
Speaker
Yeah. But what I mean is, so I'm not doubting the speed. What I'm kind of asking is, I mean, what's going on? Because 12 milliseconds, right? From the architecture you've described, you could easily have spent that already just on networking and serialization. So, yes. And that's not a join, right? So one of these fact-to-fact joins on large data sets. And this could be a query that takes a second.
00:41:40
Speaker
He said with horror. Yes, yes. No, that's not a difficult Pino kind of latency. But again, once you start coming away from the hyper-optimized thing into generalized joins, well, yeah, it's going to take a little bit. But not Hadoop time. That's the whole reason and the world into which Pino was born
00:42:06
Speaker
was a world of Hadoop and Hadoop pivoting into Spark. And same thing with Spark. I mean, what are you gonna do that takes less than 10 seconds? That's a fast thing. That's anathema. And we get into reasons for that, but there are interesting decisions about,
00:42:28
Speaker
the coupling of compute and storage, the pre-allocation of storage, the pre-allocation of compute, like, are these things going to be on demand or reserved and how tightly coupled are they going to be? And there's kind of a, you know, you can make quadrants with those two. And Pino lives in a place in that quadrant, typically, where storage is tightly coupled and storage and compute are pre-allocated. They're ready to go. You don't go find
00:42:55
Speaker
resources. The resources are there and the compute happens in the server right where the data is on an SSD on the other side of a PCIe connection. And that coupling is key to the scan faster. Right. Is it going to typically be the case that the brokers and the servers are on separate machines but in the same rack?
00:43:23
Speaker
Yes, they will likely be in the same rack, you don't know, but same availability zone for a cloud deployment and definitely separate machines.
00:43:37
Speaker
Right. In that case, I think to get the next piece of this puzzle, we need to start talking about indexes because we haven't mentioned that at all. No. And that is a key place where Pino has decided to elaborate and introduce complexity and differentiation and things. OK. Key point.
00:44:00
Speaker
It might be a good time to just remind ourselves, those segments that we make when we ingest data, those are columnar in nature. So you've got inside a segment little chunks of contiguous storage that store the column values of a table. In no case do you have a row all stored together.
00:44:23
Speaker
That goes back to the, are you transactional or analytic? If you're transactional or OLTP, you want the thing and probably the whole thing. So put the whole thing in one place because you want to deserialize it or serialize it all at once. If you're analytic, you're probably scanning over some chunk of metrics, so metric values. So put those in one place. Right. Yeah.
00:44:51
Speaker
So when a row comes in, you're actually creating lots of different segments, one for each column, right? Well, it's called a segment, but internally in that segment, you've got chunks of columns. And indexes. A segment is all of the columns of the rows that it comprises.
00:45:15
Speaker
plus the indexes that you've built on those. So what have you got? Of note, there is an inverted index. I've got this value. Can you tell me the documents, rows in which this value occurs? That's good for low cardinality dimension type columns, city, state, country.
00:45:42
Speaker
There are a couple of different text indexes. If you've got a column, that's a blob of text, of unstructured text. Pino has essentially a text index that is Lucene. So anything Lucene can do, it can do because it is.
00:46:01
Speaker
Embedded Lucene and it also has recently in the last year and a half grown a native text index That's like a higher performance subset Of the like the common kinds of prefix suffix Phrase Boolean regular expression. Yeah stuff
00:46:21
Speaker
There is, this is all building up to the cool index. So just give me a second. We're getting somewhere. It's actually called the cool index. Yes. There's a JSON index. So imagine, I mean, a lot of your data you may be ingesting could be in JSON format that you're going to flatten and things. But imagine if you've got like a
00:46:43
Speaker
sub-document object, nested object is the word I'm looking for in that JSON that is sparse. There are 200 keys that might show up, but you usually only have five of them, say. Right, yeah. The schema's very flexible, the data's quite narrow.
00:47:05
Speaker
Yeah, yeah, yeah. You can have 200 columns. That would be a life choice. Pino gives you a JSON index where you could say, let's just take that embedded object, bring it in as JSON in that column, and JSON index it. So now you can index into fields and array values and all the usual suspects of what you might want to do. Are you saying specifically which paths you want to index?
00:47:32
Speaker
Um, um, I don't think so. No, no, you're not. You're just saying this thing, go index it. And so look through all the paths in there and build something sensible.
00:47:45
Speaker
Yeah. OK, cool. There's, let's see, we've got text, we've got JSON. Is that Geospatial Index? Because Uber was an early adopter. Obviously, that's going to be a thing. They kind of care about where things are. Yeah, got to know where that barbecue is. That's a whole podcast in its own. And I will be clear, I frankly know not much about the Geospatial Index. I haven't really put that through its paces, but it's there and highly productionalized.
00:48:12
Speaker
Let's get to, what am I missing? Oh, range, ranges, metric ranges.
00:48:23
Speaker
So something, just a numeric column, that won't work well with an inverted index because the cardinality is very high. You might have a number of unique values that's equal to or in the same order of magnitude of the number of rows in the table. And so the range index will just chunk that up into ranges and build an inverted index out of those ranges. Same thing with time stamps. Time being a classic one, right? Yeah. Yeah, yeah.
00:48:50
Speaker
And the cool one. The cool one. They're all cool. The StarTree index. Not to be confused with the company called StarTree. Which was named first. The StarTree index was named first. Right, okay. Yes. If you named the index after the company, that would be dorky. Naming the company after the index is cool. Is cool, exactly. Well, it is an implementation detail that leaked into the interface. Happens to everybody.
00:49:16
Speaker
But it's cool and even if five years from now the star tree index is old news and there's newer and cooler indexes still have a cool name. This is basically. Like a like a pivot table.
00:49:33
Speaker
Save to disk. You pick columns and the use case here is a filter and aggregate query. So there are some columns that may show up in the where clause in some order and some combination of them, say three of them, four of them. We don't know which ones are going to be there. And then you want an aggregation on some other metric.
00:49:58
Speaker
some average, whatever. You can actually have several aggregations computed in the index, and it will actually build a tree of the different values of the different
00:50:15
Speaker
predicates that show up, the different columns that are going to participate in the index. And so now you've got this log in search through the index where you get the pre-computed aggregate. So you just read the index. You don't even have to go scan the column anymore.
00:50:32
Speaker
So this is where you get those crazy 12 millisecond latencies. Right. If you can pick the columns that everyone's most frequently asking about and the kind of sums that everyone's frequently asking about. You design the index because you know there are these queries.
00:50:51
Speaker
that are going to happen. And this is the typical response is when a person first, you know, configures the index and runs a query. Everybody always thinks something's broken or like this has to be in a cache somewhere. This can't be real. You know, it's, it's too fast. It's too fast. I just can't believe it. Stop, stop. You're giving me too much. No, but it's, it's a super cool index because that is you don't want to be limited to that, right? The filter on these few things and compute this aggregate.
00:51:20
Speaker
That's a lot of what you do in an old app or so you can really optimize the daylights out of that makes me think of google analytics where. Okay so in that case you can't ask any question but there are certain combinations of fields you're pretty flexible about and you can get those quickly and.
00:51:39
Speaker
Yeah. Presumably, they're using something very similar to that kind of index. And then failing to do the rest. Because I've got questions for Google Analytics that they can't answer. How are users? Well, that'd be a good Google Analytics PM. Get them on the show. Yeah.
00:51:59
Speaker
You can get Google people on the show, but you can't get their lawyers to sign off on releasing it. That's the thing. Anyway, that's an aside. I want to pick up on one quick thing you said, which was, maybe I've misunderstood the size of these segments, but if you're storing the data and the index, that seems a bit strange to me because the whole point of an index is it tells you quickly where else to go, but where else sounds like it's right by where you already are.
00:52:29
Speaker
Yes, and you want to think of a few hundred megabytes, usually. Segments get chunked off into something less than a gigabyte. And that's variable, but that's a common sort of size. But the indexes are there.
00:52:49
Speaker
The broker is doing its best to make good decisions about which segments are actually going to get asked questions. And there's a little bit of metadata around that. There's even an index. It's called an index. It's not exactly an index, but it's called the Bloom Filter Index. So if you've got
00:53:16
Speaker
Equality predicates known known equality predicates that are likely to happen on a particular column You can have the broker You know be able to consult that broom bloom filter and know Sorry, not the broker the server be able to consult that bloom filter and know should I even bother with this? so the
00:53:37
Speaker
The reason the indexes go in the segment is because you've got these little bits of metadata where the broker is trying to do a good job, not just routing to smart places, but pruning in a smart way. And so what you don't want to do is
00:53:58
Speaker
scan through all the values in a segment. You really would like to try not to do that. So you can more efficiently read indexes or consult a portion of the index in memory and make better decisions about how to scan. That's the scan less priority. Right, yeah. So there's a thing about indexes that tell you where to go efficiently. And that part, that role is happening on the broker.
00:54:28
Speaker
And then there's a thing about indexes of gathering the data up quickly once you're there. Not quite. And that's happening on the server. There's table metadata that the broker has that is something that the process of building an index might tell you, but it's not properly a part of the index. It's just table metadata. We have this timestamp column. We're going to tell you the first and last time, the highest and lowest time in this segment.
Data Ingestion and Real-Time Processing in Pino
00:54:55
Speaker
or you're partitioning on this other column, we're going to tell you the values of the partition key that are in this segment. So that metadata is owned by the broker. And it's index-ish. Like I said, could be the output of a notional index computation process, but it's not a part of the index. The role of the index on the server is to optimize the actual scanning of values in a segment.
00:55:23
Speaker
Yeah. Okay. I can see, I see the distinction, but it's certainly allowing you to narrow down a good chunk. Yes. It was just what you want to do. Scan faster, scan less. You know, you can only optimize your IO code so much. And so scan less in smart ways is where you make your money.
00:55:44
Speaker
Okay, so I want to gradually get into getting the data in here. Yes, I feel like we haven't got a lot of data. And we must. In order to get there, I think, because we're going to end up talking about Kafka, I know it. I don't know how we could not. It's sort of contractually obligated when you and I talk. Between the two of us, yeah. Yeah, absolutely.
00:56:04
Speaker
You haven't really touched, you've talked, you've mentioned, but you haven't really touched on, you have two kinds of tables. You have batch and streaming, I think you said at the start. But these all seem like batch tables to me. Yeah, it's, I think when explaining this stuff, I always start with batch tables. And the terms of art in Pino are offline and real time.
00:56:32
Speaker
Now, offline tables are quite available for queries all the time. Again, we know where that word comes from, but it's a little bit misleading. So there's offline and real-time tables. And I think it's easier to start thinking about the way real offline tables work. Because real-time tables are just offline tables plus this extra thing from a logical standpoint.
00:56:56
Speaker
There are also hybrid tables, which are actual combinations of ingested batch data sources and ingested streaming data sources. But let's put that to the side for a minute, and let's just talk about ingestion. We'll save that for the Christmas special. That's for the Christmas special. All right, so offline table ingestion, batch table ingestion.
00:57:20
Speaker
formally is done outside of the server and outside of the broker. It's a process, quote unquote, external to the cluster. And you could literally, there's an API, you can write a Spark job, write a Hadoop job if you're into that kind of thing, that takes some external data source and makes it into a segment.
00:57:40
Speaker
and then presents that segment to the controller for uploading into the cluster and movement to the servers and all that stuff, right? So you can actually, using Pino APIs, create segments, put them into the server. And that's a little bit, I don't wanna say it's weird. I mean, it's a perfectly great thing to do if you've got a lot of lifting to do.
00:58:07
Speaker
If we could just go back to the case of, I've got all this CSV in S3 or something like that or parquet files or Avro, whatever it is. There are built-in command line tools where you can point to those things.
00:58:23
Speaker
You've created the table, and the table has a schema, and you write a little ingestion job spec file. The only place YAML really appears in PINA, unfortunately, those ingestion job specs are YAML. And there's another component called the minion servers. And this is where, and we just kind of know, there's going to be these background things you need to do. You're going to need to go through and
00:58:53
Speaker
delete things for GDPR compliance. There'll be reprocessing of segment data. And so the Minyan mechanism is that. It's just a little distributed job.
00:59:05
Speaker
distributed computation mechanism on these servers over here off to the side and batch ingestion is typically done if you're not writing your own in spark or whatever it's done with minions you can. Not even really know that right you're just kind of following the script and running the ingestion tool and pointing to the things and writing the spec and.
00:59:26
Speaker
okay, it's running and I can go to the UI and I can see it running and oh, it's done now, right? But what's happening there is these minion tasks are being created and distributed among the available minion servers to read the inputs, create the segment files, present the segment files to the controller, transfer them to the servers, all that stuff. Yeah. Okay. That's batch.
00:59:49
Speaker
Good enough. Should we move to? I'm happy. Making more complicated for me. Yes, but more awesome. So Pino co-evolved with Kafka. Kafka was young and I don't know what...
01:00:02
Speaker
I'd have to look at the history, but it might've been like, you know, before Kafka even had replication, there was, there was Proto-Pino being built alongside. And so- The grapes were being squashed. They were, they were being squashed before Kafka was, was the force it is now. And as a result, its integration with Kafka is very, what's that?
01:00:35
Speaker
And so the typical way, if you have a database and you have Kafka, you know, you have maybe something like Kafka Connect to read from a topic and write stuff into the database, none of that here. Pino is its own Kafka consumer. And so when you have a real-time table, part of the, and we could consume from Pulsar and Kinesis and other places too, we'll just keep talking about Kafka here, just to keep it simple. Part of the table configuration, which remembers this JSON file, literally contains
01:00:56
Speaker
They're familial, they're brothers, sisters growing up in the same home.
01:01:04
Speaker
Bootstraps over URLs, credentials, topic name. To be a real timetable is to be connected to a streaming source. Oh, I see. Okay. What you do is you consume messages from that streaming source.
01:01:19
Speaker
And you have options like do I start at the beginning? Do I start at the most real? All that stuff is there. But that's what it means to be that table. And so as soon as you create a real-time table, it's going to try to connect to that Kafka cluster and start consuming. And it happens. OK.
01:01:37
Speaker
Does that mean that there's a minion looking at a Kafka topic, constructing a segment, and what's it going to do? Is it going to wait until it's got several hundred megabytes and send it off to the controller? That would be one way to do it, which would be terrible. And so the answer is no, minions are not involved in real-time table ingestion.
01:02:02
Speaker
only servers. So the controller is going to make some decisions about how many partitions there are, how many servers need to be involved. It's not time for this yet. We're going to come back to this. Let's just say the server will add layers onto this, but servers are Kafka consumers directly.
01:02:28
Speaker
And so the server is told by the controller, this is your topic, you consume from it. It may even be told
01:02:39
Speaker
Don't let the cluster tell you what partitions you get. You are consuming from these partitions. And that's a configuration difference. In fact, we'll just go there. There's the difference between the high level and the low level consumer. And the high level is, hey, server just takes stuff. Consumer group protocol, you'll get rebalanced. You'll get partitions. Nobody gets hurt. Low level is.
01:03:06
Speaker
we are gonna keep track as a cluster of the partition key and its values.
01:03:13
Speaker
And these partitions will be assigned to you. These partitions will be assigned to you. You only get to consume from these two partitions. And then that partition key, if it's a column in the table, and it shows up in queries later on, now we've got a new way of pruning segments. Because we remember what we know, based on the segment that gets created, we know what partition it came from.
01:03:48
Speaker
detail in there. I would like to just keep it at that hand wavy. Trust me, it's cool if you have that as a predicate. Now, the server is a Kafka consumer. Whichever way it works, it's getting messages. And as soon as it consumes a message, it puts it into an in-memory data structure called the consuming segment.
01:04:10
Speaker
Um, right. Right. Yes.
01:04:13
Speaker
And that consuming segment participates in queries just as if it were a segment on disk. And so that's the key. As soon as a message is successfully consumed from a topic, it can show up in a query result. So there's no additional latency there. It's in the consuming segment. We're done.
01:04:35
Speaker
right okay that answers my question yeah that's yeah that's why it's not that terrible thing and then it fills up for whatever you know your configuration knobs that define what fills up means yeah uh now we create a segment and there's a whole process where the server uh
01:04:53
Speaker
There's various ways of talking to the controller, and the controller is going to make decisions about what server that segment now lives on. I wish it might not live on the same server as the one that was gathering it up. It's possible for it not to. And we get into stuff here where I start getting fuzzy on the details of these things. Yeah, we'll save that for Pinot 202. Yeah, it's just way down in the weeds. It's just interesting that that can happen.
01:05:21
Speaker
And yeah, so you got the consuming segment, it participates in queries, and then it's just creating segments through a process that looks an awful lot like what a minion would do.
01:05:31
Speaker
OK, so it's quite a different mechanism under the hood, but the end result is transparently the same segments on disk, this segment like thing in memory that is a segment for all I can tell. And yeah, and again, if if I've got a bunch of batch data sitting around that has the same schema as my Kafka topic, then I can create a hybrid table where I've ingested that batch data and I keep ingesting new stuff from the Kafka topic and
01:06:02
Speaker
Under the hood, there are actually two tables, but that gets abstracted away, and I just see one. Okay, yeah. There's so much more I can go into, but for time, I think I should probably pull back out slightly to use the space. There you go. Yeah, because I know there are some fun ways that people are using this, and I want to get some idea of how it's being used in the wild. Yes, a fun thing.
01:06:31
Speaker
Christmas, early in the season, last Christmas, I, you know, it's just, I want to say you gave me your heart. I knew you. Yeah. Very next day. I gave you my heart. Very next day you gave it away. Um, you were supposed to save that for the Christmas special. There you go. But it's, you know, every day can be Christmas really. Um,
01:06:54
Speaker
I believe it was Stripe, published a dashboard around American Thanksgiving time that weekend that was fed by real-time data of transactions on their payment platform, seeing live updating numbers. Now that was to some degree, I think, advertising on Stripe's part. Like, hey, look, we do lots of stuff. And they've got a cool story. You think of mom and pop and small merchants.
01:07:22
Speaker
And it's great stuff to talk about, but they are Pino users. And so all transactions through the Stripe platform live in Pino.
Pino's Application in Real-Time Analytics
01:07:34
Speaker
And so there was more to that story. There are a bunch of interesting pieces to that dashboard on the application side and all that. But with all that data in Pino, well, it becomes fairly straightforward.
01:07:49
Speaker
To build such a dashboard because those queries are gonna be cheap things that you just get to run and show to web users. And the application programmer there is genuinely expecting to run each time it wants to display that is it going to probably be running.
01:08:08
Speaker
the same query? I mean, are you expecting the web server to run as many queries as there are users asking for the data?
01:08:23
Speaker
shape of what it feels like as an application developer to use Pino. My answer is yes. And I don't know in the case of that, since I'm talking about a specific thing that happened just a few months ago. There could be more details there. But me as a developer, I wouldn't be immediately thinking, oh, connection pools, extra caching layers. Exactly. Exactly. You should guard yourself against thinking extra caching layer.
01:08:48
Speaker
Now, caching is not bad. Caching rules everything around us. There you go. It's an absolutely necessary part of computation at every meaningful scale in our lives. I'm not trying to say it's somehow bad.
01:09:03
Speaker
When it comes to data, particularly analytical data, the answer used to be let me pre-compute this and put it somewhere I can read it. Let me run this job and fill up my key value store because that's the only thing that I can query with the latency, concurrency, and freshness requirements that I have to be user-facing.
01:09:24
Speaker
And now it's out of date and I have to pre-compute. And if it's multi-dimensional, my life is terrible. Yeah, cash is too stretched, which isn't all this. Yeah, when you go to that, I'd better cash this. That thinking is an artifact of the way things used to be.
01:09:39
Speaker
And that's not how you want to use Pino. You want to use Pino like you'd use an application database. And I suppose there's been caching there as well, but you'd rather not, especially if you're actually reading and writing entities. I don't want the cached version of my account. I want to see what my actual account is right now.
01:10:01
Speaker
That's nice, because writing that caching code is never fun, and it always ends up like you've accidentally got a second time of database. Yes, yes. And again, I don't want to be heard as giving some sort of anti-cache invective. It's a thing, but it's a very specialized kind of thing.
01:10:26
Speaker
You as an applicant speaking to fellow application developers, you don't want to build that. That's going to be hard to get right. And so, you know, is trying to say that's not a thing you need to do anymore. We've got analytics. Just ask me the question. And yeah, there you go. No one builds caching because they want to. They build it because they have to.
01:10:49
Speaker
Right. Right. Unless they are specifically doing an I want to build data infrastructure kind of motion, which is great. I'm glad there are people who do that. And that's sort of what this episode is all about. But yeah, your job as an application developer is to not build data infrastructure. Yeah. Cool. So if I want to go and give Pino a try,
01:11:12
Speaker
Where do I go and what do I expect to happen to me next? There you go. You could go to dev.startree.ai. That's my favorite place for people to go. And there's a little linked quick start at the top. And the current form of this, as we record this episode, is a one-liner. It's a Docker image. It's the Pino Docker image. And it's got some little built-in magical quick start things that
01:11:40
Speaker
simulate all the components being present, give you a little web interface. It's all kind of there and it can't. It's wonderful because it's one line and you can play with it. You can't then modify the Docker Compose file and look at the data and it's all, you know, you press a button, the light comes on. So we're in the process of building and by the time you hear this or a few weeks later, that might be the form of a proper repo that you clone and a Docker Compose file and data.
Conclusion and Resources
01:12:09
Speaker
coming very soon as we talk about this. You'll be able to kind of see what it looks like. There's also pino.apache.org and docs.apino.apache. Docs.pino.apache. Docs.apino. It's not Mario here, Tim. It's a modern tongue twister. The Docs site. I'll put a link in the show notes. Pino.apache. Links in the show notes, I think, is the best way to put that, where you can read more. And Star Tree's got a great YouTube channel.
01:12:39
Speaker
I've seen they've got just really handsome guy fronting that doing great work great voice. I love that guy. Yeah No, there's and and other people in the general too but some some great tutorials and kind of intros and and a lot of this isn't
01:12:56
Speaker
I mean, how Pino works is always of interest. If you are a technical person, you want to know that, but some of it is, you know, why is this not snowflake? Are you, are you stupid and you're just trying to make a different snowflake? You know, it's not. And there are some, we've talked about these concepts, but we've got some other little videos that just help drive those home. Like, why would you even want this thing? It's a different way of doing analytics. And what I think is going to be a lot more important going forward. Cool. Well, we will find out.
01:13:25
Speaker
Tim Berglund, thank you very much for joining me. Thanks, Chris. Thank you, Tim. Tim ended there with a bunch of links, so I'll just remind you, you can find them all in the show notes, including a link to the episode I recorded with Bobby Calderwood, Tim mentioned. It's an episode all about event systems, so if you're into event systems, or you don't yet know what they are, check it out, it was a very interesting discussion.
01:13:51
Speaker
Before you click away to there or wherever you're headed, if you've enjoyed this week's discussion, please take a moment to click like, share, rate, or whatever feedback buttons your user interface currently offers you. And make sure you've clicked subscribe because we'll be back next week with another discussion from the world of software. Until then, I've been your host, Chris Jenkins. This has been Developer Voices with Tim Berglund. Thanks for listening.