Kubernetes Bites Introduction
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
Boston Greetings and Date Mark
00:00:30
Speaker
Good morning, good afternoon, and good evening wherever you are. We're coming to you from Boston, Massachusetts. Today is May 3rd, 2024. Hope everyone is doing well and staying safe. Let's dive into it.
Bobbin's New Job Experiences
00:00:42
Speaker
Bobbin, how's it going, man? I'm doing good, man. Overall, still ramping up at the new job. Yeah.
00:00:49
Speaker
Yeah. Learning the product, which is, which is obviously a cool experience. Like asking all the dumb questions, uh, you still get new guy period where you can ask all the stupid questions and nobody judges you. So I'm just using that. Yeah. You have like what six months to do those, uh, some things. Realistically, we should always ask the definitions. Then we can start saying like, no questions are dumb questions, right? Yeah.
00:01:14
Speaker
Anything fun going on this weekend? What are you doing? No. So for the weekend, I'm at Catskills, New York. I
Weekend Plans and Red Hat Summit
00:01:21
Speaker
think upstate New York. This is my first time here. My hometown. Sort of. Sort of. Okay. How far is it from here? Well, the Catskills. So I grew up outside of Poughkeepsie, and when that's where I crossed the Hudson River is the Catskills. Oh, okay. So yeah. Close enough. I spent a good amount of time hiking in those mountains. Nice. Yeah, I'm planning on getting some hiking there.
00:01:43
Speaker
Nice. And then, yeah, next week I will be heading to like Red Hat Summit. So if you're there, just say hi. I'll be at the NetApp booth. Nice. Where is that? Denver, Colorado. So I'll meet Tim again. Oh, that's right. We'll get some ramen and send you pictures. Please do. I'll be super jealous. Super bummed that we haven't done that in a while. Hopefully in KubeCon North America. I imagine you'll be there, right? Yeah. I'm planning right now. I'm on the schedule.
00:02:13
Speaker
Let's fingers cross that I'll be there as well and I'll go get some ramen. Yes. Cool. What are you doing this weekend?
Family Gatherings and Birthdays
00:02:22
Speaker
Yeah, I'm heading to New York. I'll be probably like a half hour from you.
00:02:27
Speaker
Um, um, the whole family is getting together and we're, uh, emptying a house of my, my grandmother's. Uh, and, uh, we have like four dumpsters, like giant dumpsters, cause they've been there 52 years. So I get to go in the attic and, uh, get six dressers somehow. I, why is there six dressers in the attic? I don't know. I'm not sure, but 52 years, you know, I guess things just happens. Nice.
00:02:54
Speaker
Good luck. A lot of heavy lifting. It'll be good. The whole family will be there. It's my brother's birthday Sunday, so it'll be kind of like doing it all at once.
00:03:04
Speaker
It's perfect. Okay. Cool.
IBM and HashiCorp Acquisition Discussion
00:03:06
Speaker
So we have a really cool guest, uh, from EDB on, but of course we're going to do our news first. So why don't you kick us off? Yeah. So for today's news, I think we have like just acquisitions and funding rounds. Uh, so I'll start with some acquisitions. Uh, people in the community might already be aware of this, but IBM is now acquiring HashiCorp. Like it's not final yet, but I think it is. I mean,
00:03:29
Speaker
As much as it's been publicized, I would be surprised if it didn't go through.
00:03:36
Speaker
Yeah, true. So yeah, IBM acquiring another one of these open source ecosystem vendors. I was surprised to see that they didn't, in the announcement itself, they didn't say, oh, we are merging this with Red Hat, but I expect that to happen, right? Or at least part of the same. Yeah, sort of. Like make something else, like an infrastructure as a code BU inside IBM when they integrate Ansible and Terraform.
00:04:01
Speaker
I mean, having Ansible Terraform in, in I think that's pretty powerful. That's a power move. I'll put it out. Have you seen the memes? Like if IBM combines Ansible and Terraform, they'll make something terrible. Yeah, there, there definitely has been a lot of mixed emotions about the acquisition. And honestly, if you're listening to this, and you have an opinion, come on our Slack, tell us about it. I'd love to hear
00:04:26
Speaker
sort of the community that we talked to. How are you feeling about it? I mean, I think it's a good thing. Yeah. And good for Ashok for getting like, I know they were already public, so they did get some sort of exit, but it just sucks to see that.
00:04:42
Speaker
Like they went public at like $80 a share and this is I guess $25, $30 a share. So like the valuation definitely decreased, but hey, the technology will still be around once IBM acquired it, right? Like nobody is fired for buying IBM, so. Yeah, maybe it'll boost them, right?
Wiz's Acquisition Moves
00:05:00
Speaker
Okay, talking about more acquisitions in the cloud-native security ecosystem, Viz Security, I guess in the cloud-native or DevSecOps space is acquiring a startup called Gem Security for $350 million. That wasn't shared on the main article, but found it on, I think, TechCrunch.
00:05:19
Speaker
But yeah, $350 million, good chunk of change. They are going to merge with his own CNAB platform with gem security's CDR or cloud detection and response solution. So another acronym for us in the security ecosystem like CDR. How can they do much about Wizz, honestly?
00:05:40
Speaker
Yeah, so they are trying to become more and more relevant, right? Like, I was surprised that their ARR, they announced this year, for like $350 million ARR, which is not a small number. Yeah. Yeah. I don't know how long they have been around, but I think it was a couple of- The article says four years. So they're pretty young. Pretty young, yeah. I think both the founders came from Microsoft after having sold their previous startup to Microsoft, so they know how to do this over and over again. And yeah, they're just picking up other players and building a
00:06:09
Speaker
from their perspective, their marketing messages, building one unified solution to solve or to help with security, all aspects, not just cloud native, but also cloud security. So good for them. And then they are also in talks to acquire another cloud native security startup called Lacework. So this is an interesting one. Yeah. Yeah. So Lacework is interesting, right? Like they, they, yeah. So they raised like,
00:06:35
Speaker
more than a billion dollars valuing themselves at 8.3 billion dollar post money in the last funding round. And based on the TechCrunch article and the rumors, it's like they might just get acquired for like 150 or 200 million. So like going from 8 billion to 200 million, definitely not a great sign. But
00:06:53
Speaker
Yeah, at least it's an exit. It's not shutting its door down. Let's see if the deal closes. Based on the article, it looked like the intent letter is signed, so they are working through the process. I wonder if that has any perspective on what the security market looks like.
00:07:16
Speaker
You know, like, is it becoming more saturated? I mean, I remember talking on this podcast last year and security was, you know, probably the top thing we talked about quite often.
00:07:29
Speaker
Yeah. Even the KubeCon show floor was all security vendors, right? Like all different, like you have spoken to Armosec and we have spoken to Mondo and those guys, but yeah, compared to Wizz and now them acquiring more and more startups, like it looks like they are running the consolidation play for sure. I mean, they're not, I mean, they're smaller, right? It says a hundred million ARR, which is pretty good. I mean, all things considered, I think a lot of smaller startups have like 30, 40, 50 ARR. So, I don't know. Cool.
00:07:58
Speaker
And then next up, I think I have like a funding round.
CodeWeave's AI Expansion
00:08:02
Speaker
So CodeWeave, I know we have spoken about them a couple of times. They are a GPU cloud provider. I think when they started, they were going after like the crypto market and allowing people to run Bitcoin mining and other kinds of mining on their GPU cloud offering. With the AI boom, they've definitely had a great pivot into allowing organizations to rent out GPUs. So at this point, they're just
00:08:26
Speaker
I think previous to this round, before this round, they had raised like debt capital. This one is like a proper official CDC funding over a billion dollars. They're now valued at $19 billion. They're just building out infrastructures. So they're building out data centers with GPUs in them. I'm sure NVDA, it's funny that NVDA is one of the people that are funding them. And they're also the beneficiaries on the other end because at the end of the day, we was buying more and more.
00:08:55
Speaker
But yeah, another round for them. They had a good presence at NVIDIA GTC conference last couple of months back when I was there. They're doing some good things. In addition to them being a GPU cloud, they are completely Kubernetes-based. So all of their infrastructure inside their data centers is Kubernetes-based, so they are a good supporter there, or I guess a good part of the community. We should try to get them on the show.
00:09:19
Speaker
I should, yeah, I should reach out to a couple of contacts that I have. Yep, I can do that. Yeah, pretty interesting perspective, I think. And if you think so, listener, you know, I want to hear more about that. Let us know. But yeah, definitely reach out and try to get that. It says they secured 2.3 billion.
00:09:34
Speaker
That was a debt funding last time. And then this is like CDC of 1.1 billion. So they need a lot of money. Like they're building out physical data center locations. Your GPUs are not cheap. Yes. At the end of the day. Like I was thinking like after reading this, should I buy more Nvidia stock? Like, come on. I mean, that's one thing to do.
00:09:53
Speaker
Wait, that's it for news for me. Cool, so I'm not sure if we officially covered 130.com, I know we talked about it, but it recently officially came out. The Kubernetes 1.30 release, co-worker of mine, Kat Cosgrove, was release lead, so she talked all about it.
00:10:13
Speaker
internally and really excited for her and the rest of the team, of course. But if you're unaware, how cute this release is, the Uubernetis is the release. And it's such a cute picture of a cat. But that whole Uwu is sort of about cuteness and stuff. And if you're into that kind of thing, you'll love it. But there's a lot of great things in the 130 release. There's actually a bunch of storage stuff. Right. So like the stable features like the
00:10:43
Speaker
robust volume manager reconstruction after the kubelet restart. That's something that's been around. But now the feature gate's locked and cannot be disabled. So you get it for a box. Unauthorized volume mode conversions during volume restore. This is like when you're creating a volume from a snapshot and you want to change the volume mode. That is blocked by default. So you need to go through some
00:11:13
Speaker
not workarounds, but basically to enable the being allowed to do that. And that is a feature flag that is also enabled by default in the external provisioner and snapshotter stuff. So there's a bunch of good links.
00:11:27
Speaker
In there, lots of stuff going on, a lot of things becoming stable, some new alpha features. One that I know we've mentioned and probably have felt the pain for is the SE Linux label changes. Typically, they've always been, if you've dealt with storage in Kubernetes,
00:11:50
Speaker
Kubernetes has sequentially walked through the entire file system, and it can be very slow with large volumes. So the alpha feature is to speed up recursive SC Linux label changes. I can speak today. And so 130 extends the support to all volumes as alpha.
00:12:16
Speaker
And so there's some charts and tables in there that kind of break down which feature gate in 130 versus beforehand. And that is also not supported on Windows, of course, and Linux nodes without SEO support.
00:12:34
Speaker
Some really good stuff there and some other recursive read-only mounts to help protect data. There's a theme here about preventing unauthorized stuff, protecting data. Again, we were just talking about security. Security window. Yeah. Yeah. So lots of features that graduated to stable. I won't go through them all, but there is a lot in there. So if you're interested, go check out the release notes for 1.30 and enjoy upgrading.
00:13:02
Speaker
Yeah, I was listening to the Google communities podcast and Kat was on on that podcast with Casslyn and they were talking about why she decided to name this release. And she's like, two or three years back, she tweeted something about it. And now she just wanted to be true to her word. I was like, Okay, that's a good reason. Yeah, yeah. And, you know, who can argue with happiness and cuteness at the end of the day? We're all just masking the reality.
00:13:32
Speaker
No, I love it. And the logo is if you're into cats. Adore. Really, really plays into the name. So great job, Cat. My second thing here is from the Data on Kubernetes community. If you're unfamiliar with that community, definitely go check it out. It's all about anything data related on Kubernetes. So a lot of database talk, a lot of operator talk.
00:14:00
Speaker
They do an entire day down Kubernetes day at every KubeCon, which I think the CFPs are coming out soon. I don't have a link to that. Oh, nice. OK. We will put it in show notes maybe next show or something like that, especially out. But they did have a brand new ambassador program, I think early last year or late last year. I forget when it was. They got some of their ambassadors accepted and into there. And then the applications closed.
00:14:26
Speaker
But the article that I'm talking about is it's now back open. So if you want to become a data on Kubernetes ambassador, definitely go check this link out. The program is officially live as of May 1st and you can, they'll be accepting applicants till May 31st. Okay. And then you'll know by basically middle of June if you are accepted as an ambassador. So really what they're looking for is anyone that's
00:14:53
Speaker
contributing some way and some fashion back to the community. So do you like talking about data on Kubernetes? Do you work with databases quite a lot? Do you write content? Do you host a local meetup that's related? Anyway, anything in that that realm, definitely go check out this community because it's a really fun one. I know Bobbin and I sort of been a part of it for quite some time now. And everyone's really great over there. And they
00:15:21
Speaker
that it's a tight-knit sort of niche community around data. And I know at DOK in Chicago, I wasn't in Paris, but they had amazing talks for the day one event or day zero event, I think. And last year, I think in Paris, the tickets were like part of the whole event. So yeah, definitely go check it out. It's a really fun community. And I'm sure they'd love to have some more ambassadors.
00:15:49
Speaker
You know, I was thinking like we should apply and then like on, on the download, if we don't get it, we never make it public, but then go for it. Yeah, for sure. I'm sure I'll submit an application. Sweet, sweet. Um, that's all I had. Uh, so today we have a really exciting, uh, talk and really exciting guest. Um, his name is Torsten Steinbach. He's the VP and chief architect for analytics and E
00:16:17
Speaker
AI at EDB. A lot of letters there. Yeah, too many acronyms. But we're going to chat with him about everything that, you know, EDB is up to in regards to the company, Postgres and what they're doing in the AI space, all this stuff. So without further ado, let's get trusted on.
00:16:37
Speaker
This episode is brought to you by our friends from Nethopper, a Boston based company founded to help enterprises achieve their cloud infrastructure and application modernization goals. Nethopper is ideal for enterprises looking to accelerate their platform engineering efforts and reduce time to market, reduce cognitive load on developers and leverage junior engineers to manage and support cloud mandated upgrades for a growing number of Kubernetes clusters and application dependencies.
00:17:07
Speaker
Nethopper enables enterprises to jumpstart platform engineering with KAOPS, a cloud-native, GitHub-centric platform framework to help them build their internal developer platforms or IDPs.
Nethopper Advertisement
00:17:20
Speaker
KAOPS is completely agnostic to the Kubernetes distribution or cloud infrastructure being used and supports everything including EKS, GKE, Rancher, OpenShift, etc.
00:17:31
Speaker
Nethopper KAOPS is also available on the AWS marketplace. Learn more about KAOPS and get a free demo by sending them an email to info at nethopper.io or download their free version using mynethopper.com slash auth.
Interview with Torsten Steinbach on AI and Postgres
00:17:46
Speaker
All right. Welcome to the show, Torsten. It's great to have you here on Kubernetes Fights. Give our listeners a little bit about who you are and what you do. Welcome. Oh, thank you very much. Really excited to have the opportunity to
00:18:00
Speaker
to talk here. Yeah, so I'm an architect at EnterpriseDB. Specifically, I'm the chief architect for analytics and AI. So sort of the technical leader for all of the product capabilities that we are building and going to build in EnterpriseDB with and on top of Postgres.
00:18:25
Speaker
Before that, I have 26 years of service at IBM, actually, before I came here. Wow. I worked as architects in different capacities, different products, software, cloud services, always related to data-based analytics, and so on. My last role was a distinguished engineer in IBM Cloud for our data analytic capabilities, our services that we're building at IBM.
00:18:54
Speaker
That's awesome. Did you ever work on like the Watson offering at IBM cloud? Like was that one of your things? When I knew who at IBM is not working on that, because, you know, this brand of course was tech to everything. Yeah. So yes, but what's the portfolio, the core with its natural language and AI capabilities was always something we were sort of attached to. I never worked myself on it, but I was involved in various ways where we adopted it and integrate.
00:19:25
Speaker
As Ryan said, we're glad to have you on the show and you're kind of the perfect person who's going to be building on top of Postgres and inside EDB. I think my next question was going to be around that. Why are we using Postgres in the AI space and what are the plans for EDB in the AI space? How do you see a traditional relational database company being still relevant in the AI space?
Postgres in AI Workloads and Advantages
00:19:52
Speaker
Yeah, that's actually a very interesting question. And also, admittedly, I also went through a journey for myself to get clarity when I took on this job here. But in the meantime, I have a very clear picture of the role of that. And it's actually like this. AI, of course, is about data and data workloads. And
00:20:13
Speaker
Sure, depending on what type of AI you look at, it might be more of unstructured data, documents, images, all of that, which you don't expect in a database. However, the way the AI is conducted nowadays with these large language models often involves or typically involves that you have to provide some context to these LLMs. Context in the sense of this is the domain context that you're coming from. Your documents matter for this
00:20:42
Speaker
interactive chat situation. And finding this context actually is a database workload. It's a so-called vector search workload, which is a genuine database task to basically store and index these vectors. I think we will talk about more that in this talk here. But that's one thing, right? It's generally increasingly database. AI workloads are turning more and more into database workloads.
00:21:11
Speaker
And the other thing, my Postgres is such a good choice here is because you see everybody building this AI solutions nowadays, chatbots, co-pilots, the fancy thing everybody needs to have.
00:21:24
Speaker
It's also experimenting. And most of them are not in production yet. But just imagine what happens when basically the users start to flock to your jetbot because it's providing actual value. It becomes mission critical. And we need something to deploy and operate it with a mission critical quality of service, availability, security, compliance, always on, right? Also no planned outages and things like that in place updates.
00:21:54
Speaker
And you have more of these qualities of service that are relevant and you need a system in the back end that can serve this well. And Postgres, that's a strength of Postgres all along. Not for AI, but for transactional workloads over the past decades. That's where Postgres has matured. It has all these qualities.
00:22:15
Speaker
And that's why it's just, quote unquote, it's not so easy, but it's just, quote unquote, about leveraging these qualities of service of Postgres for AI and making it good, basically mapping it and running it in this Postgres environment. Yeah, absolutely. It's bringing those attributes to this new space, really, at the end of the day.
00:22:34
Speaker
Yeah, and I like that description because today if you look at all the different LLMs and SLMs that are coming out from different vendors, all of them are kind of near the same performance benchmarks, regardless of what benchmarks you're looking at. So it's really, it will be really difficult in the future to differentiate your application or differentiate your organization based on the LLM that you're choosing, because they might end up being plug and play.
00:22:57
Speaker
the data that your organization has in those relational databases, all the context, I think that's going to be the key differentiator for everyone. So I like this approach for sure. Yeah, after all, none of this is possible without lots of data. And you make a good point about bringing like all those other aspects, security and whatnot. I mean, the more and more I hear about some new companies coming out and creating digital workers, and they'll be kind of working on behalf of how people used to and it's like,
00:23:25
Speaker
It's a thought that we have to get that right from the get-go, especially from the security perspective. That's a whole other podcast, but that's a good point. You mentioned vector data already. If anyone's paying attention to the AI space, GIN AI space, anything like that, they might have heard this term,
00:23:47
Speaker
Um, honestly, it was a new term for me back in, uh, a newer term for me back in Chicago. Uh, I believe I went to a few talks and learned more about it. So I'd love to give, uh, or have you give sort of a, um, you know, a, what is vector data and why do we need it to give some background for the listeners?
Understanding Vector Data in AI
00:24:07
Speaker
Yeah, I think the easiest way to look at vector data is, before we look at it, first understand what it physically is. The vector data as the name indicates, it's a sequence of actually numeric numbers, typically floating point numbers, a vector of floating point numbers. Okay, that's what it's technically mathematically seen, right? What does it represent? And that's the trick here. It is a numeric representation as a vector of
00:24:35
Speaker
and dimensions of 1000, 2000 numbers. And these numbers are representing semantics of some other data. That's a trick. It's a semantical, it's a reduction into numerical space representing the semantics of something else like a document, an image or something like that. And that's why
00:24:56
Speaker
Getting to vector embeddings also always involves an LLM because you need to have LLM that can understand, oh, this is what this document sort of means. And let me reduce it to a set of numbers. Yeah. That's already a so-called encoder model. It's one LLM that you're using to compute these embeddings. And now the trick is.
00:25:18
Speaker
Vectors that basically for different documents that are close to each other, right? Geometrically, when you basically compute the distance in an n-dimensional space, they are semantically similar. And that's the whole thing, right? That's why if you have vectors, you can basically treat them as a surrogate of your actual data and do mathematical computations with it.
00:25:42
Speaker
and still basically have semantically and be in proximity to what you actually want to tell the user after all.
00:25:50
Speaker
And basically the more data you have encoded into vector format, the better your end result or output might be, right? Because if you have like sparse data, you might still hallucinate a bit, but if you have a lot of data that is actually embedded in these vectors format, then whenever your LLM is searching something, you might have a close to accurate answer. I don't want to say a hundred percent, but like your accuracy percentage will increase. It's a fuzzy science, indeed AI, which is very clear here. We are talking about approximate
00:26:18
Speaker
the neighbors of these vectors by finding them, which is typically good enough. Of course, I mean, there's a degree here, right? So there's a race to the so-called recall rate when you're looking at vectors. If you have a better recall rate, you have more accurate and finding the really the closest vectors that are most semantically similar.
00:26:39
Speaker
But there's another thing that's maybe defining for vector, which is important to understand because you're coming from a data modality, like an image or a document or some text data, and you reduce it to numbers, you suddenly have something you can bring this together. But that's why you can now basically do embeddings for images and do a text search and find an image with President Obama on it, because basically
00:27:03
Speaker
It has been reduced to the same numeric space and that's the whole trick. It's also a trick that's behind language translations. If you Google Translator whatsoever from English to Spanish, because you have this thing in the middle, which is these vectors, that allows basically the cross between languages here.
00:27:22
Speaker
Interesting. So like one more question, right? So you brought up the Google Translate thing. I know Google has a feature where you can reverse search images, right? You can provide it and imagine it will find where other websites have listed it.
00:27:35
Speaker
And this feature was available before the whole LLM or Gen AI thing. So were they using vector functionality there to do some similar search capabilities? Frankly, I don't know, but I would be surprised if not. I mean, to me, this smells so much like vector, what they are doing there. I think Google was definitely one of the adopters who were also driving a lot of the innovation here and also in the academics, I suspect, for all of these algorithms.
00:28:05
Speaker
Yeah, you know, all right. So the second part of this is really more or less about what's changing in the database space, specifically Postgres for you, right?
00:28:17
Speaker
how has EDB changed to store this type of data, right? So it's, you can't, you know, PG vector is something specifically built for this. What's different about that piece of it? You know, does it change the way it's accessed, how it's stored? Like, why make a whole different thing? Yeah, I would say PG vector is, um,
00:28:42
Speaker
in a way, basically just a normal vector database, very comparable to other options that you have in the market today. Let's say PG vector turns Postgres into a normal vector database comparable to others, other options, including also comparable to the new kits on the block, like the new vector databases, Panko, Nip. So I don't think it's differentiated just per se with PG vector.
00:29:09
Speaker
But the PG vector actually, the existence of PG vector is evidence of something very crucial. It's basically shows how easily extensible Postgres is. And you can also see that PG vector is definitely the key extension for vector search in Postgres, but it's not the only one. But if you look, there are actually others out there. So it's actually healthy competition even going on, because it's so easy to extend Postgres.
00:29:33
Speaker
That's one thing. And now why do you want to extend Postgres versus anything else is because A, these qualities of service that I made the point earlier, you want to have vector database that is always on, that is highly elastic, that you have all kinds of security measures, fine-grained access control, all of the things that over the decades have been built into Postgres.
00:29:56
Speaker
Because of this technology history of Postgres, there is a lot of incumbent adoption and data out there.
00:30:07
Speaker
Very often when you do similarity search, you don't do this in the vacuum. You don't just have a set of documents. These documents basically are belonging to some business context. Like they are, for instance, records of customer interactions, CRM data, and so forth. So they are tied to certain customers, to a certain deal, to a certain geography.
00:30:31
Speaker
All of these properties that I just mentioned is data that you store in a database like Postgres already. By putting in also the vector data there, you can now do very easily combined so-called hybrid searches where you say, I want to search for relevant documents for this customer in that period of time related to this deal or things like that. Hybrid search, that's one reason why
00:30:55
Speaker
An existing database with incumbent business data is very good idea to put vector data there. And you can see this also the new kids on the block, if I may call them this, most of them now, one of the reasons things they have added is so-called metadata because they record. Yes, you cannot just do vector search. You have to provide additional fields that you can filter on.
00:31:16
Speaker
They are also following this need. So the difference is in something like Postgres, well, you don't have to put it into another database. It's already there. So just put all of the vectors there and then you're done.
00:31:29
Speaker
Okay, so if I understand this correctly, right, you have your relational database on Postgres with all of your data already in it in a columnar format. You're adding through an extension like pgVector, you're adding a new column to that table so that you can use vector search capabilities. Is this because if I'm using just SQL based search on my relational database,
00:31:53
Speaker
I might not get an answer back from the engine if it's not an exact match for the primary key versus if I am using a vector search capability, I might get something that's closer. Yes, I think that's definitely part of the story, right? So database of course is meant for exact.
00:32:10
Speaker
matching, while vector search inherently has this approximation in nature and vector indexes. That's how vector indexes are also slightly more complicated than the traditional B3 index and so on.
00:32:25
Speaker
Um, uh, but, but yeah, but in terms of SQL, uh, because we are talking about Postgres, the interface of Postgres is of course, SQL. So also the vector search is SQL for that purpose, PG vector is just extending the SQL syntax and with additional operators.
00:32:44
Speaker
that are representing, oh, this is the closest nearest neighbor according to a certain calculation, different options, how we can calculate proximity in this geometrical space of vectors. And you just say this operator represents this basically distance metric. And then that's, that's the input to Postgres and PG vector. And then under the hood, the actual similar research goes off.
00:33:09
Speaker
Okay. And then, uh, like we are talking about PGA vector a lot, right? But it's an extension is, and you said there are other options also being built in the Postgres community. Is the community working on making this like a default part of the Postgres engine? I know Cassandra, for example, right? Has vector search capabilities when you deploy Cassandra, is Postgres heading in that direction?
00:33:32
Speaker
These discussions are going on and off. My observation currently is there's nothing clearly happening at that it's going to happen. But I would not be surprised if that at some point picks up space. Part of that is also just because the extension framework of Postgres is so good. And that's why basically you can do a lot of things just by using the official extension points. This is not just for
00:34:01
Speaker
such as specifically like vector extensions. But even if you're coming with your custom logic, it's very easy to push custom logic into Postgres. There are very strong language extensions for Python and Rust and things like that. So that's why it's one of the reasons. But I do know a couple of
00:34:19
Speaker
problems that would require that PG vector cannot really solve or an extension cannot resolve. So these discussions are going on. Also, we are engaged here at EDB. I cannot share a lot of details, but we are definitely engaged in some research efforts to see what we can do in the core Postgres site to accelerate vector search further, in particular, accelerate hybrid search further. Like hybrid, again, bringing relational and vector data together in the common query.
00:34:50
Speaker
I, you know, I, so I have a question more maybe about, uh, operationally running vector databases, right. Um, does vector data tend to be, uh, you know, large, do I mean, you know, are, are people running large single instance Postgres, uh, PG vector databases or like, what's the concerns when you're kind of managing vector data?
Managing Vector Data in Postgres
00:35:14
Speaker
You do you still care about scale out and replication and all those things as you normally would.
00:35:19
Speaker
So the surprising thing that, uh, is that vector data, really, if you really just stick with the vector data itself, uh, so these, these arrays of numeric numbers, you can do the math yourself. Uh, even if you're million or even for billion vectors, it's not big data. We're talking about gigabytes and maybe a single digit terabyte at most for large vectors, right?
00:35:45
Speaker
Yes, vector indexing adds some factor, like some of the vector indexes, they blow up, the index is twice the size of the actual vector data. But still, this is not orders of magnitude that really justify scale out yet.
00:35:59
Speaker
At least not for the data. There are some compute level scale-out needed, especially if you're saying, I have a lot of concurrent retrievals going on in this vector data, then it might be beneficial just to do a shared data scale-out, not necessarily to shard it, but to share data and then have more compute power.
00:36:20
Speaker
But that's the thing, vector data itself, many people don't realize it is not big data now. And that's the twist also where we at ADB see also a little bit our opportunity and our goal of what we want to do is
00:36:37
Speaker
For the customer, vector data is something that's just derived from the AI data. It's not the actual payload data. It's helping you to work with the AI data, to find the AI data. But the big problem is where do you store your AI data? Yeah, exactly. And this is big data. This is really terabytes, petabytes of documents, images, video. Yeah.
00:37:01
Speaker
And so the big thing is actually what we think what is needed by a customer is a database or data platform that takes this data and manages this for you and just also does the vectorization and the retrieval for it. But not make vector data a big thing. That's in my mind, one more reason why I don't believe standalone dedicated vector database will prevail.
00:37:26
Speaker
You need to have something that can do more end to end data management for you for AI. And one follow up question, right? Like when you were talking about performance and the query performance specifically on how fast people can get responses, do you see GPUs being really important for running Postgres as well? Like I know Postgres is actually like data stored on a backend storage system, but to access this, do you need GPUs or CPUs are good enough?
00:37:53
Speaker
So there is some, I mean, weird ADB, we're having an eye on that. There is, we don't have anything in our product yet that leverage GPU for this purpose. So there are, for instance, GPU projects going on that allow you to do accelerated vector index search with GPUs. And depending what paper you look at, there are factors of two, three, five, six overall non GPU acceleration that can achieve in certain conditions. Okay, that's relevant. It's not a breakthrough, but it's relevant.
When to Use GPUs in AI Processing
00:38:22
Speaker
I think where we see GPU need coming up much more is what I just alluded to before is when we want to do more in database for your AI data, like you give us your document or your image, and then we go from there and basically make it retrievable. We prepare it to make it retrievable. Like I said before, this
00:38:42
Speaker
for instance, involves creating vector embeddings. And this is an LLM that you're running. So we have suddenly processing need that the database needs to have to run these LLMs for vector embeddings or to reduce data volume to smaller data like a summarization. You may want to run a summarization before you do vectorization. These are all LLMs. And I think that's much more clearly driving GPU needs
00:39:06
Speaker
in a database or at least easily attachable and consumable from a database or context. Right, right. And I like this concept of sort of, you know, building that whole data platform, so to speak, that you spoke of, right, simplifying it for the end user
00:39:24
Speaker
And I know last we spoke before this, we talked a bit about how sort of like AI is helping improve products for end users and vendors or even for EDB yourself, right? And maybe we can switch gears about talking a little bit about how we're sort of either dogfooding or seeing how AI is helping sort of improve those product.
00:39:48
Speaker
Yeah, that's that's a very valid point. It's indeed also something that keeps us busy ourselves in terms of how can we and should we adopt AI. And one is, of course, adopting AI internally, just for our own internal purposes as a company. For instance, just profane things, if you basically want to have a report of
00:40:12
Speaker
What is the capability of this new feature coming up? You can go to the people and try to get it out of them. But increasingly, it's possible to use LLMs to just go via some design documents and things like that and interrogate just this. This is helping a lot basically to make that process development processes and so on more efficient. But we also, when we talk about AI as a strategy for our products,
00:40:42
Speaker
There are actually two pillars. One is we call it AI for Postgres and the other is Postgres for AI. And AI for Postgres just means we use AI to make the Postgres as a product, as a database better, more efficient, easier to use. And these are things that you find many places today, like things like co-pilots.
00:41:01
Speaker
In our case, co-pilot for SQL, or you will find a co-pilot for Oracle migrations, because that's our business model that basically carried this company up to this day, is going after Oracle customers and migrating them to Postgres. That, of course, has a lot of basically SQL conversion needs and things like that. A migration co-pilot can help you with that.
00:41:24
Speaker
AI enabled and using LLMs that suggests you syntax basically and things like that, that works better. Those are such good examples, right? Needing all that knowledge to understand all the SQL commands, right? That's a perfect example of being more efficient. And I feel the same way about Git commands or the AWS CLI. I don't need to memorize all that, right? I really would use something like that, yeah.
00:41:50
Speaker
And also one thing we are really very interested in and really investing in right now is
00:41:56
Speaker
something like a SQL co-pilot that makes SQL much more approachable to new type, to more users. The users that don't either don't understand SQL well enough yet, or they just don't understand your data model yet. You come in with a database, okay, I need a day to understand your data model before I can write a SQL. And we are planning to give you some co-pilot capability that just
00:42:21
Speaker
brings you across this hurdle and say, okay, you come in with natural language. Here's a sequel. Now you can go from there and start editing it. Okay. So is that like, let me dive into that, right? Like, is that going to be part of like the open source contributions that EDB makes? Like I know EDB is very open source focus. I love cloud native PG. Is this part of the open source ecosystem?
00:42:43
Speaker
I mean, some of the things definitely we're going to absorb this specific thing I just mentioned, we have we haven't made up our mind yet. That's still in discussion, whether we open sources and when right. That's what definitely you can expect a lot of us a lot of this from us because the company has been built with this genetics of resource. Yeah.
00:43:06
Speaker
Okay. And are there other things that you, you want to talk about in the open source ecosystem? Like how is EDB collaborating with that ecosystem, especially in the AI space? Yeah. Yeah. AI space is now an interesting thing in terms of open source, because it's not just about open source anymore when it comes in really in the sense of source code, which definitely plays a role here. We are leveraging a lot of open technology, open libraries, um,
00:43:33
Speaker
uh, open, but also open file formats and open models. That's what I wanted to get to. Right. So this is now a new dimension that you did. It's added because AI is about not just algorithms and code that you write yourself or use. It's about this other basically abstraction of some, of some data, which is this model now. And, and.
00:43:54
Speaker
There's basically a very healthy trend and ecosystem building up out there for open models, as we probably are all aware of. But even big commercials are basically making their models available, like llama and so on, on Hugging Face, which is great, right? Which is various, the same kind of dynamics and the same kind of business ecosystem that is evolving, like for open source itself. Yeah, we're definitely all in for that.
00:44:22
Speaker
leveraging all kinds of hacking phase integrations that you can imagine. Yeah. And I know, um, you know, we, I think it was, um, the three of us were having this conversation. Maybe not. I've had a number of them, but it's not just about the models either. Right. It's like open, like you said, it's not just about code either. It could be datasets, right? You know, I know how you based, you can go and look for a dataset that you could use as input.
00:44:47
Speaker
or into a rag system or something like that. And that's shareable as well, right? So there's very different concepts, I think, that data is king and models aren't just the only thing and code's not just the only thing. It's an interesting development. I think it's even... Yes, I mean, definitely you can start with open data, but I believe there's another option now that is on the table is even if your data itself, you don't want to make it available.
00:45:15
Speaker
you now have the option to distill this data into a model and make this model available. If that's basically abstraction that you're okay with, because it may be abstract away some of the confidential details that you don't. Yeah, that's fair point. Well, speaking about data, I know we mentioned this a little bit before about how this AI space is much more about just a vector database, it's much more about just the model, but
EDB's AI Data Platform Strategy
00:45:43
Speaker
really about the full picture of that data platform and kind of getting back to our, you know, how we kind of build out a data lake or a data warehouse, because the reality is there's a lot going on. And maybe you could touch and dive a little deeper on maybe what that full picture looks like and how EDB or Postgres fits into that space, you know, because you can imagine you'd have object, you'd have, you know, all sorts of things. So maybe you kind of
00:46:12
Speaker
give a picture of maybe what's involved there and what you're doing there. So one of the things that I can point to here that is a critical enabling factor for our AI product strategy is our investment into Data Lakehouse.
00:46:30
Speaker
I mean, I cannot tell you about a lot of details, but in fact, some of that has leaked already to some press articles. Something coming up very soon. That's why it's not totally news that I'm telling you that we are going to make a Postgres lake house available.
00:46:44
Speaker
So in Lakehouse, really in the architectural sense, which is an open data lake architecture with this aggregated storage, with object storage, and open formats like Parquet and table formats, iceberg, Delta Lake.
00:47:04
Speaker
and the columnar processing engine, where we're also leveraging OpenTek, by the way, not reinventing the wheel here. So that's sort of something that we're doing for analytic purposes, actually, where basically we sort of say, okay, descriptive analytics, right? OLAP style analytics now through Postgres, giving you ability to do columnar processing with open data formats on a disaggregated storage.
00:47:32
Speaker
But the thing is, that is an architecture that we are going to introduce that is also exactly relevant for AI. I mean, I really mean AI, I mean, also this generative AI where you don't have parquet data necessarily. Of course, you can't do generative AI on structured data, perfectly fine. But the bulk of the use case is about these big data of documents and images and so on.
00:47:54
Speaker
And this architecture of Postgres being able to talk to object storage, this aggregate storage managed data there, is pretty much what we need to achieve what I said earlier, that it's not just about Postgres storing vectors. Postgres also should help you to store your AI data and pipe it through a life cycle until it's getting vectors created and then you can retrieve it. It's a one-stop shop and this lakehouse architecture is a key thing that we are investing in for that.
00:48:24
Speaker
Right. I know Bhavan and I had a recent conversation with a colleague and friend, Johnny, and he was kind of describing this use case, more like a rag use case, but basically how he could drop in PDF documents and
00:48:40
Speaker
That was pretty much the only step and the whole pipeline would kind of process it and just make the vector embeddings and store it in the vector database so that it was just basically available. I think there's so much to that concept and what that pipeline looks for you and your business and how it drives value.
00:49:01
Speaker
There's a lot to be done there still because not a lot of that is necessarily standardized quite yet. No, it's not. I wouldn't call it standards, but some things that people leverage a lot in these application frameworks like LangChain and Alama Index and so on that help you to streamline this process from an application perspective.
00:49:26
Speaker
And basically you feel like you're very productive, very quickly have an application, but it doesn't do anything to the operational side, how you deploy this whole thing and scale it and so on. And that's why we think such kind of automation doesn't belong to a client framework like llama index and it belongs to a data platform.
00:49:45
Speaker
And our blueprint that I'm sort of trying to guide this company also through our journey is we start with a vector database. That's table stake. Everybody understands it now. And basically Postgres is as good as others. And in terms of functional side, but it has these non-functional differentiation with quality of service.
00:50:08
Speaker
But the next evolution after that, that we are after, is what I call an AI database, from a vector database to an AI database, which means you can come with your PDF documents, just dump them in, and it will basically be processed for you and make it retrievable in a similarity search. And sort of a next stage after that, that I envision is, for lack of a better term, I call it the AI data platform, is where you can do more of your
00:50:35
Speaker
AI processing for data generation in database. Like for instance, a prompt. A prompt I can see is a database asset that you can create and manage. And it has certain properties as a data attached, like knowledge data, which is coming from this vector data, like transient data, which is coming from your chat context, but your conversation history, it's more transient nature.
00:51:01
Speaker
And then you have some configuration attached to a prompt like instructions. You behave like a friendly agent, blah, blah, blah. These kinds of things that you find in prompts. All of those things are assets that can be stored and managed. And I think a data platform allows you to do all of that in the platform. Done.
Postgres Lakehouse Architecture Announcement
00:51:20
Speaker
So is that where EDB is going towards building that AI data platform and making this easier for organizations? Are you doing that through partnerships or building that capability in-house? Both. Definitely. We are definitely conscious that we can't do everything ourselves. For instance,
00:51:41
Speaker
we won't find us building out LLM model hosting frameworks and so on. They are much more capable partners that we will be leaning onto. But everything that is both in Postgres itself, like the ability to store data on object storage, this lake house architecture and things like that, that's an organic investment.
00:52:07
Speaker
Even though, I mean, in a way, it's some inorganic piece here. If you look at our history, if you look at what we did last year, there was a small acquisition that we did. It's quite interesting. A small company in October that we acquired called Splitgraph. And it gives you a hint. If you look them up, it gives you a hint what our lake house technology will be about, because that was the thing that we leveraged. It's all based in open tech and data fusion, but
00:52:33
Speaker
Yeah. We are also going in organically on that scale to accelerate our roadmap. Yeah. And like for open Lake, so I'm not an analytics expert by any definition. Uh, what I know is like snowflake is really popular when it comes to people running data lakes are like people are running things in the cloud. Is this openly going to help on-prem customers that really have to control their, where their data is stored and they want to maintain like sovereignty or just make sure that it's inside their own data centers.
00:53:03
Speaker
Right, I mean, a lake house as an architecture doesn't mean it's cloud or even public cloud. It just means that your storage architecture is much more elastic and decoupled from your compute architecture. And that very well can also work on-prem, be it with actual object storage options like MinIO and friends or other storage options like software-defined networks. And the other thing is open
00:53:33
Speaker
open file formats like Parquet and so on. They are also not cloud specific, right? This perfectly works on-prem and the openness here is much more important, less for everybody can access it. But if you have your Parquet and data and so on-prem, you are much more free to mix and match different engines together of different jobs, best engine for the job. That's the beauty of these open table and file formats like Parquet, Iceberg and so on.
00:54:06
Speaker
Great, yeah, that makes sense. Well, I think every time we talk, Torsten, I learn a lot more. But I would love to give some sort of place for our listeners to go and learn more about everything you're talking about. So if you have anywhere that they could go on Postgres community or EDB site, now would be the time to throw it out there. And then we'll make sure to put all the links in the show notes for everybody.
00:54:36
Speaker
I mean, one place indeed, but it just mentioned you can, we will find more of that now actually since very recently is on these Postgres community events. We just started to change the tune a few weeks ago or at one of the Postgres events and all the upcoming Postgres conferences. We are going to of course talk about our new AI strategy.
00:54:59
Speaker
There's also a major announcement coming up in a few weeks. Just stay tuned. Stay tuned. I like it. Yeah, no, but it's really not far out. So that's why I want to tease a little bit here. Watch out. And with that, you will find on our web presence, actually, much more things directly there for AI. All of the things I talked about,
00:55:22
Speaker
I believe even that you will see my face in some videos doing some explainers about that. So that's a good place to start. We are going to publish also around the same time some blueprint examples. AI applications with Postgres get started.
00:55:43
Speaker
We even are planning to publish our very first tech preview of this AI database concept that I mentioned before, just for PDF documents in and we go from there. There will be a tech preview initially for interested customers that basically we need to sign up them on rare request. And that's not far out. That's just a few weeks out.
00:56:07
Speaker
And I think this, Ryan, this just feels like Torsten is setting up for a second follow-up episode when all of this is now. Yeah, we're going to have to follow up on all the new stuff that's coming out. I
EDB's Community Contributions
00:56:17
Speaker
like it. Well, you know, one thing we can also do is when that also comes out, we can kind of backfill on some of the show notes. But yeah, we'd love to have you back on. It really has been a pleasure, Torsten. I really appreciate you coming on Kubernetes Spites here and talking to us.
00:56:31
Speaker
to us about Postgres. I know myself and many others are huge fans of what EDB and Postgres is doing in the community. I think you have a great presence as a company and just as an ambassador, so to speak, in the Kubernetes community. So we really appreciate it and thank you for coming out. Cool. It was a joy to talk to you. Thank you very much for this opportunity. Any time again, you can count me in.
00:56:58
Speaker
All right, Bhavan, I know every time we talk to Torsten, whether it's on the show or outside of the show, I learn a lot. There's just honestly so much going on and Postgres has been such a staple in the community and they're always involved. Really getting his perspective on everything was, I don't know, eye opening to just really how many places they touch in the community and now AI being a big one. So I'd love to hear your takeaways.
00:57:26
Speaker
No, I agree. He's definitely one of those experts that has been doing this for a while. He spent 26 years at IBM before this EDB stint focusing on this domain specifically. He's not a newcomer to the AI and analytics space, so it was good to hear his perspective. Especially when it came to Postgres, we were asking him questions around
00:57:49
Speaker
Why do we need to retrofit something that works? Like, why do we need to retrofit a relational database to serve vector data? And that was, I like the way he described it, right? Because you don't want to migrate or move all of your data that you have today in your organization, inside your organization to a new format. All you need to do is enable that semantic search capability and just visualizing things like, oh, maybe vector data is just stored as a new column. And then I just use a different algorithm to query it instead of SQL.
00:58:17
Speaker
And it just works. I think having that connection really helped me understand how organizations can just use everything that they already have instead of, as you said, that demo use case of uploading PDFs and then converting them into vector embeddings. I can just point it to my existing tables. Yeah, there's just so much involved there too. I think for me, the big part there is that
00:58:42
Speaker
The new kids on the block, as you call them, like the, the Milvus or the chroma that you hear quite often, right? Those are vector databases as well. Um, but companies like EB without that postgres and, and sort of a robust data community or elastic searches, as you mentioned before, or even Cassandra, I think, um, you know, they, they can kind of bring multiple things to the table, right? And at the end of the day, if you're working in this space, you're building out an application or.
00:59:10
Speaker
you know, a new genetic pipeline need more than just the vector database, even though that's a big part of it. But, you know, to his point, there's still all the data that's, you know, has to be retrieved at the end of the day, not just represented as a vector in that vector search. So that was pretty interesting topic. For me, it also made sense, right? Like, because there is an argument to be had that, okay, if you are
00:59:37
Speaker
Why use an older technology? Because you will suffer from a performance perspective. You might not get results as fast. I think the discussions that we had about GPU, CPU, and EDB actually investing in some of those approaches to make sure that the outputs from those vector searches is faster. I think they know that this is a thing. And I would love if, as one of the things that he teased at the end of the episode, if they share some statistics or some performance benchmarks or comparisons, like, oh, if you're using Pinecone, Milvus, any of the other
01:00:07
Speaker
vector database engines that you listed versus just using what you have with an extension. How does that work? I think that would be good to know. Yeah. I mean, to his point, right? When I asked like, how, how are we operating or managing this data as well? I mean, vector data, I mean, generally is a much smaller representation of the actual data. So even that very large scale, you know, as long as you have efficient and fast search,
01:00:34
Speaker
especially if you're already a Postgres user right at the end of the day or need to be user.
01:00:38
Speaker
Like you already have it there. You might have a relationship or you might be part of the community. So it makes a ton of sense. And I imagine Postgres, I mean, I don't know the numbers, but it's such a widely used database that yeah, enabling that feature is just a no brainer. Yes. I mean, some of the announcements that he teased us on purpose, you know, if you're interested in what they are, Bob and I will definitely make sure once they come out in a couple of weeks.
01:01:04
Speaker
to kind of backfill and we'll also cover them on the news on the next episode. So if you want to hear more about them, you know, we'll do our best to learn about them and bring them into the news portion of the next show. Perfect. Cool. Well, I mean, you know, this was a fun episode. We're going to do hopefully a lot more around AI. You know, I think there's so much to cover there that
01:01:33
Speaker
We do want to do our best to get folks on the show that can talk more about this stuff. If you're a listener that is working in this space, even a practitioner who's building out an application on the engineering side or something like that, come talk to us. We'd love to talk to you and get your perspective on sort of like the reality and the weeds and stuff like that about how it's going for you.
01:01:56
Speaker
If you have some ideas and if Emmy and you can also come to us and say I've heard enough about AI do something else for That's fair
01:02:10
Speaker
Uh, well, so we do have, uh, this is the last show we're going to probably offer the Kubernetes community days discount because that is coming up May 22nd, Wednesday, uh, use code Kubernetes bytes to get 10% off on that registration. Um, yeah. The speaker lineup is looking really good. So if you haven't registered yet, make sure you do and use our code. Absolutely. And of course, as always, please share this podcast with your friends and colleagues and family, if they're into that kind of thing.
01:02:40
Speaker
Join our slack, come send us a message, and give us some feedback, good or bad, we'd love to hear it. Well, that brings us to the end of today's episode. I'm Ryan. I'm Bob. And thanks for joining another episode of Kubernetes Bites. Thank you for listening to the Kubernetes Bites podcast.