Transcript
Speaker: James, we're back.
Speaker: Episode number eight.
Speaker: Number eight.
Speaker: You know, we're just long enough for all our predictions about Reddit to come true.
Speaker: JK, they had some more details on their S1 filing this week.
Speaker: Something like they're targeting a $31, $34 stock price.
Speaker: Okay.
Speaker: Well, you know, the funny thing, I was just trying to look up something about the Reddit IPO on Reddit.
Speaker: And there's so many threads about, you know, should we shard Reddit?
Speaker: There's five threads on r slash WallStreetBets on how nobody on that thread wants to buy Reddit.
Speaker: It's kind of like an extended or another, a recurrence of the backlash that a lot of the subreddits did when they, when they started charging for their API.
Speaker: Yeah.
Speaker: As a lot of them like went private, like really affected Reddit traffic, but it's kind of a similar one where...
Speaker: One of the subreddits is like, ah, let's short them.
Speaker: You know, we started posting some clips from our podcast on TikTok.
Speaker: Ooh, yeah.
Speaker: There was literally only one comment from the clip that we posted, which is, Reddit is not profitable.
Speaker: You shouldn't buy it.
Speaker: All right.
Speaker: Well, it sounds like we're reaching our target audience.
Speaker: I guess that's progress.
Speaker: Yeah.
Speaker: Cool.
Speaker: So we have a pretty fun topic for today.
Speaker: This has probably been the hardest topic I've had to research for both the writing and the podcast.
Speaker: We're going to spend a lot of time on NVIDIA.
Speaker: So NVIDIA came up as a topic of conversation from a couple of folks that listen to the podcast.
Speaker: It's an interesting one.
Speaker: I mean, it's been one of the hottest tech stocks.
Speaker: There's the whole, you know, quote unquote, Magnificent 7, which now includes NVIDIA.
Speaker: I don't know if you've looked at their stock price recently, James.
Speaker: Their stock has basically 5X'd in the last five years from, you know, somewhere about $40 in 2019 to about $900 something.
Speaker: Right now, which is pretty insane.
Speaker: I mean, a lot of tech stocks have grown pretty steeply over the last few years, but this is a staggering change in their valuation.
Speaker: And that was one of the things which prompted us to look a little bit more into this topic.
Speaker: Yeah, I'm just looking at the share price on a graph here and starting at six months, it's, oh, that's a pretty good rise over the past, you know, three months of this year already.
Speaker: And then I scroll out to the, you know, like one year of data and it's like an even steeper rise.
Speaker: And then you go to five years of data, you're like, wow, look at this crazy, wow, journey, you know.
Speaker: I know in tech, a lot of compensation comes in the form of stock for the company.
Speaker: So I'm sure there's quite a few happy people at NVIDIA.
Speaker: That's where it's at today.
Speaker: So maybe we should start with some of the history of NVIDIA and how the company got started.
Speaker: For the second half of the topic, we'll dive a little bit more into is there a valuation justified?
Speaker: Again, neither of us are financial analysts.
Speaker: So our commentary is going to be less about is the current valuation, is the number right?
Speaker: Like it's a stock price.
Speaker: Should that be like more or less and things like that?
Speaker: But I think we will talk about this more from like a
Speaker: product and business strategy standpoint of what does it mean for this valuation and what does it mean for the prospects of the company and things like that.
Speaker: So let's start with the history.
Speaker: The company was founded in 1993.
Speaker: It's been about 30 years now.
Speaker: 31 actually, just throwing that out there just on my head.
Speaker: Yeah.
Speaker: So they started primarily with a focus.
Speaker: They realized that there was a lot of opportunity with accelerating, particularly 3D graphics, which were fairly difficult to do.
Speaker: Back in 1999 was the first time when they released this thing called a GPU.
Speaker: It's a graphic processing unit.
Speaker: James, you know a lot more engineering than I do.
Speaker: Dumb explanation, GPU versus CPU.
Speaker: Like what was this whole GPU thing about?
Speaker: So, you know, 12 year old James would be so gushing to tell you about how well your video games look a lot better when you have a dedicated GPU in your computer.
Speaker: But the general sense is the GPU is a little bit more specialized than a CPU.
Speaker: CPUs over years have evolved and gotten better as well, but they're more for like general processing tasks like...
Speaker: Anything you do on your computer, you probably do tons of different things and the processor can handle them.
Speaker: But the GPU is a lot more parallelized.
Speaker: If I quote a number, you might hear that like a CPU has eight cores to do, you know, processing.
Speaker: Well, GPU have 4,000.
Speaker: You know, so like the type of, it's a little bit more specialized where it can't do everything.
Speaker: It's restricted to what it can do, but what it can do is parallel computing and like really fast.
Speaker: That's a good description.
Speaker: I guess I remember, you know, back in the day when we used to think about computers, graphic cards used to be a big deal.
Speaker: The ones, the computers that did have graphic cards were a lot more expensive.
Speaker: And if you were into gaming or even if you were, you know, like a designer, for example, that used heavy Photoshop on your software and things like that,
Speaker: These GPUs used to be a huge value proposition.
Speaker: So...
Speaker: Launch of their GPU in 1999 did unlock a lot of these really interesting applications which were not possible before.
Speaker: Gaming being the most notable one.
Speaker: The other ones that they've talked about is video and movie editing, things like that.
Speaker: And even from game environment creation, Epic Games, for example, as part of their Unreal Engine, they use a lot of this technology for creating and designing of new game environments.
Speaker: And then the next phase after the launch of the GPUs, somewhere about 2006, they launched something called the CUDA architecture, C-U-D-A.
Speaker: That stands for, for my notes, it's Impute Unified Device Architecture.
Speaker: James, what's the deal with CUDA?
Speaker: Like, why was that a big deal in their history?
Speaker: I'm glad you knew what the abbreviation was because I don't actually know it off the top of my head.
Speaker: And so the way I look at it is it's a, we talked about before where the CPU is more general computing, it can do lots of different stuff.
Speaker: And then a GPU would be more specialized to do these kinds of graphics, video rendering, things like that.
Speaker: Well, what CUDA actually lets you do is maybe goes back a little bit, a few steps and it goes back to being more general purpose, but not so general purpose where it's like running an operating system.
Speaker: It's more general purpose in the sense that, okay, let's look at the GPU as a
Speaker: specialized processing unit that I can program with this special language or language is probably the wrong word with CUDA or I could use CUDA to program it and and use its you know thousands of cores and it's on on the graphics card it has its own memory there too and using its onboard memory to do a specific task you know maybe maybe you're a scientist that's
Speaker: you know, running some analysis, running some, doing protein folding or something like that.
Speaker: So you can use CUDA to program the GPU to run something that might not necessarily be graphics related.
Speaker: Yeah, I'm trying to get like a really dumb explanation for myself to understand what is like Kuda actually unlock.
Speaker: And I asked your favorite tool that you're obsessed with, Gemini.
Speaker: It came up with a really cool example, which I thought was actually helpful.
Speaker: But give the example of, let's say you're a baker who is trying to add frosting on top of a cupcake.
Speaker: And CPUs in a lot of ways were kind of linear processing or like fewer cores.
Speaker: But for a simplistic example, it's like you're one baker, you're essentially adding frosting on a cupcake, one cupcake at a time.
Speaker: Versus with a GPU, you can actually do like multiple bakers adding frosting on like multiple cupcakes at a time.
Speaker: And then the analogy it gave for CUDA was CUDA was essentially like
Speaker: If you could write instructions for the baker saying, here's the amount of frosting you need to take per cupcake, and here's how you move your knife so that that's the right way the frosting lands on the cupcake.
Speaker: So that was a really nice and simplistic example of what something like CUDA unlocked with GPUs.
Speaker: Yeah, good simple analogy, better than my description.
Speaker: Well, Gemini really is amazing.
Speaker: Thank you for bringing it up, Viggy.
Speaker: Well, maybe it's time to look for another co-host here.
Speaker: Is Gemini going to replace me?
Speaker: I'm not upset, Gemini.
Speaker: That's cool.
Speaker: And I know you mentioned a couple of those specific examples around like protein folding simulations when I was like reading about the history of NVIDIA.
Speaker: Those were also examples of use cases which were not possible without GPUs before just because of the complication and the amount of compute it involved.
Speaker: So that was definitely like a pretty significant term.
Speaker: change in construct that NVIDIA brought in.
Speaker: The second one that was also interesting to me is they've had a history of interesting acquisitions.
Speaker: They've actually had a large number of acquisitions that they've pursued.
Speaker: I was reading about this company called 3DFX that they acquired, which was one of their bigger competitors back in the day when they actually launched the GPUs and everything.
Speaker: They've acquired another company called Hybrid Graphics, one called Portal Player.
Speaker: They've essentially had a sequence of acquisitions
Speaker: The latest one being they tried to acquire ARM.
Speaker: We'll maybe get to ARM a little bit later, but that was essentially blocked for antitrust reasons.
Speaker: But that's also been like one of the big reasons why NVIDIA has actually managed to keep however much market share they have today.
Speaker: Yeah, some of those names are familiar from back in my I'm obsessed with video games days, but I think I'm just a little too young not to recognize some of them.
Speaker: But like they were like the old cards when I was building computers.
Speaker: But yeah, interesting acquisitions for sure.
Speaker: Yeah, I know we were talking a little bit about this off mic, but you had a good framework for how the focus of what GPUs unlocked has changed from when they were introduced to what it is today.
Speaker: Maybe you can walk us through what your mental framework for that is.
Speaker: Yeah, so I think we touched on CUDA, and I think CUDA was an important moment because it took a step towards a little bit more general purpose parallel programming, and I thought that was helpful.
Speaker: Something that I think was probably common knowledge to think about is video gaming, if that was a major part of NVIDIA's revenue.
Speaker: You know, over time, I think you mentioned CUDA was in 2007.
Speaker: So you imagine 2007, I think the first iPhone probably just came out
Speaker: That was probably the beginning, maybe just before peak desktop computer, because, you know, I think the phone is probably the go-to computing device for regular people.
Speaker: And so they were probably hitting peak desktop computer, desktop gaming by then.
Speaker: And so I think what's interesting about CUDA is that it's programmable.
Speaker: So they kind of have a chance to find escape path where they're not tied to consumer desktop devices.
Speaker: you know, video card demand to continue growing.
Speaker: And, and so I was thinking about that.
Speaker: And one of them, I think that was one of their hype cycles.
Speaker: I think if, if the, the AI stuff hadn't made their, their stock price go so crazy, I think we could, you could probably zoom in and find the point where there was like crypto hype was making their stock price go crazy as well.
Speaker: And the reason I bring up crypto is that crypto is exactly a similar use case where you want to do this parallel computing and it's kind of an arbitrary thing.
Speaker: It's nothing to do with graphics, right?
Speaker: It might have similar different demands than graphics.
Speaker: Maybe it needs more memory or less memory than graphics and more cores and things like that.
Speaker: So I think that's an example of now that they kind of had this
Speaker: general parallel processing kind of platform, they can find another domain to move into and crypto provided one of those.
Speaker: And so I think that was one of the many hype cycles they went through.
Speaker: Sorry, many, not many.
Speaker: And so I think that's another interesting example of them having a mini heyday.
Speaker: That's a good point.
Speaker: The other business line I was also reading about, which is much smaller compared to their other businesses, is their autonomous vehicle platform where they do power a lot of the AI capabilities in cars specifically.
Speaker: Maybe you can talk a little bit more about that.
Speaker: Yeah.
Speaker: So my reaction to that is really I wouldn't be surprised.
Speaker: that computer vision is another area where I think traditionally you would apply GPUs to more performantly access it.
Speaker: And I'm definitely far from an expert, but LIDAR and radar sensors that the autonomous vehicles have on them is probably, you know, you have to understand a huge matrix of information.
Speaker: So it sounds like a good application to, for a GPU style hardware product.
Speaker: So wouldn't, wouldn't be surprised about that, but we think we're on the down
Speaker: the downslope on the hype around autonomous cars.
Speaker: So that's probably why it's not so important to their business.
Speaker: Yeah, that's a good point.
Speaker: And I know we spent like way too much time reading about the history of NVIDIA and some of the older products as we dug into this episode.
Speaker: But I think this was really helpful for me to understand the trajectory of the company and especially, you know, what's changed over the last five years where, you know, pre-GPUs, it was simpler CPU type use cases.
Speaker: With GPUs, it was more of gaming kind of use cases that got unlocked along with some of these video editing, video creation kind of use cases.
Speaker: And then now there is this evolution into AI being one application, but also other similar, you know, specific but compute intense applications, whether it's like crypto, whether it's autonomous vehicles, but AI being the large unlock that they've had.
Speaker: Yeah, I'm curious to see what domain or industry will necessitate them to go more application specific than stay in like Nvidia's playground because I do feel the CUDA allowed them to be more general purpose parallel programming.
Speaker: But at some point, maybe you're looking for better performance by building something even more specific, but the market has to be big enough for someone to want to build that chip that would either Nvidia or competitor to build like a specific
Speaker: let's s ay a protein folding chip.
Speaker: It's an example of a market that probably will never exist, but it would be interesting to see if it ever goes back to being more specific for some of these.
Speaker: That's a good point.
Speaker: So let's bring it back to the NVIDIA of today.
Speaker: So right now, the way that their business is structured, they have four primary businesses that they have invested in.
Speaker: One is what they call the data center business, which primarily is a lot of the AI applications and enabling compute in those applications.
Speaker: There is gaming, which we talked about before.
Speaker: There's something called professional visualization, which is all of these, you know, creating complicated videos and gaming setups and things like that.
Speaker: And then there's automotive, which is small, nascent, but potentially like a big opportunity.
Speaker: The thing that I found really fascinating while...
Speaker: looking into how the revenue has changed.
Speaker: In 2021, the data center business, which is a lot of the AI and AI adjacent use cases, that was about 40% of the revenue in 2021.
Speaker: In 2024, that's roughly 78% of the revenue.
Speaker: Huge job.
Speaker: That's a huge change.
Speaker: Like it's essentially doubled from 40% to 78% in a span of three years.
Speaker: That's a massive change in portfolio and how their business lines have changed.
Speaker: Yeah.
Speaker: And do you know off the top of your head how their revenue changed between those two years?
Speaker: Because I think a discerning listener would ask, did they...
Speaker: The one cannibalized the other.
Speaker: Yeah, that's a good question.
Speaker: I don't have the exact numbers at the top of my head, but it's roughly something to the tune of.
Speaker: I think the revenue in 2021 was about like 16 billion, 17 billion, something like that.
Speaker: And the revenue in the year that ended January 2024 is about 61 billion.
Speaker: So it's about three, three and a half X increase in revenue.
Speaker: So it does look like there's not really been cannibalization into their old businesses, just that this particular data center business has exploded in the last four years.
Speaker: Crazy.
Speaker: Good time to be in video.
Speaker: That is true.
Speaker: And I was also, as part of their earnings report, they also break down the growth rate of their businesses.
Speaker: So this business, the data center business, essentially has a 75% compound scagger, which is like compounded annual growth rate.
Speaker: That's a year over year growth over the last four years.
Speaker: So it's basically grown at like 75% year over year growth.
Speaker: which is pretty incredible for something that's a baseline of, you know, 47 and a half billion, something like that.
Speaker: Yeah, it's got to be high.
Speaker: They're going from 16 to 60 billion.
Speaker: For reference, let's say, let's pretend that their core business was the graphics, the consumer graphics card.
Speaker: Do you have the growth rate on that one?
Speaker: the gaming part of the business.
Speaker: Yes.
Speaker: Yeah, it's basically 11% for the gaming part of the business and 11% for the automotive and like 75% for the data center.
Speaker: Yeah, so like that 11, just a little bit over 10 is kind of like, I guess like a
Speaker: mature growth rate.
Speaker: There's other companies out there that are older than tech companies that have even lower growth rates that are still healthy.
Speaker: So 11 is already kind of high, but then 75 is ridiculous.
Speaker: That's true.
Speaker: The thing that made us maybe dig a little bit more into this topic is
Speaker: How much of this valuation is justified is probably the interesting question here.
Speaker: And again, this is less of financial analysis since it's neither of our forte.
Speaker: But just some back of the envelope math.
Speaker: Nvidia made $61 billion in the year ending Jan 2024.
Speaker: Their current valuation is somewhere about $2.1 trillion.
Speaker: That's roughly like a 35X multiple on the revenues.
Speaker: So their market cap is 35 times the revenue?
Speaker: It's the market cap is 35 times the annual revenue.
Speaker: That's right.
Speaker: And for comparison, AMD, which is one of their biggest competitors in the GPU business, they have roughly 10x multiple.
Speaker: I do think a lot of these multiples, it makes sense to look at them for a specific sector rather than looking at it broadly.
Speaker: So like hardware, for example, semiconductor is like different from a software business.
Speaker: So AMD has a 10x multiple.
Speaker: Intel has a 4x multiple.
Speaker: And NVIDIA has a 35X multiple on revenue.
Speaker: Which is crazy high because you would expect them like the high cost.
Speaker: So they're similar businesses.
Speaker: They should be a similar range.
Speaker: They should probably even have a spectrum of which one's most succeeding.
Speaker: You would expect it to be like similar multiples, but not orders of magnitude.
Speaker: That's true.
Speaker: And interestingly, I know you pushed back on...
Speaker: What does this look like for other industries?
Speaker: So a couple of more data points for comparison.
Speaker: Meta, which has had a fairly phenomenal last year or two.
Speaker: Meta's multiple is about nine and a half X, their revenue multiple.
Speaker: And Apple is about like seven X. And Meta is like primarily a fully software, high margin business.
Speaker: With billions of users.
Speaker: Absolutely.
Speaker: And in an extremely dominant market position.
Speaker: So it is pretty mind-blowing that a hardware company all of a sudden has 35x multiple on revenue.
Speaker: Which definitely is why we're here, why we have this question, you know, from like a fundamental sense.
Speaker: Why does this make sense?
Speaker: Is this a believable valuation?
Speaker: Yeah, absolutely.
Speaker: So let's maybe peel a couple of layers, right?
Speaker: I think for me, the question is like,
Speaker: what do you have to believe for this valuation, justify this valuation?
Speaker: And it probably comes down to a couple of broad things.
Speaker: Like I do think a large part of what is driving that valuation is the 75%
Speaker: annual growth rate that they're seeing on their data center business that's primarily driven by AI.
Speaker: So I do think that there is an assumption of this massive growth rate fueled potentially, like fueled primarily by AI demand that and there's an assumption that this continues for a reasonable amount of time.
Speaker: And then I think the second assumption is also that even if competition catches up over the next few years,
Speaker: there's an assumption that NVIDIA still continues to be in a dominant market position.
Speaker: So I think those would be both interesting layers to start peeling up.
Speaker: Yeah, I do think it's those are the two sides, you know, any other suppliers going to compete with them and take business away from them?
Speaker: And will you and I, you know, continue to have this huge demand to use Gemini or whatever other products out there that are powered by AI?
Speaker: Yeah, absolutely.
Speaker: So let's start with the AI demand.
Speaker: I think it's probably helpful to talk about like, how does AI demand tie into demand for NVIDIA?
Speaker: Can you connect the dots?
Speaker: So I kind of buried the lead here a little bit, I think is the phrase, because we talked about NVIDIA's transition and we talked about how crypto and autonomous vehicles are examples of
Speaker: kind of more a new type of specialized computing that you could do with gpus well so actually ai is the same way at something under the hood of all the ais the ai
Speaker: bubble is how I characterized it before today are there are large language models that are powered another characterization is it was a generative free trained transformer the transformer word in there GPT yeah the transformer word in there is like a specific type of of machine learning or machine learning concept invented by Google and it's it's a
Speaker: a neural network that's underneath it.
Speaker: And this is a lot of extra detail to convince you to believe me.
Speaker: But at the end of the day, what that means is like doing these machine learning, running them, running the models or training them is actually
Speaker: Maybe you could boil it down to just doing a lot of matrix math or a lot of like parallel math and like certain orders and things like that.
Speaker: So that's where the relevance of AI comes to NVIDIA is we alluded to before, you can now kind of more, you can utilize this parallel compute to do arbitrary tasks.
Speaker: And in this case, it'll be AI.
Speaker: That's a good point.
Speaker: And correct my understanding on this.
Speaker: So this is especially true with large language models where something like the GPT model, for example, if I'm not wrong, has like a few billion parameters.
Speaker: So this is different from training like a more simplistic model where with the transistor based like the GPT-esque models, there's a larger demand in terms of how much computers required for training these models.
Speaker: Yeah, I would think so.
Speaker: I'm not enough of ML specialists to describe succinctly.
Speaker: But I would say the kind of things on the hood that get generated when you train this thing, the size is a function of all of its inputs.
Speaker: And so large language models, the L for large means they have a lot of parameters like you talked about.
Speaker: And so the more of those you add in, the bigger the...
Speaker: like the brain that needs to run to do that.
Speaker: And the bigger the brain, the more parallel computing needs to happen.
Speaker: Yeah, that's fair.
Speaker: And I think the dumb explanation that's helped me is the more complicated the compute is, the more there is a need for parallel processing, which kind of ties like why, especially with these large language models, that has accelerated the need for NVIDIA GPS.
Speaker: Yeah, certainly.
Speaker: I think it's a little bit of the trade-off is you can build something more specialized, do it more performant.
Speaker: And so we're like one thing you might, we talked earlier CPU being general purpose and GPU was like specific for graphics.
Speaker: Well, if you keep going along the line at GPU, as we see them as actually wasn't specific for graphics, we figured out a way to make them or Nvidia has make them a little bit more general purpose.
Speaker: But then you can go one step further and you can find some more specialized chips that are, that have,
Speaker: you know, maybe different hardware components, but like a similar architecture.
Speaker: And in fact, NVIDIA does offer different products here, whether it's their GPU, they're not selling GPUs to companies that train largely good models.
Speaker: They're, they're selling like a different product that is kind of adjusted using the same core technology, but adjusted to be dedicated for AI.
Speaker: And so, so it's a little bit more like on the spectrum of general purpose to specialize.
Speaker: It's further down the specialized direction than the GPU is.
Speaker: That's a good point.
Speaker: I guess the part that I was trying to reconcile is how much of this is like hype-based demand that's, you know, shot down versus, you know, does this actually continue to be demand that exists in the long term?
Speaker: I mean, maybe we generally agree that
Speaker: There is no reversing of the AI pace that's happening.
Speaker: There's probably going to be more and more applications of AI and use cases that are coming up in the next decade, a couple of decades.
Speaker: I am curious your take on how much of this do you think is short-term demand spiking, causing the 75% year-over-year growth versus this actually sustaining the next 10 years?
Speaker: Well, we've seen, even in the past two years, three years of just recent LLM craze, we've seen new versions of models come out.
Speaker: So, you know, OpenAI has GPT-3, 3.5, 4, Gemini, previously called Bard, and as a whole, you know, generational thing.
Speaker: And so as these companies have been developing them, these kind of foundational model companies, there's other startups that are not mentioned here.
Speaker: As they generate, they develop more models, they are maybe potentially bigger models, so like even more parameters.
Speaker: And so they'll have even more need for, they'll have need for even more powerful or complex or GPUs or specialized devices.
Speaker: Or will they be developing to find some way that they model becomes more capable, maybe put aside the size of it, maybe they find a way to make it more capable, but it demands even more parallel processing.
Speaker: So I do think there's some demand on the foundational model level of as they improve in capability, whether it's from
Speaker: maintaining more information in their quote unquote brain or being able to do more things at once.
Speaker: I think there's going to be at least a component of demand there from that.
Speaker: I guess the one counterpoint is probably a lot of the large language models that we've talked about this before, where I do think that there's a race to a little bit of commoditization where, you know, maybe a lot of these models are, there's a few big companies, they potentially have access to the same pool of data for the most part.
Speaker: I know that I'm like,
Speaker: more data licensing deals that are coming up, but a lot of the models right now trained on the same data pool.
Speaker: How much of the compute requirement is like
Speaker: There's a wave of these models coming in now.
Speaker: Maybe there is not going to be like 10 other GPT models that come up over the next two years.
Speaker: So I'm curious, like how much of this is short term train a lot of these models, create a lot of these models, put them out.
Speaker: And then the future use cases maybe become like fine tuning these models.
Speaker: Maybe it's like iterating on these, but...
Speaker: It's potentially lesser compute heavy and therefore the demand for Nvidia and GPUs is not hold up.
Speaker: So to be able to answer that, I want to define first two things you do with the models.
Speaker: There's the training you do, which is kind of
Speaker: Like you, you build the model, you know, like you bake it all together and now you have the brain, you make the brain, right?
Speaker: So you train it over like a huge corpus of data and you have a special sauce way of doing stuff and you generate it.
Speaker: And so now you have this thing and then there's what we call inference or that's kind of the term you use to describe when you actually use the model for a task.
Speaker: And so as, as long as we were talking about like the demand from the use case, it'll, it'll come from training a model.
Speaker: a new model, which is a big investment process, or consumers using features that require using the model or using inference.
Speaker: And so I think that's... Just to clarify, folks who are maybe not as deep into this,
Speaker: So training a model is like OpenAI gets access to, you know, all the crawled data in the world.
Speaker: It's like behind the scenes happening passively.
Speaker: The training, like the GPT 3.5 model, the GPT 4 model, they put that out in the world.
Speaker: So that's the training part of it.
Speaker: And then the inference part of it is like when you go to OpenAI.whateverchatgpt.com,
Speaker: And you ask a particular query, the model is not being trained again, but the model is essentially being like accessed or however you want to call it.
Speaker: Executed.
Speaker: Yeah.
Speaker: Okay.
Speaker: So the model is being like executed to generate an inference, which does not require training at that moment.
Speaker: It's essentially using the already trained model and, you know,
Speaker: applying it to your particular query.
Speaker: Exactly.
Speaker: I think that's a good description.
Speaker: There's some maybe nuance in that you'd want to keep retraining the model regularly, let's say a monthly basis to make sure it's using the most up-to-date version.
Speaker: You know, everyone's buying their data for Reddit, make sure you have the latest posts in there.
Speaker: But regardless of these distinctions, both require a...
Speaker: both perform better on a specialized hardware as in they'd be really slow on a cpu they're much better on gpu can you go further to a more specialized piece of hardware and and so yeah so both the training and the inference are the components to the demand and i think we're going to see the demand coming from for training and inference from open ai or google other model providers yep but then
Speaker: the inference demand is going to be driven by your regular boring business needs.
Speaker: Did you replace all your customer service reps with an automated chatbot that can go crazy with a lot of powers?
Speaker: Does that actually turn into a compelling business use case?
Speaker: Can all the promise of AI be able to replace expensive costs for your company actually come to fruition and that demand actually be there?
Speaker: That's what's going to drive the inference part.
Speaker: So let's say the training part of it, there are existing models, there is a bunch of upfront work that's happening now, there is retraining involved for these models, which requires compute, but maybe not as much compute and not at the same frequency.
Speaker: At the current models, that's probably the
Speaker: That's probably the negative take on, that's probably the thing that would drop demand.
Speaker: And then the things that would increase demand are one new models coming up.
Speaker: And if these models require training, maybe there are more specialized use cases.
Speaker: Maybe, you know, even at the applications layer, like Bloomberg, for example, is like developing their own model based on Bloomberg data, stuff like that.
Speaker: So there are like new use cases, which is probably the case for an increase in demand.
Speaker: And then on the inference ones, it's probably fair to assume that more and more of this application, applications that use these models come up.
Speaker: So like the inference demand is definitely going to go up.
Speaker: So maybe the question that would be worthwhile to answer is,
Speaker: Is the inference part of it also as computer intensive?
Speaker: Do you need an NVIDIA GPU to run like inference also?
Speaker: Or is that, can you use that using, you know, like a typical CPU kind of model?
Speaker: This is not precise, man, but I would say the inference part is still needs...
Speaker: still would be slow on a CPU, but it could be use a lesser GPU or something that is out there.
Speaker: You know, there's actually other companies that have developed specialized hardware for this and a common thing that I saw.
Speaker: this is a spoiler for a future component of our discussion, is that often they would have two pieces of hardware.
Speaker: One is their card for training and one is their card for inference.
Speaker: And generally the way it works is the training card would have
Speaker: two quote unquote computing units and the inference card will only have one.
Speaker: So that's kind of that, that's how you can, what's the math there?
Speaker: Is it 50% of the demand?
Speaker: I don't know, but like it is lesser for inference versus training.
Speaker: Got it.
Speaker: Okay.
Speaker: Well, maybe components important for the demand actually just came to my mind is though inference demand for hardware is lower in general because you have fewer for the training needs two and the inference only needs one.
Speaker: You can do training anywhere, but inference you're going to want to do quote unquote on the edge.
Speaker: So like you're going to want to deploy maybe more in total inference chips around the world because you want to be near your users because
Speaker: Though it's slow or compute intensive to do the AI thing, you also want to do it near where the user lives so you don't have this huge latency from the network.
Speaker: So I think that's maybe where they could also sustain demand a little bit more is though they're training less, their inference needs to be happening in France and the US and India and Japan.
Speaker: It has to be everywhere.
Speaker: So these data centers need them.
Speaker: That's a good point.
Speaker: Yep.
Speaker: So let's pivot to the second part, which is competition.
Speaker: You know, right now I know that there's a lot of demand for NVIDIA GPUs.
Speaker: I know every large company has talked about how they are trying to get as many GPUs as they can so that they can unlock some of these applications that they're trying to get out the door.
Speaker: What's your outlook on NVIDIA's competition?
Speaker: Like, who are their competitors today?
Speaker: And how do you think about competition for GPUs in a three to five year horizon?
Speaker: So the main competitors I see for this, for these use cases, is cloud providers.
Speaker: Yeah.
Speaker: I think... So this is like on Microsoft, Azure, Amazon Web Services, Google Cloud Platform?
Speaker: Exactly.
Speaker: I think the leader right now in that list would be Google Cloud Platform, but maybe less so because everyone's using their TPUs, but...
Speaker: Sorry, they're called tensor processing units.
Speaker: It's maybe less because there's a huge demand out there for them specifically, but more that Google invented this technology, right?
Speaker: So they've actually had this developed behind the scenes for a long time.
Speaker: And so they have a, they're on their like fifth generation of their AI chip, let's say.
Speaker: And so they're pretty far along in what they do.
Speaker: And whereas there are other companies, I think Microsoft has like some initial prototypes they've built.
Speaker: But AWS, I think might be in their second generation of their, you know, training and inference chip.
Speaker: And so I do think it would be these cloud providers because the average company probably isn't looking to have their own data center with all these NVIDIA GPUs.
Speaker: So let's take like Google or Amazon and their cloud services as an example.
Speaker: What is stopping them from not using NVIDIA GPUs today?
Speaker: What is stopping them from just going all in and using only their own chips for this?
Speaker: Well, I think there's the dimension of can their own chips compete with NVIDIAs?
Speaker: And I don't have the benchmarks off the top of my head.
Speaker: But it does look like NVIDIA has a significant performance advantage compared to a lot of these.
Speaker: It's hard for me to say that absolutely because I know Google largely only uses TPUs.
Speaker: And why would they do that unless they, it's a, why wouldn't they use more NVIDIA if that were the case?
Speaker: So it's hard for me to say that off the top of my head.
Speaker: So there's that component.
Speaker: And then the other component is maybe more of a practical perspective is a cloud provider is trying to provide cloud services to people
Speaker: the way they lose money is if they have too much hardware for their demand.
Speaker: Whereas NVIDIA, so some extent maybe has this problem, but really they're trying to sell as many as possible.
Speaker: That's how they're going to drive their revenue.
Speaker: The way that the cloud providers drive the revenue is utilization.
Speaker: So they wouldn't,
Speaker: They wouldn't want to keep utilization high.
Speaker: And high utilization means like kind of keeping that amount of GPUs used over amount of GPUs available, like really close to each other.
Speaker: And so they wouldn't want to over-provision their own chips.
Speaker: Assuming their own chips are just as, you know, they could replace those with NVIDIA and vice versa.
Speaker: They'd want to try to like best project how many chips they need.
Speaker: So the way I see that they would still need NVIDIA is
Speaker: They have their own supply chain and develop their chips and have their chips in as their main use case.
Speaker: And then they kind of fill up the top of if they under project the demand with NVIDIA chips is how I could see it evolve.
Speaker: Got it.
Speaker: I think the nuance that was helpful for me to understand is a lot of NVIDIA's demand
Speaker: does not come from like me and you buying from NVIDIA.
Speaker: Like we don't buy from NVIDIA and even most enterprises like don't buy from NVIDIA.
Speaker: If you're using, I don't know, if you're like a random mid-sized company that's trying to build like an LLM-based application on like your software interface, you're not actually talking to NVIDIA and like buying chips from NVIDIA.
Speaker: You're usually, you know, let's say you're using AWS and like AWS is enabling you to use NVIDIA
Speaker: compute which could be powered by Amazon's own chips or it could be powered by NVIDIA chips.
Speaker: So I think that was an important nuance for me to understand which I think helps put into context what you're saying about the motivations of cloud providers where if you are a new cloud provider and you know maybe you've not increased your chip production significantly in the short run it makes sense for you to
Speaker: cater to it using like predictable demand and kind of balance using your own chips versus Nvidia's chips.
Speaker: But I would imagine that if this market becomes more and more solidified as we go, is it fair to say that
Speaker: a lot of these kind of cloud providers as one example, maybe, you know, companies like Meta, which use their own, which are trying to essentially build their own compute, for example.
Speaker: Is it fair to say that all of them would be incentivized essentially to build their own chips?
Speaker: I think so.
Speaker: I think that even they would always want the backup plan to be able to fill in with the video chips, but I do think that
Speaker: they probably know their customers the best, they know their use case the best.
Speaker: And as we've talked about, like going from general purpose to more specialized, the Amazon's version of this chip and Google's version of this chip, they're like one step more specialized for, you know, certain types of use cases.
Speaker: You know, we talk a lot about AI and large language models.
Speaker: Well,
Speaker: When you go from GPU to the more specialized one, you're kind of specializing towards the deep neural nets that power these style of AIs.
Speaker: If, you know, a new discovery happens in the next couple of years around a different way of doing things,
Speaker: The GPU is going to be better at it than these specialized things, unless it's very similar.
Speaker: So there's some risks there that will this demand from their customers be high enough and forever there, or will there ever be a, you know, disrupting technology where everyone's back to GPU again?
Speaker: So that's a little bit of a bullish case for Nvidia is there's always that fallback of, get me back to the GPU.
Speaker: That's a good point.
Speaker: And I think a good example of this is I know Apple, for example, uses their, I don't know if they do it for all of the devices, but for a bunch of their Macs, for example, they have their own chips now, the M1 chips, which are, if I'm not wrong, primarily fully manufactured by Apple.
Speaker: And for, I can imagine you gave the example of like on the edge applications, let's say there is Siri on your phone, which is based on a large language model.
Speaker: You probably want to try to answer the queries that Siri has asked as quickly as possible, maybe, you know, minimize the number of online versus offline kind of use cases.
Speaker: So it would make sense for a lot of these companies to build, you know, specialized chips that are
Speaker: targeting especially some of these more complicated use cases.
Speaker: Yeah, I think the mobile is a good example where you have less resources, so a more specialized chip and a more specialized model can help you get there.
Speaker: And yeah, certainly it would be weird for me to see Apple farm this out to NVIDIA to do their on-device stuff, especially given that I think their desktop chips include some, they say, neural engine.
Speaker: Phones for the past five years are famous for having started to include a lot of things into their
Speaker: into their their chip quote unquote that like image processing or you know video transcript so do you think this is a risk for nvidia if you think about this in a five ten year horizon do you think that a lot of these cloud providers for example building their own ships does that change you know your outlook on how much of the demand that's coming up could actually be captured by nvidia
Speaker: Yeah, I think it's tough because if their valuation wasn't so crazy, I would be like, I don't know.
Speaker: But because their valuation is so crazy, I'm like, yeah, it definitely feels overvalued because they're probably going to retain the customers they get.
Speaker: But I don't know.
Speaker: The growth certainly isn't sustainable in the face of competition.
Speaker: I think it really depends on
Speaker: how big is this demand?
Speaker: I don't know if we actually know, you know?
Speaker: And so are they right now able to get 1% and then, you know, Amazon and Google are each able to get 1% and then there's 97% of the market left?
Speaker: In that case, yes.
Speaker: Crazy multiples probably make sense in valuation and there's no threat in the three to five year horizon.
Speaker: But I think we'll know more for sure as these, you know, I think one of the callback to one of my 2024 predictions is we'll see some level of popping in this AI bubble.
Speaker: And I think that'll be our sign of will there be, is there an issue in NVIDIA's short-term future?
Speaker: Because if we can see, if we see that a lot of this, despite how crazy cool these LLMs are and how much I love chatting with Jim and I to do my menial tasks,
Speaker: it being fun to use as a consumer doesn't make it a business, you know?
Speaker: And so if we see those applications bearing fruit this year, I think that shows that, okay, there's definitely at least some of that demand.
Speaker: But if we start seeing, you know, startups failing and no business applications coming out, you know, suddenly no one ever talks about Microsoft Copilot anymore.
Speaker: I think those are signs that the demand's not going to be there to, you know, warrant as many manufacturers in the space.
Speaker: That's fair.
Speaker: So I guess maybe our general read on this is we would probably expect the AI demand to continue growing for the most part, both on the training side as well as on the inference side, even though there's probably going to be some cool off in terms of amount of compute required for retrain versus like training it for the first time.
Speaker: Generally, it looks like that market is sizing up for the most part.
Speaker: And then the competition part is DPD, but the optimistic case for NVIDIA is that there's enough of a market that competition doesn't really matter in the short run, at least.
Speaker: Yeah, and it really is totally derails the conversation.
Speaker: It's a much more complicated thing to analyze.
Speaker: But from that summary, if those hold true and the competition doesn't actually matter, I think there's other companies out there with a similar expertise that are valued much lower today, like Intel.
Speaker: that has one or two years ago released its first GPU.
Speaker: So they're on that path and they're valued like horribly right now.
Speaker: So is this upside for NVIDIA also there for these other companies that have some experience here?
Speaker: And, you know, AMD is also in the list, but Intel is just valued so lowly right now that it's got my mind racing.
Speaker: Yep, that's fair.
Speaker: So I know we mentioned at the beginning of the episode that this is not a stock analysis and we are not experts at any kind of financial analysis for the most part.
Speaker: But based on everything we've talked so far,
Speaker: Are you buying the Nvidia stock now, James?
Speaker: What's your non-financial analyst read on this?
Speaker: I think my risk tolerance is I'm more on the conservative end of risky choices.
Speaker: So I think I probably wouldn't.
Speaker: I'd much rather be sure I'm buying something when it's low.
Speaker: Of course, this could go crazy and it could be worth it.
Speaker: It's $900 right now.
Speaker: Share price maybe goes up to $1,800.
Speaker: But if it was worth $4 right now, then I would be much more excited about there being an opportunity.
Speaker: So I think I'm too conservative.
Speaker: I think that's fair.
Speaker: What about you?
Speaker: Yeah.
Speaker: And I think I agree with you.
Speaker: I know we've talked a little bit more about investing philosophies and I know neither of us do very specific, you know, picks these stocks and buy kind of thing.
Speaker: I do think that like my approach, which I'm just experimenting and trying to learn more about is trying to identify the
Speaker: companies that have a lot of potential that may be undervalued at this moment because of some kind of externality that's happening.
Speaker: We've talked about Google a little bit.
Speaker: I think that Google is a good example of something where both of us generally agree that even though the sentiment around Google is they were caught off guard around the AI race and all of that, they have the fundamentals of what it takes to be successful.
Speaker: And it is really, really hard to actually replace a search tool completely.
Speaker: So that's probably an example of a stock that I would buy at this point of time.
Speaker: But NVIDIA definitely seems like it's valued based on an optimistic scenario.
Speaker: And my read is it's probably going to take a lot of optimism for this number, this valuation to go further.
Speaker: If we were discussing other stocks, we might discuss, is it correctly valued or is it undervalued?
Speaker: But with NVIDIA, it's, is it correctly valued or overvalued?
Speaker: There's no way it's undervalued from my perspective.
Speaker: Yeah.
Speaker: I was listening to this random podcast by this hedge fund person.
Speaker: I mean, there's this idea of like margin of safety when it comes to investments, which is you don't want to buy a stock that's, you know, quote unquote, fairly valued.
Speaker: you ideally want to buy something with the margin of safety so that, you know, even in a pessimistic scenario, you're not really losing that much money.
Speaker: So I think with that context in mind, like NVIDIA maybe is fairly valued, but it does account for a lot of optimistic things to happen over the next 10 years.
Speaker: So yeah, I think I generally agree with you.
Speaker: Yeah.
Speaker: Don't think my own personal Gemini use will be driving the AI rocket that is in NVIDIA right now.
Speaker: Cool.
Speaker: That was a good discussion.
Speaker: We...
Speaker: Definitely spent a lot of time navigating how to research this topic.
Speaker: It was one of the more complicated topics we've done, given neither of us are experts on hardware and semiconductors.
Speaker: But hopefully this was informative.
Speaker: We are aiming to get episodes and analysis out weekly-ish.
Speaker: You can follow us on Spotify, Apple Podcasts, or YouTube.
Speaker: With that, we'll wrap up and we'll see you again next week.
Speaker: See you.
