Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
That's good Mojo - Creating a Programming Language for an AI world with Chris Lattner
3 Plays4 days ago
What does it take to design a programming language from scratch when the target isn’t just CPUs, but GPUs, accelerators, and the entire AI stack? In this episode, I sit down with legendary language architect Chris Lattner to talk about Mojo — his ambitious attempt to rethink systems programming for the machine learning era.
We trace the arc from LLVM and Clang to Swift and now Mojo, unpacking the lessons Chris has carried forward into this new language. Mojo aims to combine Python’s ergonomics with C-level performance, but the real story is deeper: memory ownership, heterogeneous compute, compile-time metaprogramming, and giving developers precise control over how AI workloads hit silicon.
Chris shares the motivation behind Modular, why today’s AI infrastructure demands new abstractions, and how Mojo fits into a rapidly evolving ecosystem of ML frameworks and hardware backends. We also dig into developer experience, safety vs performance tradeoffs, and what it means to build a language that spans research notebooks all the way down to kernel-level execution.
Brought to you by Mailtrap. Email delivery built for developers: https://l.rw.rw/hanselminutes
Recommended
Transcript
00:00:00
Speaker
LVM, like it's a good thing. I'm a fan, but it's also 25 years old and not a great thing. right And so following 25-year-old compiler technology is actually not even that awesome. LVM is one of the most widely understood ah compilers out there because there's so many people working on and it gets used in a lot of ways, but that doesn't make it the best.
00:00:22
Speaker
And so I've been working on lot of new compilers since then that are way better. Okay, so that's a great question. it's Scott, I want to thank our new sponsor, MailTrap. Modern email delivery for developers.
00:00:34
Speaker
They integrate straight into your code with their SDKs. You get unified transactional and promotional email delivery. 24-7 support. You contact humans, not AI chatbots. We'll give you 3,500 emails monthly in the free tier, and you can try them out at mailtrap.io today. That's mailtrap.io today.
00:00:59
Speaker
Hey friends, I'm Scott Hanselman. This is another episode of Hansel Minutes. Today I'm chatting with Chris Lattner. He's a creator of LLVM, the compiler infrastructure that underpins huge swaths of modern software. And he's also the original architect of Swift, and he's had engineering leadership roles at Apple and Tesla and Google, but now he's at Modular AI, focused on the next generation of AI native programming tools. How are you, sir?
00:01:22
Speaker
I'm doing well, Scott. I'm excited to be here. I'm happy you finally had me on. I'd only took a thousand episodes and you know, well, you have standards, so I can, I understand why I wanted more accomplished people. So I was waiting for your Wikipedia page to scroll more.
00:01:36
Speaker
yeah i mean, maybe if we don't offend you and your audience too badly, you'll have me on again sometime. So I think a lot of things we could talk about. Yeah, it is it is kind of funny, though. You're right. I mean you should have been on 700 or 800 episodes ago.
00:01:50
Speaker
so that is 100% my bad, and I will take that feedback to the team, which is me. um I'm curious, though. I'm thinking about history, and I'm thinking about you know you know doing this for so many years, and you've been doing this for, you know, we're kind of contemporaries.
00:02:07
Speaker
But LLVM was a research project, right? You just kind of had a head a Jones for this when you were a grad student at University of Illinois. Was this a long con? Have you been working on all this stuff for 30 years to get to this moment right now?
00:02:20
Speaker
Well, so, I mean, I wouldn't say it's a con, it's a passion. It's my life's work. That's probably a nice way to put it. Yeah, yeah. I mean, i'm I'm a special kind of nerd, I think, you know. I am very enthralled with compute and with compilers and systems and hardware and chips and like how it all works together, but I really identify as a developer, right? And so I write a lot of code still. You can check me out on GitHub.
00:02:42
Speaker
the um I care about the craft. I care about what it means. I care about people developing in their careers. I care about building teams of people. I care about you know building things and hopefully making an impact on the world.
00:02:53
Speaker
yeah I'm good at certain things. I'm way better at building a compiler than an iPhone app or an HTML page. and But that's cool with me. I just kind of try to be useful. Did you see the AI moment coming? Like, i feel like most a lot of people will say like, as of October, now they can code for us. But like, I suspect you saw this coming a little sooner than October of last year.
00:03:16
Speaker
Yeah, I fell in love with AI or enthralled with AI actually in like 2016. So it's about a decade ago, which makes me, I guess, a pretty old timer at this point. I was right when the Photos app at Apple was getting the ability to see cats and dogs in photos. And I'm like, how do you write for loops to figure out if there's a cat in a picture? And because of that, i just kind of fell down the rabbit hole.
00:03:37
Speaker
What is what is a neural network? What is a convolutional neural net? How does all this kind of stuff work? And that's actually when I was kind of getting kind of bored and ready to go embark on new things. And since then I've been chasing what is AI? What is the developer platform? What are the tools look like? What are the systems? How do we unlock the hardware? And this has been my mission for the last decade now, which is kind of scary how that works. but But did you feel that GPTs were going to be able to spit code out as well as they can now?
00:04:04
Speaker
Well, so the the funny thing about AI, if you've been in the space, is that it's always changing, right? And so being in the AI space necessarily means that you become like acclimated to discomfort. Like you you are used to things changing all around you. You never know what model or capability or demo or whatever will launch next month, right? And so when ChatGPT happened,
00:04:25
Speaker
That was a huge deal. I think that's that was the first wake-up call for a lot of people when they started to realize that AI is a real thing. But ChatGPT, for me, was just the next the next step. And so what's happening, I think, even the last few months is that AI coding has really come onto the scene in a way that people didn't see coming, but it's really also the accumulation of a bunch of exponentials. And so I've been tracking the AI coding thing for over a year now. um I've been using Cursor for my daily driver personally and been experimenting with some of the agentic stuff. And it's amazingly it's amazing what can be done, but it is also more a continuation of the same. And so I see it building. and The thing that we all wrestle with as developers now is what does it mean? How do I best use it? What is hype? What is real? Like, do I still have purpose? And so for these things, like, again, it's not like a new question. It's more about like, what is the purpose? How do we shape our time?
00:05:20
Speaker
How do we best use the tools? And as the tools evolve, I think we we need to adapt. the Speaking of, I think about every era of panics. Like I remember when I went from Assembler to C and some of the old heads would tell me that like, no, no, you got to do Assembler. Like real, real programmers use Assembler. And then when I got color, syntax highlighting, they're like, no, it's going to rot your brain, man. You can't do that.
00:05:42
Speaker
You know, so like there's always every era is in a panic and you know, Oh, you're copy pasting code directly from stack overflow into production. Don't do that. a Recently, Anthropic announced the Claude C compiler and you did a blog post about this just literally today as of the record or yesterday rather the recording of a this, this, um this podcast and I think we have similar takes on it.
00:06:06
Speaker
Like you said, it's an AI building a C compiler is not revolutionary, but it does tell us about like where AI is right now and where it's heading. Yeah. Well, so when, when I bubble out and just a plus one year humans don't like change.
00:06:21
Speaker
So when when I was building Swift, when we came out with Swift, there was a really interesting aha moment for me that I didn't really expect. So when we launched Swift, just contextualized, only about 250 people in the world knew about it.
00:06:35
Speaker
And so because of Apple's secrecy, it turns out most of the software engineers within Apple had no clue. And then it went from not even being on their radar to, what, Apple's switching to a new language? and It was just a big head-exploding moment for a bunch of people.
00:06:48
Speaker
That was fun. But the thing I didn't expect was that the existing community really resisted it. Not everybody, but particularly the people that were super expert at Objective-C, they really did not want a new thing to come onto the scene because they were experts. they They really understood how to use the existing tools. People came to them. They had built their careers as being the guru. And so getting reset back to the same starting point as everybody else, being a Swift newbie, was not something that they wanted. And on the flip side,
00:07:19
Speaker
people coming to Apple platforms for the first time or people that had been in for six months or something like that are like, oh, thank goodness. Now I can actually do way more. It's so much easier. I can grow and scale faster. Now I can close the gap with the experienced people. And and it's not that they're against the experienced people, but they're just, they want to get more done. I see that pattern playing out exactly today, right? Where you have a lot of people that are taking these very diametrically opposed opinions, either AI is completely nonsense or, oh my God, we're all doomed.
00:07:47
Speaker
But really there's a middle path. like neither Neither of those is right. It's not that we're all doomed because of AI. It's not it's not that ah AI is stupid. The question is, we have these amazingly powerful tools, how do we use them?
00:08:00
Speaker
And some people will deny them and say, okay, well, this is nonsense. But other people will upskill very quickly. And the people that adopt the tools and figure out the best ways to use them, figure out both AI, the good way to use it and not the bad way to use it. Because there's a lot of learning that we have to do. like those Those are the folks i think that will grow quickly, that will have new opportunities, and that their careers will be progress will progress faster and things like this. And so to me, I always look at, you not every new technology should be adopted necessarily, but this one is so obviously profound.
00:08:31
Speaker
It becomes a new tool. It's just like source control 20 years ago or something. Yeah. Profound for all things or profound for software engineering? Or do you have different levels of?
00:08:42
Speaker
Yeah, the art of software engineering. and I think that you and I love building things, right? And so the um I don't see building software as writing code. Writing code is one thing. That that is that is an issue, and and a lot of people struggle with that. I'm at the phase where I have flow moments of writing code. like Code, the syntax, like the the art of like hacking stuff out on a keyboard isn't actually challenge for me, which good news, I'm a senior developer or something. But but that's not true for everybody.
00:09:09
Speaker
right And so again, I think that a lot of people associate coding with the writing of code. But when you bubble out and you think about, what are we trying to achieve?
00:09:20
Speaker
oh What are we building? Why? What does better look like? What does the architecture look like? What does good design look like? These questions are things that I think are purely human. they They are something that we are innately responsible for. And I think that a lot of people, again, the extremists say, oh, well, when AI means you don't have to know how anything works, right? The AI will just write the machine code for you.
00:09:41
Speaker
When AGI happens, all this stuff blah blah, blah, blah, blah. Like that train of thought it completely dismisses the fact that it has not happened and it may not happen. And so I like to live in the real world. And so the way I kind of contextualize and look at this moment that we're in is how do we best use this technology, right? Because it is profound. It does allow us to accelerate a ton of stuff.
00:10:01
Speaker
As an experienced coder, it means i can get out of doing a lot of the boilerplate mechanical stuff, but for other people that are earlier in their journey, it means that they can be way more productive and can close a lot of gaps. so Yeah. i was I think about it in the context of if you come at this with code is art,
00:10:19
Speaker
and the AI is stealing my art from me, that would hurt a certain way. I don't like AI-generated images, I don't like AI-generated video, and I would not use an AI to generate poetry.
00:10:31
Speaker
So if, but if the job of poetry is to evoke emotion, then I could see where one could argue in some philosophy, you know, to a one class that like, well, I evoked an emotion. doesn't really matter how I did it.
00:10:41
Speaker
That doesn't, that's not the purpose of poetry in my opinion. Well, it depends on your goal. And I can't say that everybody's goal is the same, right? But is goal of writing poetry to get paid for it? If so, then accelerating the production of poetry is probably a good thing. But isn't that poetry slop?
00:10:57
Speaker
But if if the goal if the goal of writing poetry is to enjoy the art of building and discovering and making the poetry, then that's like saying, hey, Scott is a woodworker.
00:11:08
Speaker
Like using a hand plane is like actually a very fulfilling thing to do. Probably not what you're going to do in a cabinet shop where you're volume producing things, right? But it doesn't mean that one is good or the other is bad. It's about what is the right tool for the job.
00:11:21
Speaker
Right. And then that's where things get get artificially broken down into ones and zeros. And by ones and zeros, I mean there's either IKEA or there's the Yankee workshop and there's nothing in between.
00:11:33
Speaker
Or it's like code is poetry or it's AI slop and there's nothing in between. And so I think that the thing that we're both channeling is that we're both artists when it comes to code. Like we think of ours. Yeah, for sure. Yeah. And we care about the craft and things like this.
00:11:50
Speaker
I care about that for two different reasons, actually. One is, and this is something i think that people hugely gloss over in the AI discussion is that outcomes are not the only thing that matters. It turns out that, in my opinion, building a software artifact, building something in code, is not about solving today's problem. It's about building an investment in a technology.
00:12:11
Speaker
And the thing that's always true about software is that the the requirements change all the time. And so what you need is you need both to have the outcome, but then you also need to be able to move it quickly, adapt to change, add new features,
00:12:24
Speaker
move into the new economy, add the mobile features, like whatever the thing is that you're doing, right? And so to do that, you need not just an outcome-oriented, okay, the the code appears to pass the unit test, but you need people that can manage that, that can work with it, that can understand the considerations, that can make a judgment about where to invest and how to achieve things. And for that, you need a really visceral kind of understanding of how this stuff works.
00:12:50
Speaker
um I think the AI, and again, this this gets back into the AI is very powerful, but it's a tool. It can encourage really sloppy work. It can encourage laziness. It can encourage a lot of really bad things. I think that we we as a team it can can certainly encourage people wasting other people's time on the team.
00:13:08
Speaker
If I've vibeve coded this thing, go review it for me. right that's That's not actually a good outcome. And so I think we need to figure out how to best use these tools. Okay. So putting all of this into the context of this the C compiler, which you can again wrote about recently, you call out a lot of things that I very much agree with, like good software depends on judgment. It feels like human judgment is the only thing that matters right now. And, you know, good taste or code smell, as we would have called it in the past, matters.
00:13:35
Speaker
is I'll just ask you out straight, is this a good C compiler? Did they just one shot a C compiler? Or is it a little more nuanced than that? Because it's a long article you wrote. Yeah, well, so it's not a one-shot C compiler. They burn $10,000 worth of tokens over the course of two weeks. And so it's definitely a a very agent swarm iterative thing, right? And if you look at this kind of agent swarm building against a objective function, it kind of looks like training in neural net itself. Like, because you have a loss function of like how many tests fail and then you're, you're hill climbing into that.
00:14:08
Speaker
Is it good? I think it's extremely impressive a, a um What they achieved in terms of zero human input and proving that you can match to an objective function, I don't consider that to be wildly surprising. That's what AI is good for. Like, in if you look at what happened, basically, transcoded LLVM, GCC, other C compilers that it has in its training set into Rust.
00:14:34
Speaker
And so it uses the language translation abilities of transformers very effectively. But again, AI is good at that. I didn't see anything novel. I didn't see anything where i'm like, wow, that's a good idea.
00:14:45
Speaker
doesn't that make sense though? Because i I've been using it. Like everyone knows that Claude, as an example, in Opus makes websites of a certain style. So I've skinned a couple of my websites and you can see the kind of pieces of Claude where it's like, oh, they love a text gradient and their their light modes and their dark modes look the same. And I asked my son about my website and he says, it looks mid.
00:15:06
Speaker
because he's young and everything's mid when you're 18. But mid is the statistical fat part of the bell curve. So people are like, oh man, they stole that and there's LLVM and they stole some GCC codes, total ripoff. and it's like, well, no, it's it's the one that we all use. Like I would look at it if I were to do a compiler tomorrow. Like it's mid, not in a bad way, in the fat part of the bell curve way. It's expected. You would have been surprised If it had come up with something novel, because it's not trained on novel, it's trained on the standard.
00:15:39
Speaker
Well, absolutely. I completely agree with you. And i both agree with you. And then I'll lean in further, right? Which is AI and LLMs are distribution followers. Right. And so they're finding that midpoint in the distribution and they can rapidly follow that. They can do some air quote, innovative stuff in limited spaces, but, um but really that's what they're designed for. But let me also give you another hot take LVM.
00:16:02
Speaker
Like it's a good thing. I'm a fan, but it's also 25 years old not a great thing. Right. And so following 25 year old compiler technology is actually not even that awesome. LVM is one of the most widely understood ah compilers out there because there's so many people working on and it gets used in a lot of ways, but that doesn't make it the best.
00:16:23
Speaker
And so I've been working on a lot of new compilers since then that are way better. Okay. So that's a great point. very interesting to see that. Yeah, because there's a mid-bias, and it's like, yes, I'm going to use the last 25 years of work, but is it interesting that it didn't discover and think about Mojo and the work that you're doing there? and why did it Or MLIR, or these things that are much more new. And so this is also where it's very funny. ah The LLVM community is actively working on getting rid of this thing called Get Element Pointer out of LLVM. And so it's...
00:16:54
Speaker
Totally in the design for the Cloud C compiler. And so, again, this you can project forward and say, well, teams that proactively and actively adopt LLM-based code gen in the absence of judgment, right are they actually going to be held back?
00:17:12
Speaker
Because they're not even going to be at the forefront of tech. They're going to be, don't know, probably not 25 years behind, but They're gonna be not not getting the best technology and the best outcome for the product. Now, I think the mediator on that is that not all software needs to be quote, the best or the industry leader. It just needs to be effective. And so yeah there's probably space for all these things.
00:17:33
Speaker
Yeah, and there's arguably like they had a very well understood goal. They had a test yeah they laid out, like making one. Like you you said in your blog post that it's something that a good quality team of undergraduates would come together and get a B on or something like that. Like it's pretty pretty decent. Well, they'd probably get an A. You think they'd get an It is quite a lot of work, but – But yeah, I mean, I think it's it's super interesting as an artifact. But this is also where when people jump to, it's the end of times and humans are obsolete and things like this. It's obviously a hyperbole. And it's I don't think it's constructive. And I think it distracts from where the actual value is.
00:18:06
Speaker
Yeah, I think i do I do agree that every time some president or or you know some billionaire person says, oh yeah, all all of these jobs will be gone in 12 months, that's probably not helpful for anyone to be saying stuff like that. Yeah, it's well, and it's not backed up by data. yeah I mean, there are more programmers now than ever. The job market is actually quite hot.
00:18:26
Speaker
A lot of people are scared and there's a lot of questions, right? But it's more about projecting forward. And so this is also the question of like, okay, when AGI happens, dot, dot, dot.
00:18:37
Speaker
Well, you can you can you can speculate whatever you want, but AGI hasn't happened and nobody can even define AGI. So I think that there's a lot of work to be done between now and that theoretical future. Yeah. And will next token prediction and all the things behind it lead us into like a really smart,
00:18:54
Speaker
fella or you know gal that's like we can talk to. But also, I mean, again, i look at ai as closing gaps, right? If you don't know what radix sort is or some exotic data structure or something like that, you don't know what a Btree is, well, cool. You can say, hey i like I have this problem. What is the best way to solve it? And it can come up with some exotic data structure and maybe you have no idea and it magically lifts you and fills in something that you didn't know. And that's incredibly powerful. That can lead to much better software.
00:19:24
Speaker
And so it's not BS, right? But it's also not magic. And so that's the nuance you have to find. Well, that's a great point. I like to always abuse Arthur C. Clarke and say that like each additional layer of abstraction is indistinguishable from magic. And like the stack is pretty deep now. And if you're if you're a React programmer, the world's magic for you. And like maybe you don't even know that V8 exists as a concept. And you know people tease me for my Altair's and my PDP 11's, but it's a way of staying grounded just as you going and working on woodworking as opposed to going to Ikea as a way of you being grounded right as well.
00:19:56
Speaker
Well, the analog world's also pretty cool. so Yeah, it is pretty awesome. Knowing how things work is pretty cool, too. And everything's a conspiracy when you don't know how things work. One other thing that I think worth noting, of course, is that they they targeted CPUs. But I know that you believe that we're not necessarily in a CPU-centric world right now and that we've got these perfectly cromulent GPUs that are out there and so few ways to talk to them. But you're going to crack that.
00:20:21
Speaker
Yeah, so i mean if you extrapolate even more, like forget about AI, go back to humans. because i um'm I admit I'm kind of fond of humans. I don't know about you. I think humans are actually the more the more important thing here, particularly at writ large. um So many humans are worried about CPUs.
00:20:37
Speaker
CPUs are cool. They're very important. they like They run a ton of compute, but that's not where the billions of dollars of CapEx are going. That's not where... All the big developments are happening.
00:20:48
Speaker
And what's happened is that you know kind of you can peg it in different ways, but more as lot ended. We're not getting free single core, single thread CPU performance like we used to back in the day. yeah And so we went from single core machines to you know two cores or four cores. you but Now a cell phone has eight cores. Woo-hoo. But now you have...
00:21:10
Speaker
GPUs with hundreds or thousands of cores. You have ASICs that are designed for specific workloads. And let me tell you, Rust does not run on Cerebrus system. That's not a thing. And so to me, I think it's really interesting. And you go back to the 2016, 2017 time for me, it wasn't just about AI as interesting. it became a, well, how do you do this effectively?
00:21:32
Speaker
And I you know got AI pilled way back in the day because of the things that only AI could do back then, you know started by seeing dogs and cats. But then it turned into this really interesting form of compute.
00:21:43
Speaker
And one of the things that I think is just sitting there in plain sight that I'm looking at all day long that very few other people see is that, well, rewriting C code into Rust might be fun, right? But that's not actually moving the world forward. Meanwhile, we have all these crazy GPUs and all this compute out there that nobody knows how to program.
00:22:02
Speaker
And so to me, my mission, current mission, is saying, like let's crack that open. Let's make it so it's way more accessible. Let's make it so that the tools are actually a joy to use. And by the way even CPUs are crazy complicated vector SIMD. They have tensor cores on your CPU, all this stuff. And most programming languages can't even acknowledge or spell float 4.0.
00:22:23
Speaker
or float eight or things like this. And if you're using languages for 15 years ago, which all of at least Swift that I worked on and Rust and TypeScript all kind of came from 2010. Well, then how are you programming modern hardware and CPUs?
00:22:41
Speaker
There's just not a good answer there. And so this is where I think we we all benefit from leaning into this and solving this. And that's my current mission. This leads us to Mojo and Modular and what we're doing here.
00:22:52
Speaker
I hope I can ask ask this right because I'm trying to see the big picture and my own personal context window is still quite limited. But when I teased you earlier about the long con, I mean that in a positive way. By the by the long con, I mean life's work is like you can kind of see down a tunnel and you're like, you know, You built LLVM, now you've got multi-level intermediate representation, MLIR. It's not the intermediate language from.NET. It's not bytecode, but it's a way to have a hybrid intermediate language that can have different representations and also be specific to certain hardware.
00:23:25
Speaker
yeah So did you know that you were kind of building these stair steps and then stepping on them and you knew what you were reaching for? Because this is a long series of steps. and Two to five years goes faster than you might think. so but to look back and go, yeah, that's exactly as I planned. You know what i mean? Because you're building Mojo on top of MLIR, which is you know part of LLVM, yeah which is part of, which is part of, we part of like it's it's it's it's thoughtful turtles all the way down.
00:23:54
Speaker
Well, so so my journey is is, how should I say this? like Not satisfied with the status quo ever. like I'm always pushing and kind of i mean Part of the reason I left Apple was I was bored.
00:24:07
Speaker
right and It was not exciting. and I wanted to be learning and growing and doing new things. and so Amazing. i love the Apple team. I love my position and love Swift. is It's a good thing. but But also, there's a whole world out there that's happening and changing. and so What I've been kind of on is this quest of figuring this stuff out. and so AI, i i figure I feel like I finally understand what's going on with compute.
00:24:31
Speaker
Finally, now. This is the time. Finally, 10 years in, right? And what we need is LVM, yeah but for ai chips, basically. Like we need we need a way to program it that scales across all the silicon.
00:24:45
Speaker
We need something that's easy to use, that's familiar to people, and we need people to be able to adopt this, which means good tools that a good experience and easy to use and like all these things that are consistent across DevTools, but in their own context.
00:24:59
Speaker
And so what that journey is, what that bridge is, is the world's best way to program CPUs, because that's what everybody's on. A lot of people are in Python, for example, and C++, plus plus and that's the way things are. and But none of these things are actually that great either.
00:25:14
Speaker
my humble opinion. And so if you can meet people there, you can lift them, then get them onto GPUs, get them onto ASICs and give a progressive path. I believe that we can actually crack open this gigantic compute problem.
00:25:27
Speaker
And part of my core hypothesis, which has been consistent for years now, is that if you look forward at any point time, five years from now, like compute's are only going to be weirder. It's only going to be more specialized, more heterogeneous, more chaotic, more fragmented. And so we as a programmer community need to have a the ability to build that and use that because you don't want to use 52 different compilers from all the different vendors that don't work together. They don't even speak the same language. like not to like pull out a specific example, but it's like, I got this brand new Windows arm machine. I was so excited. gonna do this. gonna, I'm gonna take over the world with my Windows arm laptop. And it's like, okay, I'm doing some Python. Now we're gonna do some computer vision. And it's like, oh, now I just brought a C compiler along for the ride. And oh, that one doesn't support arm. And oh, this wheel doesn't do this and that. it's just like, oh man.
00:26:15
Speaker
I just want to do a thing. i want a hot dog or not a hot dog. This was not a hard hello world. and Then suddenly it's supposed to be the stack leaked. and Because I was on a new processor or a new NPU or a new this, I was juggling literally Python and C-sharp and Rust and then like wondering where CUDA was and da-da-da-da-da. Like you say, it all works until it almost doesn't.
00:26:37
Speaker
This is the problem with these deep deep piles of layered abstractions where none of the layers knows how the other layer works. It's beautiful in the demo, but then it falls apart when you do anything weird. Let me give you an analogy because you and I come from a similar epoch.
00:26:51
Speaker
Most people forget what the world was like before GCC. Oh yeah. Right. And they don't know what autoconf was or why it existed. yeah But if you go, if you go back in time, everybody building CPUs had their own C compiler.
00:27:05
Speaker
yep They didn't have AI to build it for them either. right And so you got the DEC C compiler and the sequence C compiler and the borderline C compiler and the ah Sun and the SGI and the HP UX and all these different C compilers.
00:27:22
Speaker
ah C wasn't even really standardized at that point either. There was a vague understanding what it was. And so none of them actually worked together. And what drove the world forward is when GCC came on the scene, it was free and it was portable and it kind of scorched earth all this proprietary nonsense and he unified the world.
00:27:37
Speaker
And as a consequence of that happening, Linux was possible. like An entire ecosystem like exploded in vibrancy and like capability because people were able to move on with life and get get rid of all this weird junk. And so what I see today in the hardware is exactly the same thing. I see every hardware maker building their own stacks out of necessity.
00:27:59
Speaker
They don't really want to do all this stuff. It's actually really hard and expensive and slow. Some of them end up with a better thing than some of the other people, but none of them is compatible, right? And particularly in the AI and the GPU space, everybody's trying to take NVIDIA's lunch to certain extent, but CUDA doesn't run their stuff and their stuff doesn't run on NVIDIA.
00:28:20
Speaker
And so a lot of what our aha moment is is, saying several different factors have to come together to crack this open. One is there's the technical pieces. You know, you can talk about ah being able to target GPUs and unlock the for full performance of silicon and scale from CPUs GPUs, all that kind of stuff.
00:28:39
Speaker
But the other is incentive structure. You need an organization, which is modular, that is not tied to a chip. It's not tied to a hyperscaler, not tied to an LLM, not tied to autonomous car, right? I dont want to make sure people understand years you're talking modular the proper noun. You're saying modular, you're a company, not a company a modular company. company, my mission is...
00:29:01
Speaker
form because we need to solve this problem of having a team of people that could go and build this stuff. But then we also had to solve another problem, which is an economic problem, which is this is a really hard problem.
00:29:13
Speaker
You need a very large team. You need years to do this work. you're moing You're not in the pocket of like NVIDIA or the pocket of these people, right? Which would be able to provide all of those resources, but then they would make the the default be the NVIDIA chip.
00:29:26
Speaker
That's right. Well, and so I was at Google, for example, and i was trying to solve and crack and i learned a tremendous amount from Google working on TPUs and scaling their software stack. Amazing team, amazing people, but it it wasn't nefarious. Google's a very open source friendly community and that's actually where we built and released MLIR, donated to LLVM. Google's amazing and lots and lots and lots of different ways. But like any business, they have their priorities, they have their internal workloads, and of course all your best people always get sucked into the fire drill of the day. They also at the time had a performance review process that ticked every six months. And so projects that took longer than six months to do pretty hard to justify both up up the chain, but much more importantly to anybody that wants to get good performance reviews and wants to get, you know, demonstrate their impact on key workloads and things like this. And so I found it to be extremely difficult to justify that work. And I'm a pretty stubborn, pretty bullheaded guy. And so like, yes, was able to build MLIR, was able to build a new runtime for TPUs, this PJRT thing that underlies jacks and all that kind of stuff, and made some major contributions to the stack moving beyond what they had. But it was very, very, very difficult.
00:30:40
Speaker
And so this is this is actually a pretty significant challenge when you want to do something that is like a fundamental step forward in terms of languages, systems, particularly in the AI space. that You couldn't, you could have just taken the language and just pick one and just made rust plus plus you could have made swift, super swift, but you didn't do that.
00:31:02
Speaker
That's right. Cause none them, they were all designed for CPUs and everyone's thinking about CPUs. Cause that's what everyone had in 2010 when they were thinking about these languages. Well, so so funny story, when we started Modular, which is now over four years ago, ah my co-founder and I looked at each other and we said, we're definitely not doing a language.
00:31:20
Speaker
Everybody knows the languages are a terrible idea. We have a lot of stuff to get done. like We have a big mission of unifying compute and democratizing and locking all this compute. But Everybody knows that's a bad idea. And so we spent almost a year working on hardcore runtimes, low-level thread pools, asynchronous communication primitives, and then dove into the, okay, how do we generate high-performance kernels first for CPUs?
00:31:48
Speaker
And so in that case, we said, how do we go meet and beat Intel MKL and Intel's libraries on their own chips? Can we beat them? We need to be able to do that. And we need to be able to show generalization and scale.
00:32:00
Speaker
And so what we did was we built a whole bunch of really fancy compiler technology using MLR from first principles. We started with an empty Git repo and built an entirely new construct and proved that we could generate very highly parametric, very powerful, very high leverage abstractions for generating high performance matrix multiplications and similar algorithms, but then ran directly in the problem of, okay, well, nobody wants to write compiler i r by hand.
00:32:25
Speaker
That's actually a really painful and very terrible idea. And so then we said, okay, let's go shopping. What syntax do we want? And I did look at Swift. I did look at Rust. I did look at Clang. Like I know many of these systems, but to the point, they don't solve the problem. Like saying, okay, how are we going to add all the floating point data types to c well, AI has seven forms of float eight, it feels like, all kinds of new forms of float four, Bfloat 16, like all of this stuff, and it's evolving at a rate that you can't just keep hard coding this into the compiler. And so he said, okay, well, clearly we need library extensibility.
00:33:02
Speaker
And so Mojo, the way it turned out is like, yeah, almost everything's in the library, not in the compiler. We also needed the ability to to scale across them you know predictability and compile time and all these things that really matter at scale for developer experience.
00:33:16
Speaker
And so these Python DSLs we looked at really were unsatisfactory. And so despite not wanting to do this, we basically, kind of as a team, looked at this and said, OK, well, Building a language is heck of a lot of work.
00:33:28
Speaker
It's also not rocket science. Like, I've done this before. ah Swift is a good thing, for example, but like LVM, there's a lot of Swift the bad parts, and we can do something that learns from that experience and do something that really moves the world forward. And though it would be a long journey, we thought that, you know, the stakes and compute and the future of all these architectures are so important that it was worth the investment.
00:33:51
Speaker
Yeah, it feels like if you want to get something done today, you end up bouncing around from language to language. You prototype it in Python, then you rewrite it in C++ plus plus and CUDA, and then you bunch it do a bunch of bindings, then you end up maintaining two code bases.
00:34:03
Speaker
So my kind of like armchair my armchair quarterback analysis is that you kind of wanted Python ergonomics. You kind of want C and C++ plus plus perf. You like Swift-style safety, but then you need awareness of things that didn't exist 10 years ago, like AI hardware. That's right.
00:34:21
Speaker
all of those things combined. and It's not really. While you're at it, bring in some cool new ideas like linear independent types. Let's bring in like much better ownership and memory destruction behavior and things like this. And let's move the world forward because, you know, again, people like to zero in on Rust with ownership and stuff like this, but Rust is 15 years old. It's not, it's, it's, it's, it's a wonderful thing for what it does, but it's not the best thing.
00:34:45
Speaker
Yeah. Yeah. Interesting. Interesting. Okay. So then how do i like this seems like this is the vision, right? And surely this is a vision that's many, many years. This will come out in a half a decade or a decade, right? When am i when do I get to actually have this though?
00:34:59
Speaker
ah So this exists today. A lot of people are actually using this. people It's in production, which is pretty cool. It's in prod now. That's awesome. Oh, yeah. Yeah. We have customer workloads that are running all this stuff. it's It's not a theory. Today, you can run Mojo. You can download, pip install it.
00:35:12
Speaker
You can run it on wide variety of CPUs, embedded devices like Raspberry Pis and stuff like that. It's fine. Okay. Runs on NVIDIA Pro. H100, B100, and a ton of their consumer cards runs on AMD, the 300, 355, their RDNA cards, which are the consumer ones, um and Apple GPU.
00:35:32
Speaker
And so we have other stuff that we'll announce later, but this isn't a theoretical thing. This actually works. And one of the things that's really cool about it is, you know zoom back to the modern world and AI, not only is it possible to do this stuff, but we've unified the programming model for GPUs.
00:35:50
Speaker
That's huge because that's definitely not in NVIDIA's best interest. Correct. Or anybody else's. it's It's not Apple's best interest either. right it's It's a hardware company problem, right? Yeah, yeah. And so we have, I think, the biggest open source kernel library that exists with all the matrix multiplications around all this hardware. It's like over half a million lines of mojo code.
00:36:13
Speaker
And it's now indexable by AI. And so one of the coolest things about AI, and one of the things it's actually really good at is translate thing A into language B. And so it turns out that if you have Python code running on CPU, we have people that are saying, hey, Claude Skill, move this to Mojo, and they get 100 times, 1,000 times speed up.
00:36:35
Speaker
and Seriously. And you've got Python interop as well, right? You can pass Python objects into Mojo functions. yeah Yeah, and it just works. And so this is also where, because Mojo is designed for modern like modern cpus Yeah, it has really badass SIMD vector support. it's It's natively multi-core. And so you can just kind of goad the AI tools and say, hey, can you make this go faster?
00:36:58
Speaker
we'll say, oh yeah, I'll vectorize this for you. Oh, I'll parallelize this for you. And because all those abstractions are there, like what what happens is it just up-levels your code and hits you. And so you can understand it, which is really cool.
00:37:11
Speaker
And so this is where like we're in a different era in this co-design between what AI can do and what the language can do. is I think it's a completely different dynamic than what we had five years ago. Yeah. I think for the rank and file developers, just there's there's the CPU and then there's these other mysterious chips that happen to be in my computer.
00:37:30
Speaker
And to make to unlock that and be able to do stuff with it, even on laptops and TPUs and NPUs and things like that, is going to be huge because they're very opaque. If you start thinking, I'm going to learn how to program a GPU. Yeah, good luck, buddy.
00:37:43
Speaker
Yeah. and's i mean fun If you're interested, go to the GPU puzzles on the modular website and we'll teach you how to program a GPU. It's actually not that hard when you get decent tools and that actually work together. Now, let me tell you the bad thing about Mojo.
00:37:56
Speaker
okay So the bad thing about Mojo is it's not stable. We haven't hit our 1.0 yet. That's what we've been building it for just about four years now. It's really cool. People were seeing amazing, amazing results, and a lot of people love it. But again, i have this burden of wanting to do something that's actually good.
00:38:14
Speaker
It's not just the outcome, but it's the design. It's the ability to scale. It's the ability to like enable new things and last for the next, hopefully, 10, 20, 30 years of compute. Right? And so we've been very deliberate about that and we've been building in the open, but we're saying like, yeah, we're going to fix it. We're going to change things. We're going to laser focus in on getting to a 1.0 and stable.
00:38:34
Speaker
And the thing that I'm very excited about is that it's almost here. so come April, May, I think we'll have our 1.0 beta. And so it's, not very far away anymore, which is, oh my goodsh and I think we'll declare a 1.0 and then open source the rest of the compiler and all this kind of stuff, probably in, know, August, September ish. It's going to be a big year coming very soon. And so, The team's working super hard on this. I think that we all realize what a big moment this will be for the community because we have a bunch of amazing people in the community doing amazing things. But we want them to not be constantly thrashed by change. So the stabilization point will be a big deal for us as a community.
00:39:15
Speaker
Very cool. Mojo 1.0 coming soon. And folks can go to puzzles.modular.com. There's a really great eight-part tutorial series. You can just go through it and you can do hands-on GPU programming using Mojo. And you can do that today, getting ready for the coming 1.0 in the next month or two. This is super exciting. I really appreciate you spending the time to sit down with me today.
00:39:37
Speaker
Yeah, it's awesome to chat with you, Scott. like i i love I love your work and you're legend. I'll also say all this stuff's free. So please do join our open source community. I do love developers and I love seeing the creative things that people can make. And this is this is why, coming back to the top, what I care about in the human experience is the act of creation. now Being able to build something and being proud of it and saying, i did that. But also I think that many folks out there are looking at, you know what what do I do in this uncertain future?
00:40:06
Speaker
hu right? Do I just keep building the thing I was building yesterday? I mean, i think it's such a great opportunity to lean in. And again, there's this amazing form of compute, all these new tools and technologies. It's the easiest time ever to learn something like Mojo or whatever your new technology is. And so I really do encourage people to go out there and try out new things. And yeah, AI has got a