Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Building the Zed Text Editor (with Nathan Sobo)
3.2k Plays5 months ago
I’ve often wondered how you build a text editor. Like many software projects, it’s a simple idea at the core with an almost infinite scope for features. How do you build a solid foundation to expand on? Which features matter for launch? And how do you hope to satisfy the needs of every programmer, working in every language?
My guest for this episode is Nathan Sobo. He’s tackled this problem once before with the Atom editor, and he’s back older & wiser with Zed - a new editor written completely from scratch in Rust. It has a modern UI, a wide spread of language support, and a completely different way of looking at team collaboration. But with so much ambition, what are Zed’s priorities, and what’s been left for a future version?
--
Zed Homepage: https://zed.dev/
Segment Trees: https://en.wikipedia.org/wiki/Segment_tree
Ropes: https://en.wikipedia.org/wiki/Rope_(data_structure)
Rust Executors: https://rust-lang.github.io/async-book/02_execution/04_executor.html
More about Roc: https://youtu.be/DzhIprQan68
More about TigerBeetle: https://youtu.be/ayG7ltGRRHs
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Kris on Twitter: https://twitter.com/krisajenkins
Recommended
Transcript
Reflecting on Atom's Journey and Microsoft's Role
00:00:00
Speaker
Nathan sobo has unfinished business he was one of the developers on the text editor that came out of get up and i had a lot to like about it it looks great it felt modern and it was really starting to get some traction when get hub got acquired by microsoft.
00:00:18
Speaker
Microsoft didn't need a text editor they already had vs code and so the axe fell these days atom is no more.
Introducing Zed: A New Vision for Text Editors
00:00:26
Speaker
But nathan's desire to build the perfect text editor that has remained so now he's putting everything he learned from building atom into a new editor called said and said is very ambitious it's worth looking at.
Technical Foundations: From JavaScript to Rust
00:00:40
Speaker
So in this episode of developer voices we're gonna talk about what it takes to build a text editor for programmers.
00:00:46
Speaker
Which algorithms matter? What fundamentals do you need to know before you start building? Which language should you pick? Atom used JavaScript on Electron, and that was easy to make pretty, but much harder to make fast. Zed has chosen Rust, which is easy to make fast, but can you make it look good too?
00:01:08
Speaker
Even if you manage the job of making a great editor, you still have to think about adoption. How do you get new users? How do you pry people away from their existing editor? Do you need to have an extension language? Should you add collaboration tools of some kind?
00:01:26
Speaker
Should you specialize, should you aim at day one to be the best rust editor in the world, or are there ways to support lots of different languages really well early on? We're going to find out the answer to all those questions and more. I'm your host, Chris Jenkins.
Challenges and Aspirations Beyond Atom
00:01:44
Speaker
This is Developer Voices, and today's voice is Nathan Sober.
00:02:00
Speaker
Joining me today is Nathan Sobow. Nathan, how are you? I'm having a blast. We were having a good time talking earlier and, yeah, excited to have some of it on tape now. Cool. I think you're the first guest that's in their garden with a few of the mountains behind them, so that's nice. Yeah. You know, there's a couple of power lines in between, but
00:02:23
Speaker
Overall, it's still pretty good. Well, this is mostly radio. We can't hear the power lines. That's got to be a good safe thing. If you can hear them, you're about to die, I think. Anyway, we are not here to talk about power lines, but the lines of power that have been drawn through your life. How's that for a link? I was tempted to ask you why you're writing an editor, but you've got such form in writing editors.
00:02:48
Speaker
Really the question is, what are you doing with your latest editor? What's your unfinished business in the world of programming editors? Yeah, I feel like the why did you decide to write an editor was sort of for the first editor. And then this editor is because things didn't go entirely as I wish they would have with the first editor. When I set out to write, that was Adam, when I set out to write Adam,
00:03:16
Speaker
I had another like Zed in mind, but I did not have the ability to actually accomplish that. I had so much to learn. And we also picked a technical stack that was extremely challenging, not impossible, extremely challenging to deliver, uh, an acceptably performance experience. And, you know, so that's a review of electron as a platform, right? Yeah. Yeah. Tell me about that. Well,
00:03:46
Speaker
The only tool that really felt easy to reach for when thinking about building an extensible cross-platform editor that wasn't something we would have been building ourselves for us, and maybe this was our bias coming out of the Web 2.0 generation or whatever, but for us, a browser environment, like browser tech actually felt like, oh, if I could just use browser tech and then put a,
00:04:12
Speaker
put a desktop application power veneer under that. I know I can ship an experience on all the platforms that will work. And so that was one thing driving it. But what scared me from the beginning, I mean, I almost didn't even join to do it, was I was worried that web tech would be too slow.
00:04:35
Speaker
And I think the way we approached it and Adam it actually was like we opened up too much extensibility and it was just too slow. With the extensibility we opened up and like we're sort of unable to wiggle out of it. But anyway, ultimately we decided like we can't go down this path any further.
00:04:59
Speaker
Yes, Webtag has opened up this opportunity to ship this on every platform and have familiar tools and let people malleate it. But none of it matters if it's not the original vision that I had, which was an editor that felt a lot more like text mate. It's like I wanted like text mate, but powerful. Does that make sense?
00:05:21
Speaker
What was it about? Because I've never used TextMate, and I can understand why you want to add more power to it, but what was the core experience that was so good? For me, I experienced it as very fast. I remember when DHH busted out TextMate, that was where I was exposed to it in his original Rails thing, where he's doing, here's how you auto-generate this stuff or whatever.
00:05:45
Speaker
you know, at the time I was like fascinated with lists, but I was fascinated with all these other things. But I'm like, dang, that's a pretty cool experience. It's so, it's so simple. There's no, there's nothing. It's just this dumb editor that he's typing it into. And you, when you use that editor, the tactile experience felt responsive, but it did nothing. Of course you were using a dynamically type programming language. So the fact that it did nothing
00:06:08
Speaker
didn't really matter because there was nothing to do. Yeah. Yeah. I know what you mean. Yeah. It's like when you hit run, everything's going to happen when you hit run. Right. So maybe the editor doesn't have to be so smart for it to feel good. Right. And so I wanted that experience. I think a lot of people that want that responsive experience, they are adopting them, right? They're going back to an editor.
00:06:33
Speaker
that was written for so little memory and so little CPU that it's just inherently performant,
Exploring Editing Paradigms: Vim, Emacs, and More
00:06:41
Speaker
you know, or it's easy to be made performant. And I did, you know, I wrote Adam in Vim actually. So I did go there for a little bit. Yeah. Until I finally got, but then,
00:06:55
Speaker
I got the bindings kind of ripped out of my hands because we didn't have Vim mode in Atom. Sorry, that was not the first feature that we implemented. And I went back to Emacs bindings, where your pinky's just always down. Destroying the hands of a generation. It's not ideal. Yeah. And I sort of have the greedy algorithm running where to go back to modal editing is going to introduce this, like,
00:07:23
Speaker
temporary slowdown and I just keep not being willing to absorb that, but I really want to, yeah. Okay, so yeah, I think my first job, my very first job, the CEO there made everybody learn Vim, even like the company secretary had to learn Vim, so there was no choice because it steeped in you quickly. Wow. Okay, but I can understand why maybe you never jumped on that modal editing bandwagon.
00:07:50
Speaker
doesn't leave you a lot of choices for a really fast editor, does it? Certainly not that time. No, no. I mean, and I think I, you know, Adam, Microsoft bot, GitHub, Adam was sort of the writing was on the wall and I was over Adam because I felt like I needed to start from a new foundation anyway.
00:08:11
Speaker
But I was left in this little void period of like, okay, I don't have to keep doing text editor stuff. I really don't. I could work on something else. And so I explored some other stuff. But as I did that, I'm like, okay, about anything I'm going to do, I'm going to do in software. And there I was in Visual Studio Code at the time.
00:08:38
Speaker
And now could I have been in a different editor? Could I have gone in and adopted VIM, NeoVim? I think the answer is probably yes. But I just felt like, ah, there's so much I know. There's so many mistakes I've made that I can make up for. I have so much more knowledge about how to actually structure
00:09:02
Speaker
the design of an editor that is actually the thing I want, which isn't, for me, going back to, for me, it is kind of a GUI, a graphical editor,
Revisiting Text Editors: Learning from the Past
00:09:15
Speaker
not a terminal editor, but in all other respects, sort of has the same vibe of a tool geared toward experts, but graphical.
00:09:27
Speaker
And anyway, the cursor just sat there blinking on me, mocking me. Literally, I think I've described this maybe on one of their podcasts of just sitting there. And I think I was implementing elliptic curve cryptography. I was like, well, let's explore cryptography. Maybe there's something there. And that was like writing a slow version of elliptic curve cryptography and Rust. And just watching that cursor blink.
00:09:57
Speaker
And just with each blink, it's like. Yeah. This seems like a very natural programmer instinct. Like you did it once, now you know how you should have done it properly. And that's always going to hit you for the rest of your career until you finally go at it. So.
00:10:16
Speaker
I, so I've dabbled, I've dabbled myself in writing an editor. I never went all the way down the rabbit hole. And I certainly never got to the point where version one was done and I wanted to crack on with version, completely new version two. So you know a lot about writing an editor that even younger, you didn't know. Give me some hints about like, how do you, what's the right way to architect an editor? What do you need to worry about? Which algorithms matter? Right. Well, I mean.
00:10:48
Speaker
For me, the choice of tool really does matter. So I don't want to spend a ton of time on that. Maybe we can come back to it. But I think on an algorithmic level, something that has really mattered to me is mastering this particular data structure, which I think it's called a segment tree. I didn't know that at the time. I sort of made it. And that was long, so I called it a sum tree. But it's basically this data structure that's like a copy on write.
00:11:15
Speaker
B tree where B plus tree because all the items are stored in the leaves. But instead of what we're summarizing up the tree is sort of this, the sum, the aggregation of summaries of all the items in the tree. And so it's sort of this persistent universal indexing data structure. So having that data structure is unlocked a lot.
00:11:43
Speaker
So, am I imagining then, if I open a text file in my editor, that's actually the leaves on a fairly deeply nested tree? Not only is it the leaves on a fairly deeply nested tree, but it's also been filtered through the leaves of, you know, five, six, maybe, I don't know, maybe that's over-exaggerating. A few, I don't have a count, different trees that are sort of in charge of efficiently translating
00:12:12
Speaker
you know, a point in the file to, oh, but now we've injected synthetic soft wraps. Now we've folded text. Now we've expanded tabs. Now we've injected inlay content. And each of those intermediate structures, which is used to map that point is also expressed as one of these, as the leaves of one of these trees, basically, which is an efficient index on a sequence.
00:12:38
Speaker
Okay, so if I look in this tree, I'll see something roughly like this is a fold of a block of code, which has a blue thing, which is actually the character's F U M for function, right? Like something like that. Yeah. Like inside that in our structure, we have the fold map and the fold map. Um,
00:13:03
Speaker
you know, has a sum tree of a bunch of folds and for each fold they're sort of
00:13:10
Speaker
in the output coordinate space, how many rows and columns or offsets, I think it might be expressed as how many bytes are spanned by the fold in
Architectural Insights: Efficient Text Management
00:13:19
Speaker
the output, closer to the viewer, and then how many bytes are spanned by the fold in the input, closer to the file. And so sort of we have these, a sequence of these trapezoids. So they're sort of trapezoids that are,
00:13:35
Speaker
know, isomorphic, like they might be skewed in their relationship left and right, but you basically like have seasoned in these weird spatial transformations and you can keep a running total on the net result of, oh, there's a fold here and then it's the same.
00:13:57
Speaker
And then maybe there's, you could go the other way, we don't support that in the fold map, we separate that out. But there's a spatial transformation basically being expressed in the sequence inside here. And then by efficiently being able to seek in one dimension while summing in another, we can efficiently traverse that transformation so that we can say, oh, we're scrolled down to line 5,000 in the buffer.
00:14:23
Speaker
Where is that though? Because if we have a soft wrap line, that's 3000 lines long. Yeah. Anyway. So yeah. Yeah. Anyway. So this translation between the raw text and all the different layers that go on top of that into screen space. Yeah. Yeah. Uh, and it's really helpful to be able to. Right.
00:14:49
Speaker
interact with the index on that transformation in an efficient way. And so all the focus is on the rope, but the rope is just where the text is stored. There's all these other persistent structures that I think that we've discovered need to exist for getting the rope onto the screen. I think you need to step back and define what a rope is.
00:15:13
Speaker
Yeah, so rope is a way of representing a string. The typical representation of a string is basically one region in memory, right? And we record sort of how long that region is and where it is in the string handle, and then it's allocated on the heap somewhere. And we've got this contiguous block of bytes.
00:15:37
Speaker
But if you're manipulating that a lot or you get a really big, imagine you have a string, a file that's 200 megs long and you just want to insert one character. Are we really going to be moving memory?
00:15:48
Speaker
No, the answer is no, we can't move all that memory. So this direct allocated representation is not good for editing. What's good for editing is to take the string and break it up into these tiny chunks and then structure those chunks into a big tree. We use persistent trees, which like I'm sure your audience is probably pretty familiar with like from functional programming talks and stuff. Like in a mutual data structure.
00:16:16
Speaker
Yeah, immutable data structures. We, you know, our representation is persistent. You can imagine a mutable one as well, where you're basically when every time you perform an edit, we're essentially performing path copying, where we're recycling all areas of the tree that contain text that was unchanged. And we're, you know, bounded by some, you know, O log at rhythmic depth going down, going down that tree.
00:16:45
Speaker
We, yeah, anyway, unlike storing a file system on this, right? It is broken up into little pieces that and pointers to where everything is within the hierarchy. I mean, I've never written a file system, but I have imagining that there'd be. Yeah, I mean, logically rather than practically.
00:17:06
Speaker
And one thing that I'm excited to do is actually start persisting these persistent trees to durable storage. Yeah. Um, and, you know, starting to use that to go deeper in terms of what we can index and
Navigating Rust for Zed's Development
00:17:20
Speaker
store. Cause we also use the sum tree to store the sequence, which represents the CRDT, which is like, we have to get into CRDTs. Okay. Yeah.
00:17:32
Speaker
That's a rabbit hole in itself. I want to get too crazy, but I want to do it in the right order. So there you are becoming an expert in manipulating these data structures, which you must do.
00:17:47
Speaker
That feels like it's gonna be a little bit painful in rust. I mean, I don't know. I don't know. Just tell me. Because like borrowing and passing things between functions, that feels like the performance ergonomics trade-off thing. Yeah, that's interesting. Definitely when I tried to learn Rust, I was like,
00:18:12
Speaker
I gave up, I gave up for a while to be honest, and then I tried again. And I've definitely gotten better at it, and it's definitely hard. The learning curve has been brutal and hard also just because like, how do I even express this in Rust didn't feel immediately apparent. Like reaching for like paradigm or patterns, ways I've done it, I had done it before was not helpful because none of those patterns worked.
00:18:38
Speaker
Yeah. That reminds me of my experience learning Haskell, which has become one of my favorite languages, but boy, did I have to change the way my brain thought to get used to it.
00:18:49
Speaker
So I don't know. I never mastered Haskell because I'm really excited for Richard Feldman's language rock. And we're hiring. He's joining our team, which is going to be really cool. No way. Richard Feldman's joining it. Yeah. When I talked to him,
00:19:09
Speaker
Honestly, one of my first conversations with him was on his podcast and he told me about Rock. Anyway, sorry, I'm going off on a tangent. I'm really excited about that. But I think maybe coming from the standpoint of a functional programmer, probably it's a little like expressing UI and building these constructs the way that we are.
00:19:34
Speaker
Maybe it feels, you know, more verbose than it would need to be in a functional setting. Um, for me coming from, you know, languages like JavaScript, uh, and or C C plus plus. It's like a, I dunno, there's a little bit of ceremony needed to kind of, uh,
00:19:58
Speaker
use patterns that end up being good ideas anyway that make the borrow checker happy. But other than that, I don't think it's like a huge, it's not a huge deal. I mean, when I see how expressive things can be, like I started thinking like, oh my God, is the entire state of the application, you know, a state monad and like, you know, starting to expand my mind into like even deeper ways of thinking about it. But for me, like, yeah,
00:20:27
Speaker
from the perspective of a day-to-day application development, being willing to use MutableState when we need it. It's like, I wanted to emphasize like just simple structs. You can mutate if you want. You don't have to opt into any immutable world. Like it embraces the reality of what Rust is as a language, I guess, is the idea. But you know, is there room for some
00:20:54
Speaker
fancier functional language on top of all that system infrastructure someday, maybe, but I think you can make Rust expressive enough to build these ideas out. Yeah, I would certainly believe that true. And I would certainly believe that when you get there, it's very high performance. One thing that surprises me though in the choice of Rust is it's not, it's something I really associate with like low level C code, like
00:21:20
Speaker
if you want to do systems programming. Ideal for an editor. I don't associate it with building graphical interfaces. Right, right.
Rendering and Performance in Zed
00:21:30
Speaker
And I think that's because just, I mean, there are Rust UI frameworks at the time when we started. There were some too. I looked at them, they weren't very well documented. They were clearly changing a lot. And I'm like, well, the one thing
00:21:45
Speaker
I know I can do is just build it. And so I ended up, you know, for some, yeah, because you don't associate it with building a graphical interface. Having to build that out to a certain extent, not the only one, but building it out. And what I just like about it is like, it's simple. It's all in one language. That language is incredibly fast. And if you can figure out how to contort that language into doing what you want,
00:22:12
Speaker
It'll be fast. And now the question is, how do we make that ergonomic? And I feel like that's kind of a lot of what we focused on is like, what are some ergonomics of that that are reasonable to make it super fast. But then, I don't know, we can also, it could be under some other language potentially. Yeah. Yeah. But when you say ergonomics, do you mean for you as the Zed development team, or are we talking about the ergonomics of a Zed user?
00:22:39
Speaker
for us to develop it. Yeah, exactly. Yeah. Tell me about your UI framework, because having used said, it doesn't look like it's written in a C like language, right? I mean, it looks great, if I may say. Oh, thank you. So tell me about the rendering framework you've made.
00:22:59
Speaker
Yeah, I appreciate you. Yeah, I care about how things look. And I think that deserves deep thinking. So I think that's all. And we've had a designer on. It was the three founders and Nate. He was our first hire. He was a designer. And sometimes he's just like,
00:23:15
Speaker
he's basically had to learn to code and he's gotten really good, which has been incredible. But like, so how things look matters to me. I want it to feel, and that doesn't mean like, let's slap some gradients on it and have it look awesome. It's like, no, let's have it be tasteful and put your content first. So I think that's driven the design of the UI framework of like, okay,
00:23:41
Speaker
In an environment where I care how things look, you know, in the rewrite of GP UI, we introduced Taffy, which is this layout engine written by the Dioxus Lab guys, or people, I should say, the Dioxus Lab people, and
00:24:00
Speaker
that provides like a CSS esque. It's pretty good, but it's not perfect. Like it's a community effort sort of layout engine, but it's like, can I tap into getting a flex box style and you know, some basic inline block layout? Um,
00:24:20
Speaker
But then not, I don't have the DOM. So instead of that, like we actually build out a tree of these elements and they're allocated in an arena that we discard at the end of every rendering frame. And the elements sort of express a tree and that is your one time DOM for the frame, kind of. Oh, okay. If that makes sense. Yeah, yeah, yeah. And you just render that out. Is it?
00:24:49
Speaker
But that's... I almost want to say a physical thing. It's not like a virtual DOM-esque approach. No, I mean it's... Yeah, well, I think you need a virtual DOM in the case where you're trying to diff. And the best way to get the window from one state to the next is to mutate some underlying structure.
00:25:12
Speaker
But if you can just get your DOM to the screen in its entirety faster than all of that diffing,
00:25:20
Speaker
And you can just bypass virtual, and it can just be sort of, it's a DOM, but it's a DOM for one frame. You build this tree of elements, then we walk the tree, and they all register themselves with the layout engine. We perform the layout, then we walk the tree in a pre-paint pass, which we use to determine things like occlusion. Because if we want paint to be sensitive to occlusion, we have this pre-paint pass, which takes care of occlusion. There's a couple other details.
00:25:49
Speaker
And then inclusion is one thing sitting in front of another, right? Correct, exactly. Yeah. So it's like, if I want to color the background of a button based on whether or not it's hovered by the mouse, I need to know whether it's actually hovered, because it could be occluded, which means I shouldn't give it a background color. But I can't know it's occluded until I prepaint the scene. Anyway, that's it.
00:26:18
Speaker
That's a nitty gritty detail of this larger idea of like, we sort of build this tree, we walk the tree in a couple passes, and then we discard it. We've arena allocated it so it's not a huge deal and we can do it really fast.
00:26:33
Speaker
I can, I can understand that because coming from like, um, web front end, that sounds crazy, but you think about people building games in Rust, having to do 60 frames a second, and they're not complaining about the speed of Rust if they're complaining about anything. So yeah, why not? Why not render, why not render your editor at 60 FPS?
00:26:57
Speaker
And obviously, we only do it when you edit it. After every edit, we'll render frames at 120 frames per second. I forget what it is. It's not that long, a period of time. Because if we don't do that, we discovered in this funny incident with another YouTuber, Theo Brown,
00:27:17
Speaker
We discovered that ProMotion, which is a feature on the newer Apple laptops, downclocks the display. And so you end up not actually getting 120 FPS responsiveness out of it.
00:27:33
Speaker
So we actually render synthetic frames, but then once that little quiet like that, you know, warm up or keep warm period has elapsed, then obviously we stop rendering. We're not actually a video game, but the ability to re-render the entire window, you know, my favorite demo, I was at Boulder startup week last week and I just like, I was like, okay, everyone just come around. Like, I can't show you this through like a,
00:27:58
Speaker
an HDMI cable plugged into a television, like you have to look at my computer and just hold down the tab switch button on Z. And I have my keyboard repeat rate a little overclocked, so I like my life that way and it makes for a good demo of just
00:28:21
Speaker
You can just see the editor burr, right? And it's like, yeah, if you optimize this around the assumption that we're just, you know, and there's a little bit of caching here and there to like save power. But like if the assumption is that it's no problem at all that just like render the entire window at the frame rate of the display.
00:28:42
Speaker
That's a cool demo, who really cares? But what it really means is you can actually be positioned to consistently deliver reliable responsiveness on the next frame.
Enhancing Zed with WebAssembly and Extensions
00:28:53
Speaker
Yeah. Yeah. And there's a decent end-to-end test if you're just, because you're constantly changing the underlying buffer. Yeah. That makes sense. And is that perfect? Do we have bugs? Yes. It's not always perfect, but like it's a hell of a lot closer than a garbage collected single threaded scripting language could deliver. I guarantee that. Yeah. Yeah. I can believe that totally.
00:29:14
Speaker
So I have an image in my head of the text file I want to edit being turned into a tree with lots of annotations on it for all the various editing features and that tree being turned into another tree for rendering. Yeah. Yeah. Is that the right mental model? Yeah. So you might want to just like love trees, but that's just computer science generally, right? Yeah. Yeah. Yeah.
00:29:38
Speaker
It's reminding me of, I've been writing a little lure in Neovim recently and they only have one data structure. It's basically a tree. If you want an array, build it with a tree. Yeah, Lou is interesting. I've never written any. I studied it a little bit, but I never written any. Um, but I liked the simplicity there.
00:29:56
Speaker
Yeah, there's things to like about it, lots of things to like about it. But that makes me want to come, so if I've got the rough shape of how I would build an editor in my head, that makes me want to come up into user space. I think my first question is, is there a language in which one extends it? Will there be?
00:30:16
Speaker
Right. Well, I think WebAssembly is a pretty reasonable language, meaning that's an interface that doesn't pose some constraints. So it would be the WebAssembly components interface. That's where I'd like to start and see, will that work? How much will we need to sort of the worry that I have and I just don't know. And I think even members of my team might be able to speak more confidently about this.
00:30:46
Speaker
Are we going to need some sort of WebAssembly-side library code? Can we just use the WebAssembly component interface directly? Or are we going to need to sort of sit in front of that, to solve that interface on the WebAssembly side? And will we need to pick a language, basically? And in that case, it would probably be Rust, with obviously the ability to switch to other ones.
00:31:14
Speaker
But anything supporting web assembly, uh, so rock, I mean, now I'm excited to like experiment with that. I mean, we'll have to offer. Richard's bring Richard's language in. You've got the expert in house or about. Yeah. Yeah. I mean, I'll admit I had, you know, I've definitely been thinking about that of like, you know, where.
00:31:34
Speaker
where rock seems to shine, although I barely used it, and Rust falls down. It's just the feedback loop time. That's the big, hard thing about Rust is how long it takes to compile. So that would be something discouraging my use of Rust in any kind of official hosted API surface beyond the WebAssembly interface.
00:31:58
Speaker
But on the other hand, there's a hell of a lot to be said for monoglotism. Just like, it's all rust. You learn one thing, you learn it all really well. Like, how far can we push that? But I don't want to do that to the point of being stupid. It's like, there may be a strategic move that we could take, but I think we'd want to take it very carefully. Going from one to two feels like a big deal, as anyone who's had two kids will tell you. Yeah. Yeah.
00:32:25
Speaker
I think I've been triggered. Yeah. I mean, this is reminding me of, we've spoken briefly about this offline, but about Emacs. One of the things about Emacs is it's just, there's one language all the way down. The language you extend it in is the language you configure it in is the language it's written in. Are you tempted to go into that kind of world where
00:32:53
Speaker
Where if you open people, if you open the editor up to use a space, why not open up the entire can of worms? Like the entire can of worms in terms of.
00:33:06
Speaker
giving them, yeah, what's in that can? What kind of worms crawl out of that can? I mean, you know, if I, let me, let me give it a concrete example. So having looked at Zed, I don't think there's current, correct me if I'm wrong, but I think, don't think there's currently a plugin for like Git support. Correct. Inevitably one day there will be. Right. So how deeply integrated into the editor can that be?
00:33:34
Speaker
I think it's going to be organic. Um, more than, yeah, there'll be more power than we have now for sure. Yeah. Are you rendering custom UI? I mean, if we can build a scheme that makes that efficient eventually. Yeah. Um, but like, I dunno, I want to be judicious about what we open up when we open it up. Um,
00:34:01
Speaker
Yeah. There's some things that we should just build into the editor. It's fine. Like not everything needs to be done in an extension. We can actually build things into the editor. Um, and not everybody needs to use every single thing we build into the editor. Obviously at some point your binary gets ridiculous, but like we have that valve and we're an open source project. So like we should never forget that we have the source. Um,
00:34:27
Speaker
But yeah, over time I'd love the extensions to get better. So what is that? That's about opening more API surface area into those components or deriving some kind of scheme where these WASM components are, you kind of plug them in, like a, you know what I mean? What does the API look like? Just designing that all is not something we've invested in a ton yet.
00:34:51
Speaker
Yeah, but that's fair. I mean, it's to build something like this is the project of many years and you have to choose which parts you pick first, right?
00:35:01
Speaker
One of the most important parts of startups is prioritizing. Yeah, and it's really hard. And I think we have room to improve, honestly, on doing it. It's hard to construct a narrative that's coherent in the face of so many different people with such disparate needs.
Collaborative Editing with CRDTs
00:35:19
Speaker
It's hard. Yeah. To succeed, you need to satisfy a lot of programmers with a lot of opinions, working in a lot of different languages with a lot of expectations.
00:35:29
Speaker
But extensibility, yeah, it's one thing I'm excited about. I mean, we're already thinking about injecting it into like prompting where, you know, just that this is an opportunity of like, okay, let's make this extensible. We've got the WASM dynamic prompt. You can run arbitrary script that produces the text.
00:35:51
Speaker
That kind of leads us into perhaps a fairer question, if I've pushed that too far. In your priorities, extension is on the roadmap, but you've picked some interesting things to prioritize right now, like CRDTs, a few other things. Where does Zed excel today?
00:36:15
Speaker
Yeah, that's good. Where does that excel today? Then where are we shooting? What's the next big improvement we're focused on? I guess where it excels today is we're really fast. I mean, we really are. There's been a couple of bugs where that's regressed here or there, but for the most part,
00:36:34
Speaker
I never experienced a delay using Zed. And so we have that. We have really cool non-local editing experiences, which are this concept of a multi-buffer and something we're going to be investing in further. Imagine performing a search.
00:36:54
Speaker
and having all the constants of that search or all the compile errors summoned to a single editable buffer, which you can perform a multi-cursor edit across if you wanted to, or address one by one. So enabling some powerful workflows around, especially with compiler-centric languages like Rust, where you could just bring all the compile errors to you
00:37:19
Speaker
in one editing session, one editing experience, like a lot of times like you can really work through errors with a level of efficiency that isn't possible in any other way. Are you saying like,
00:37:33
Speaker
So when i get a list of compiler error errors i go through this list and jump to the place in the file where it is and then edit then jump back to the list are you saying your list will actually contain a view into the file. Yeah you can get this portal into another location that's kind of embedded all within one document what we have now is this thing called multi buffers which.
00:37:57
Speaker
I think what I'd like to move toward or experiment with moving toward is more creating a solution that's closed under composition where rather than sort of there's a buffer and then a multi-buffer, there's just buffers and then portals from one buffer can be embedded in another, create these like
00:38:18
Speaker
non-local editing experiences, but for right now, the multi-buffer is the approach, and it does enable some pretty awesome refactoring patterns. Our VIM support is quite solid, I'm told, although obviously based on our earlier conversation, you know, talk to Conrad about that, who's been working on it.
00:38:39
Speaker
And what you can do is like multi-cursor them, which arguably maybe you could use macros for that instead, but it is a kind of a modality, a way of working that's particularly interesting. Our collaboration support is pretty awesome. We use Zed to build Zed with each other collaboratively pretty much all day long.
00:39:02
Speaker
That's something we really need to go further into, because I've seen a few web services take a crack at collaboration. I've seen a few editors try it, but I've not seen like a first class, let's work on a file together in our editor experience.
00:39:21
Speaker
Yeah, so like in the lower left, there's a little icon with like people on it. You click that and that's your channel tree. We've made everybody a member of the Zed channel by default. That's where we do our work, mostly in public.
00:39:36
Speaker
Like we have our meetings in public. We had a meeting about some of the AI work I've been working on with Antonio and Max. Just a public meeting. Anybody could pop in and listen to us talk. And the tree of notes is associated, or the tree of channels, each channel is associated with some notes. I don't know that this is how it's always gonna be, but then you can kind of talk to each other in audio, edit notes which are persisted on our servers as kind of like,
00:40:05
Speaker
a lightweight project planning process, if that makes sense. You're sort of meeting, talking, sketching with these notes. And then into the channel, you can also share your editor. You can share your project. And once you've shared your project, anybody with access to the channel, there's guests, and then there's people that would rewrite access.
00:40:27
Speaker
They can follow you around inside your editor and edit. And they can also edit. Yeah. And this isn't like, I'll push my repo to you and you have your own copy. This is more like a Google Docs, we're editing the same thing at once experience. Yeah. It's for a different kind of interaction. Yeah. It's for an interaction where it makes sense to be.
00:40:51
Speaker
coordinating in real time, I think, for the moment. We obviously, as we get into some of what I was talking about earlier with persisting our persistent data structure, I think we can start to get into more async-focused use cases where you're, you know, imagine you're, this happens to me all the time, I see some piece of code, I encounter it, maybe it's a change to some decision I had made and I see it and I'm like,
00:41:20
Speaker
Okay. There's probably a good reason, but why did you change that? Cause I kind of like that or whatever. And I want to highlight that and just say like to the person that wrote it, just leave an annotation on the code. Hey, uh, why'd you change this again? I don't want to like,
00:41:35
Speaker
commit right now, push up, right? I want this more, but anyway, that doesn't exist yet. What exists now, I think, is just a great place to talk about code in real time with people, whether you want to write it with them, like that pair programming, or just do some live code review and say, here's what I did. I find it's just often a lot more productive than trading comments on a snapshot of the code.
00:42:01
Speaker
That's a way of communicating. Yeah. I think we're both old enough to remember before you even got that level of like commenting on the PR collaboration. And the only way to collaborate was to literally walk over to someone's desk. But there's definitely something beyond batting a pull request back and forth. Yeah. Which we really need to get to as an industry. Yeah. And I think we kind of did have it inadvertently when we were sharing offices more.
00:42:28
Speaker
Just like you said, you could literally walk over there and now you have a shared memory access, shared device access to the same state or whatever. You don't need a CRD2. But as soon as you are somewhere else, it gets a lot harder. Now you kind of do need a CRD2. I don't want to go back to the office. I want that experience to be transformed into code for me.
00:42:52
Speaker
I mean, I like it both, but I get lonely, but I also, I'm at home right now. I could be in an office that I'm not. You could be lonely in an office these days because no one else is going in. That's also true. So tell me about the implementation of that though. Go under the hood for me. Okay. So.
00:43:12
Speaker
It's fundamentally like the editing is based, every buffer in Z is fundamentally a CRDT. What is a CRDT? It's a conflict free replicated data type. What is that? It's basically like an eventually consistent data structure is I think a reasonable way to think about it.
00:43:33
Speaker
data structures that you can build in a way such they're eventually consistent. I do something to my copy over on my computer. You do something to your copy over on yours and we swap some sort of data. And at the end we see the same thing. That's what we're trying to build fundamentally. Now, uh, I kind of wish, although hindsight is 20, 20 and, um,
00:44:00
Speaker
I don't know. Yeah, I kind of wish we would have gone even harder in on CRDTs. Oh really? Where we use them was on just the buffers, but for the. I had previously done this project at GitHub was not in a good place. It was kind of in that what am I doing with myself phase that was mentioning earlier? Called memo where we tried to build a.
00:44:27
Speaker
fully replicated CRDT for an entire source tree.
00:44:32
Speaker
I think we were going about it in some wrong ways in retrospect, but I just felt like I'd been burned a little bit by biting off too much complexity. So I was like, we're going to do the buffers. This is CRDT. And then we're going to do the rest of it, like the file system and interaction with the language server, et cetera. It's just a simple hub and spoke design where like every project has a host machine, that host machine.
00:45:01
Speaker
has access to the language server and the file system for the project that's being shared. And everyone else's experience is basically delivered via RPC to that project host. But obviously, you can't edit text via RPC. There's too much latency. But you can run a project fine via RPC. That's fine, right? You don't expect
00:45:24
Speaker
you know, CRDT and then we don't have to fully replicate the data. There are a lot of advantages to it of like, I can share my project and boom.
00:45:34
Speaker
you have the illusion that you have access to it. Some of it is being kind of RPC it on the fly, but it's tricky to get right. I mean, we've had to write these randomized tests that the cool thing about Rust, I don't know if you've gotten into this with any of your other guests, but like the idea of an executor, the idea that their futures in Rust, async code,
00:45:58
Speaker
sort of has to be explicitly run on an explicit thing and you have control over that thing. The executor that's running your async code is not like attributed to the environment like it is in Go. You can actually like write an executor and so and an executor is kind of like how that async code is run
00:46:18
Speaker
is a parameter that you control. And so what we've been able to do is actually build a fake executor, a test executor, that where we create a server, we create our server, we create multiple copies of said simulated clients, each running inside their own application context. But they all share one executor. And then we can do these cross-network
00:46:45
Speaker
like integration tests of the entire collaboration protocol and all the different combinations of things that can happen, where we have full control over all concurrent operations are being randomized relative to one another with a seed. So we can be like, oh shoot, on iteration 20,739, there was some freak thing where this took way too long, which way only happened like,
00:47:09
Speaker
you know, once in a century. Yeah. But we're able to track that down and isolate it and fix it. So you're doing kind of generative tests where you can control what the network would be like. Yeah. Yeah. Yeah. That's a cool thing that Rust enables that. I didn't know that that is cool. Yeah. Because this kind of concurrent networking code is notoriously difficult to get right. And I've written concurrent networking code where you try to test it.
00:47:40
Speaker
and you let there be noise, right? You try to write a test for this async code, but it's actually using the scheduler. It's actually doing IO.
00:47:54
Speaker
And it's like, then you find the bug, right? The only problem is you find it like 0.1% of the time on CI and it's not reproducible, which is hell. It just grinds everything to a screeching halt. So it's like with this approach, it's like we don't really, we have a couple, but for the most part we don't have non-deterministic tests.
00:48:17
Speaker
we have been able to effectively model non-determinism and then drive it with our own RNG. Yeah. And then when you find a bug, you've got the receipt, the seed that tells you exactly how to replicate it. Yeah. And I think people do stuff like this with property testing. I mean, a lot of our stuff is like pretty brute force, like custom bespoke, like we reduce the problem ourselves. I know there's like, yeah, fancier functional languages have
00:48:46
Speaker
automated frameworks that do this. But I don't know, we've just found a very pragmatic technique of isolating the non-determinism completely randomized test that. Yeah. This is reminding me of Tiger Beetle. They're doing that with Zig. Cool. If you want to do concurrency and that kind of thing without losing your mind, you've got to simulate it in a controlled way, but a realistic way.
00:49:15
Speaker
Yeah, and I like that. That's like an interesting, it's like a different take on, Russ is like, we are going to make sure that problems don't happen by construction. And it seems like if you're not going to do that, then it does seem like a cool way to
00:49:32
Speaker
fight back against the potential pitfalls of that or just like really invest in ways of detecting the problems and really good error reporting. And like, anyway, that sounds cool that they're, it makes sense that they're pursuing that path. Yeah. Yeah.
00:49:48
Speaker
So give me some thoughts on how you construct to CRDT. I mean, obviously that could be a podcast in its own right, but what are the core bits of distributed editing?
00:50:04
Speaker
Yeah, surprise, surprise, it's a SumTree. It's yet another one of the same type. And the SumTree contains these like, it's not just one SumTree, there's a couple involved, actually, there's multiple indices that we need to be maintaining, but we're basically
00:50:23
Speaker
breaking the text up into these fragments. And once a fragment is inserted into the tree, it actually is there forever. So the structure only grows, it accretes, but you can delete text by sort of marking a fragment as hidden. And so say that I like put my cursor in the middle of a paragraph, for example, of text that you wrote, and you highlight that paragraph and delete it,
00:50:50
Speaker
and I type that key, it's fine. We still know where my key goes because when we deleted your paragraph, we didn't delete it. We just tombstone it, right? We just hid it. And so over time, these structures obviously accrete. That's one of the challenges I think about persisting the thing to disk is that we'll need to kind of
00:51:11
Speaker
deal with that, find ways to deal with that accretion potentially, I'm not sure. Or maybe we find ways to just stow it off in parts of the tree that we barely access. And it's just there for archival purposes. Because the nice thing is by accreting, you are able to sort of represent every state that the document has been in over time. But anyway, we divide the text up into these fragments. And the idea is that, like,
00:51:36
Speaker
Even if I'm inserting inside of like relative to your, let's say you pasted a paragraph, right? And now I'm gonna type a character. I can say, oh, I'm at this particular ID, we assign your fragment a unique ID that's unique across the collaboration. And then I provide an offset into that fragment. And so since the fragment never goes away, I can always find where that offset is inside the document.
00:52:03
Speaker
But the problem is, is the fragment could get fragmented, meaning you might have pasted that paragraph, but then come into the middle of the paragraph and started typing. So now you've taken what was one piece of text and sort of split that piece of text apart.
Innovating Code Development and Collaboration
00:52:20
Speaker
I don't know how much sense I'm making so far. No, I'm with you. I'm with you. OK.
00:52:27
Speaker
Okay, and so, yeah, go ahead. You're kind of pushing, you've almost got a cursor into a tree for each user, and they're being pushed around the tree by other editors. Yeah, that is an interesting way of looking at it. Yeah, that's pretty accurate. We have these things called anchors, and an anchor is basically essentially an identifier for
00:52:53
Speaker
a fragment, one of these fragments that has an ID that's been assigned, plus an offset, and a bias of which direction that anchor should go if something right at that location was inserted. But yeah, that's a cool, really powerful construct.
00:53:15
Speaker
use these anchors in all kinds of interesting ways, like to demarcate a region in the text in such a way that I can come inside and edit inside that region. And now that anchor ending the range of the demarcation is getting pushed downward. Yeah, it's a really powerful tool that we don't only use for collaborative editing is my point. Yeah, yeah, I could see how, I mean, it seems like a very generic structure that could be applied in other things.
00:53:44
Speaker
It's also making me think you have potentially the complete edit history of a project and you have a really fast game speed rendering engine. Are we going to see the entire history of editing a project displayed on a board, like at a conference, like just watch the whole thing evolve from scratch.
00:54:05
Speaker
God, that's a great idea. I know we don't have that persistent history yet. It provides me yet another incentive to be like, we have part of it built CRDB. There's a branch. But yeah, I mean, that's the dream. And also to be able to sort of, because once we have that, we have the backbone of,
00:54:31
Speaker
We have a new level of backbone right now. The backbone for social coding is commits because that's just what that's like the lowest level of shared state.
00:54:43
Speaker
Yeah. That's the atom of communication among programmers. Exactly. And that kind of defines what the interaction is like, that atom. And I'm excited to see what the interaction starts to feel like when the atom of communication shifts down to the character, like the keystroke or the paste. Obviously, you could paste multiple characters too. But that will be...
00:55:13
Speaker
I don't exactly know what it looks like. I mean, I think we're getting, we're getting a taste of it on the Zed team of just, we don't really use pull requests a lot. What we do is just talk about code. Like even if.
00:55:24
Speaker
Antonio may spend the whole morning and I'll pair with him for like an hour. We'll talk about a few different things. I'll pair with him a little bit and get the flavor of what's going on. Maybe I'll come back later and solo. I sold earlier, but like, by the time that's ready to merge, like we both have seen all that code. We don't need a pull request review. Yeah. And we both contributed a lot more value to it than we could have had we been like trading comments on this or something. We're just talking to each other. It's a different experience.
00:55:54
Speaker
Yeah, I can imagine that. So you're ending up with a kind of experience where people, it's almost like people dropping into a party. Is that what it looks like? Yeah, I think that's very similar. Uh, like we're hanging out in a channel and like, I'll hear boop, boop, boop as people popping in. And we'll also even have like,
00:56:17
Speaker
Yeah, it depends on the time. Sometimes people are very solitary, but there are other times where you'll see it inside Zed's channel tree, like a decent chunk of our company at any one time, like paired up in these channels. And then there's even times where like one channel will go visit another one. So both members of the one channel will just go, okay, let's go talk to them about that. And some people are like, oh my God, you're ambushing me or whatever. I don't know. It just depends on the vibes. I think,
00:56:47
Speaker
Working on sort of how do we build those same kind of cues that you'd have in a physical workspace of like, can these people are busy, leave them alone? Providing more like ambient awareness, I think is going to be important to making this really scale. Well, but it works pretty well, even the dumb thing.
00:57:05
Speaker
Yeah, it's one of those things where if you enable a lot more, a much richer experience than we currently have, there are going to need to be social and etiquette changes to go along with it. Yeah. Yeah. And yeah, we're exploring all that on our team. You need to get an anthropologist in to study you early so you can get all the data.
Language Support and Parsing Technology
00:57:27
Speaker
Yeah, I mean, Torsten is kind of an anthropologist, to be honest. Have you ever talked to Torsten Ball? No, I haven't. He's awesome. Yeah. But anyway, he's been kind of, he's come in and...
00:57:42
Speaker
He's a bit of an archeologist maybe and a bit of an anthropologist of just like been talking to us about all this stuff we built. Cause the problem is we're so busy building it, we forget to talk about it. And it's helpful to us. Yeah. But it's fun to talk about and it's helpful, you know, so. Yeah, yeah. So he's come in and really leveled us up in our, in terms of,
00:58:08
Speaker
A lot of the things he's talking about are something other people have done. And he's kind of being there as a really good documentarian. And I've been loving that. Okay. So yeah, he almost is your historian and anthropologist on the team. Yeah. Cool. This makes me wonder, so collaboration, I'm thinking about the differences between the way programmers work.
00:58:33
Speaker
You don't yet have the problem where everyone has a different bunch of extensions installed. You're going to enjoy that problem. But you do have, having looked through the list of extensions that Zed currently ships with, you've got a lot. You've got like covering languages I wouldn't expect you to cover. Like you cover Elm and Gleam. How is it you cover that many like fringe or non mainstream languages in a young editor?
00:59:03
Speaker
Two big investments have paid off for us. One is our investment in really Max's investment, but something that was an investment of the Adam team as well, TreeSitter. We've been working on TreeSitter and how to apply it to integrate it. TreeSitter has been baked into Zed from the very beginning and we have the grammars available for those languages.
00:59:32
Speaker
Give us an overview of TreeSitter for those that don't know. Sure. TreeSitter is an incremental parser generator. A parser is basically something that takes a formal grammar in and then produces a program that can consume a stream of bytes.
00:59:50
Speaker
and spit out a tree that describes how the stream of bytes relates to the tree. And there's different types of parsers. TreeSitter is what's called generalized LR.
01:00:05
Speaker
LR it's like left to right. I don't know left to right bottom up It's basically very similar in nature to these table driven parsers that are very efficient like Lex and yak But the difference is that it's generalized you should get Max on and he'll talk about it more intelligently than me but the problem with these formal grammars is they can have ambiguity and what tree sitter is able to do is sort of create these localized bubbles of ambiguity to permit
01:00:34
Speaker
grammar constructs that are quite a bit more liberal than what those compiler front ends like Lex and Yak enable. You're able to sort of express more complex grammar formalisms more easily. And it also produces a concrete syntax tree, which means that every node that it gives you back is mapped to a position
01:00:56
Speaker
in the source code. Um, so that's a concrete syntax tree, which is obviously important for an editor. It also has very good error recovery because again, the editor use case of, okay, this isn't making sense. Let me leverage my ability to generalize in the presence of ambiguity to generalize in the presence of errors. And so that's another source code that's halfway written halfway rewritten. Yeah. Yeah. And it's, and it's incremental.
01:01:26
Speaker
And it's incremental, which means you could pay, I don't know. I don't know what the benchmark is tens of milliseconds, you know, maybe for a medium size file, big file, actually no idea. It's fast enough that it doesn't present a problem, but the reparse times are like in the single digit milliseconds, like very, very frequently, like one millisecond. It wouldn't be that rare sub milliseconds sometimes. Um,
01:01:51
Speaker
We still do it off thread, meaning if, if reparsing takes too long.
01:01:57
Speaker
then we will still give you the interpolated tree where we don't reparse. It's important to never block, but it's quite efficient. And usually by the next frame, you'll have the next tree. So that's TreeSutter. The next big innovation investment, I think, is the language server protocol, which is Microsoft's doing. Oh, yeah. Right. Which is almost table stakes for an editor these days. Right. And for a language.
01:02:23
Speaker
And it was like, in my mind, a very smart thing that they did to be like, how do we get a bunch of languages supported in our editor? Let's build a common protocol. Everyone's standardized on that protocol. But the cool thing is now we get to plug in on the other side of it. So we're benefiting a lot from that. So from
01:02:42
Speaker
And then we have a little WebAssembly extension that basically is how do we interface to the language server? How do we run it and download it? And there's a little bit of cleanup that we do on some of the data. I'm not sure if that's included in the extensibility piece yet. I should probably figure that out. But anyway,
01:03:01
Speaker
A tiny bit of glue code, basically, is what we essentially express in WebAssembly now to manage the interaction with the language server. And then we have the tree sitter grammar and then some tree sitter queries, like queries such as what should the outline look like? So if you hit Command-Shift-O, you pop up an outline view.
01:03:24
Speaker
which, what should we show in there? It's different for every language. So the language also expresses these pattern match queries on the syntax tree, which is like, this is a function. I want this to be included in the outline. This is an impl, and that's different in every language. It's different than the tree. Okay, that makes sense. So if your language supports LSP, which it probably does these days, and there's a tree sitter grammar written for it, which there probably is these days,
01:03:52
Speaker
You can build two things and sweep up a whole host of languages. Having heard it, I can't think of any other way an editor could be written these days, unless for a single purpose. Right. And I think we're very flavored by our Rust experience.
01:04:15
Speaker
But even so, I'm tempted now by getting the bandwidth to start doing our own language specific things for super popular languages and doing it in a way that's tucked away and potentially you could jettison it from the binary if necessary at some point. And I'm not opposed to that, but we've also tried to be quite agnostic in our approach to things, language agnostic and lean on
01:04:45
Speaker
things that are universal across languages as much as we can. Because languages are cool and it'd be great if new languages had good tooling. Why not? There are all kinds of programming personalities. I'm certainly one that likes to play with a lot of new languages. An editor that just worked for one or two mainstream languages would be useless to me. I just write Rust these days, but
01:05:15
Speaker
I don't know, I might want to write rock after watching Richard Feldman stuff, but yeah. I'm willing to bet that a year from now, he will have got some rock in there. So this, but this is, this takes us onto like, uh, the strategy of an editor because you come up with a new editor. It's, it's mature enough. Now is my impression that you go to people and say, Hey, try our editor instead of the current one you're using.
01:05:45
Speaker
How do you get to market? How do you get enough users? One way would be to specialize and be like, we're the best JavaScript editor ever.
01:05:59
Speaker
but that's not the road you're going down. What is your, what's the carrot you've got on the end of your fishing rod? That metaphor is broken, but you know. Yeah. Yeah. I've spent a lot of time thinking about it. I've spent a lot of time thinking that was the answer that we had to go deep on one language, but I don't know. So far it's been like we build what we really want, which we still don't fully have, to be honest, and then try to give,
01:06:28
Speaker
people, like tools they need to solve their problems too.
Balancing Specialization and Broad Appeal
01:06:32
Speaker
I don't know. It's a little bit of a, like, I really want remoting to work. Um, why? Because I want to work on Linux stuff and I want to be able to do that in a container. Like, uh, and so I need that. So let's just build that. And it's also hopeful that companies tell us that they need that. It makes sense that they need that. So I don't know. Yeah. I guess.
01:06:59
Speaker
Yeah, is there a faster route though? Like is there a shortcut? Can we get more dominant in one particular niche? It's something I've racked my brain around. And I think we probably could. Like if we wanted to really go after Russ, I think.
01:07:13
Speaker
We can make a bigger dent in putting a few weeks aside. You're literally doing the product planning with me right here. It's hard to figure out what to do next. What we're focused on right now is remoting because a lot of people want that and it's a blocker for a lot of people being able to work in ZED. It's just like my code for reasons I cannot control is not on my local machine.
Integrating AI and Large Language Models
01:07:38
Speaker
I can't use that at all to edit it right now until remoting is solid. And we're getting really close on that. So that's one big focus. Then the other category is like paper cuts, which is just it's hard to even form. I've just like stuff that goes wrong for people, little things that don't quite go right. But then the next big question. Yeah, it's a good question. Like, let me pull up from my notes this week. What were what were paper cuts this week?
01:08:08
Speaker
HTML tag auto-closing. Yeah. Yeah. Right? Like that's a simple little thing that people are asking for. A lot of people are asking for because they're like, Zed's amazing. I just really needed to close those HTML tags for me. And so you might say like,
01:08:26
Speaker
Eh, we're focused on rust right now. Yeah. It's, it's hard though. It's full of those little things that are like not deal breakers, but yeah, it'd be really, really nice if it did this one extra small thing. Yeah. Or for some people that is, yeah. So it was just so easy to do. And so many people were asking for it that we decided to do it. Um,
01:08:46
Speaker
We have Kyle contracting, implementing Jupyter kernel support for us. Oh yeah, I can see that being important to people. And then a lot of the focus is on how do we use these language models? Oh yeah.
01:09:05
Speaker
When you say language models, are you thinking of like tree set of grammars or large language models? Large language models, right. Language models, because what I'm excited about is like we're building a text editor and we have what is essentially like a universal text transformation function at our disposal. What are the implications of that?
01:09:29
Speaker
And I have to explore that. I want to put time there. And so does that target a particular niche like Rust programmers or whatever? No, it's a little broader than that. It's kind of exploratory in a way. How do I want to interface to the LLM if I want to do generative coding where I'm having it
01:09:54
Speaker
maintaining control over what I'm doing the entire time. Yeah. I guess my take on it is like interacting with the LLM. I want it to feel like programming because it kind of is like programming in a weird way. Like what is programming? It's sort of feeding a sequence of bites into a machine and getting it certain behavior out. Like looks like sort of if you squint like crafting a prompt and feeding it to be generated into an LLM.
01:10:23
Speaker
But for me, it's like, I want to feel the control given to me by that. I want to get leverage from that, not have the control abstracted away, at least at the moment. That's the level that I'm interacting with it at. And I've had moments where I've gotten the right things, programmed the right things into this input that what came out the other side was like incredibly useful to me. And I've also had moments where I've had a terrible experience where I have not been able to craft.
01:10:50
Speaker
a program that would give me an output that was useful to me. But either way, like I'm the programmer, it's text. That's going to be like the, that's what the take I'm trying to push forward and kind of feel like we need to, we need to, it's a real risk to not be investing there, I think, and figuring out in a world where some amount of programs are generated,
01:11:20
Speaker
How do we keep control and make that fun? Yeah. You know, I can see a parallel between collaborating with humans and collaborating with AI at the moment, because it's like it all happens in one way or another. It feels like you talk with someone on Slack and then you go back to programming.
01:11:43
Speaker
And it sounds like you're trying to make it all the kind of shaping of the code happen in the same place. Right. Whether you're collaborating with a human or, uh, with an LLM all at once. A hundred percent. And I think right now, like we're not anthropomorphizing the LLM. I have this, uh,
01:12:08
Speaker
appliance in my mouth that makes pronunciation of certain words a challenge. That's a tricky word, the best of days. And pro-morphizing. Yeah, we're just treating it as an LLM, which at its base is just text in, text out. That's the ground truth. But what is true is that the CRDT features are great for integrating multiple streams of edits.
01:12:34
Speaker
You know what I mean? And so it's like, it makes sense. Like we've built a feature for collaborating between humans, which I have no choice. Those are anthropomorphic. You know, it is a human on the other side. And the same constructs I've built to support that
01:12:54
Speaker
into giving us a lot of leverage, I think, when it comes to how we integrate interaction with the LLM as another source of potential edits. And so like one of our models is like you highlight text and you do control enter and you type what you want for an inline transformation. And it just transforms that in place. So I think like that,
01:13:20
Speaker
experiences like it's text for a text editor, you can sort of provide this transformation place. It's not exactly, you're not pair programming with a machine when you're doing that, but when it's streaming the edits back that are transforming the text in front of you,
01:13:39
Speaker
we still want to let you move on and edit or go do something, right? And so in some way you have this sort of asynchronous process that's providing the edits. You are kind of collaborating with it, but we don't like yet, I'm not opposed to it. It just feels like let's start with the basics. Yeah. It doesn't have the vibe of like pair programming with someone.
01:14:02
Speaker
Yes, but it's a lot it sounds like it's going to be a long way away from the if you use a refactoring tool, it's a stop the world, make a change come back thing as opposed to a kind of you can you stop the world.
01:14:19
Speaker
Go away and do this thing. Come back when that one specific thing is done. Change. Yeah. I mean, right now we just rewrite it in place and that's it. What we're working on is sort of rewriting it in place and then just queuing you.
Looking Forward: Zed's Future and Community Engagement
01:14:33
Speaker
Hey.
01:14:35
Speaker
a rewrite has occurred here, a transformation has occurred here, give you an affordance to see the diff if you'd like. And then you can sort of like discard it, which the cool thing there is we've written it directly into the buffer, but CRDTs give us non-linear undo history. So we can just do it. And then if you're like, no, I don't want that, then we can discard it and it's fine. We just undo the operations of that thing being inserted. So that's another place where like,
01:15:03
Speaker
infra we designed to support collaboration is going to be giving us a lot of leverage when it comes to providing a good experience. But yeah, my flow is like you ask to do something and sometimes you sit there and babysit it, but other times you sort of go on and do something else and then you can come back.
01:15:22
Speaker
And it's right there in the code. Yeah. That's the idea though. You're not waiting on a commit or something. That feels like a lot of friction. Yeah. And then, so what you've done really, as well as building an editor is you've built a platform for these kinds of collaborative edits with different entities.
01:15:41
Speaker
Oh yeah, that's interesting. Yeah, that makes sense. Right. Yeah. So, so to, I have two broad categories of question I need to ask you. The first is like, where do you think that platform will be two years from now? Two years from now. Um,
01:16:02
Speaker
Yeah, I think we'll have CRDB in place and we'll have widen the scope of the types of interactions that are possible from mostly synchronous to a range spanning from synchronous audio chats all the way to like leaving annotations on some corner of the code that was written a few years ago. So sort of expanding the scope of the types of conversations and the pacing of conversations
01:16:30
Speaker
that we enable. And I do envision us having figured out how to make LLMs a robust part of that experience, a way that keeps you in control, but also gives you leverage at the same time and gives you room to experiment with what that looks like on whatever model is coming out the door.
01:16:55
Speaker
And I'd like there to be a lot more extensibility in place so that people can solve their own problems. And for us to be having potentially more of some of the beachhead activities you've been talking about in languages like Rust and some of the popular languages where we're really starting to take that second pass of
01:17:21
Speaker
All right, we have full control of this thing. What can we really do? Now we have extensibility starting to cover more of the long tail of things, but like, you know, how can we really support X, Y, Z community in a special way, um, beyond what we have now. So I don't know. That's a lot of promises in two years is a long time, but, um, very short time. It's the same time, right? Yeah. Yeah. It's true. I mean, I think we can get that all done.
01:17:50
Speaker
Um, but you know, everything changes. Not to give you another task on top of this, but do you think you'll open source parts of it? Like the, it's all open source, but do you think you'll like spin out a rust, uh, rendering framework or a CRDT database? I see what you mean. Well, right now. Yeah. Right now, GPY is just in our repo. It's MIT license.
01:18:18
Speaker
We haven't put a lot of effort into like turning it into a sort of Django or rails type thing, if that makes sense, which does, I think is a big investment. On the other hand, there is a risk of like,
01:18:33
Speaker
being sort of the, I don't know, like, you know, those technologies that would be inside of Google and then there'd be like an open source reinvention of it outside of Google or whatever. It's like at some point if people want it and we just have that code in our code base, like do they just end up taking those ideas and running with them? Is that a bad thing? I mean, maybe it's fine. So I think it's a great thing to inspire people on projects that you don't want to necessarily take on yourself, right?
01:18:59
Speaker
Yeah, so I don't know. The challenge obviously with a wider effort around GPUI is just like, it's going to be an effort. We already have a lot on our plate. The opportunity is like, the more we get written in GPUI, the better. If people were interested in sort of
01:19:16
Speaker
taking it into their own hands a little bit. And even, you know, it's, it's permissively licensed right now, right? Like, uh, I would love to kind of take things back if that makes sense. Like, um, I dunno, it's tough. I don't have it figured out. How do, how do we account for the bandwidth of supporting additional clients on top of the UI framework? Yeah.
01:19:40
Speaker
Yeah, a startup more than any other company has to focus and say, that would be great, but we can't do it. And the fact that it's open source means if someone else wants to, they can. Not only can they, they can literally build a closed source app on GPI, on our code today. Like it's Apache licensed. So, um,
01:20:00
Speaker
I don't know, maybe that's stupid, but that's how it is. And like, I think what I'm hoping is that if people start building open source, like closed source apps on GPI, that they'll be interested in enough in what is coming from us upstream to like maintain a openness and reciprocity so that we can actually get good stuff. Cause I'm sure there's a lot that people would have to contribute to it as they started. Yeah. It's just,
01:20:26
Speaker
I'm open to it, I just don't know. You can look at the gpuia.rs website to see the amount of attention we put into it. Okay. And maybe in the future someone will run with it and then collaborate back with you using Z. Yeah, that'd be incredible. So that's really the final question we have to ask. If someone wants to get started with Z, what do they do?
01:20:49
Speaker
They go to z.dev. You can download the stable version. If you want to live dangerously, you can download preview. If you want to live more dangerously, you can build it from source and you can also build it from source on Linux today. We don't have an official distribution because there's some more things, the T's to cross and I's to dot, but hey, you're on Linux. You can build from source, right? You can build stuff from source. Um,
01:21:16
Speaker
And so, you know, again, I think we're not ready to put our stamp of like, it's official, Linux is supported, but like, it's getting there. It's getting pretty close. I know it's on Mac, is it on Windows as well? It's not on Windows. And the main issue with Windows is it's just so much different than Linux is from the first platform. Like Linux is a little easier because it's Unix-y. Slashes go the same way and there's some of the same ideas.
01:21:43
Speaker
There's windows, there's work underway on windows though, entry. I don't have status for you on where it's at. I should have that too, but yeah. Yeah. If you want to get started as a contributor, we have,
01:21:56
Speaker
Lots to do. We have thousands of issues. Um, and, uh, come hang out in the channel. Also like, uh, you can pop in, listen to us. Um, we need to make better affordances for sort of notifying ourselves and stuff, but it's at least a way to get a little bit of a flavor of like, what's going on. We hired Bennett. He was just the, uh, guy from Germany that kept showing up and, uh,
01:22:25
Speaker
We hired him. He's great. Open the editor look in the channels and before long you could be drawn in. Exactly. Yeah, nice. Awesome. I think at that point we'll leave people to go and check it out. Nathan Sobo, thank you very much for joining me. Thank you as well. Really thank you for the opportunity. Appreciate it. Great fun. Cheers.
01:22:45
Speaker
Thank you, Nathan. If you want to give Zed a try, either as an editor or just an excuse to see a really good Rust user interface in action, head to zed.dev and there's a link to that in the show notes. Also in the show notes, we mentioned the language Rock and the database Tiger Beetle. If you're curious about those two things, we have previous episodes speaking with the creators, so check those out. You'll find the links down below.
01:23:12
Speaker
As ever, if you've enjoyed this episode, please take a moment to click like, rate, or share, or all three, and make sure you're subscribed so you can catch next week's discussion. But until then, I've been your host, Chris Jenkins. This has been Developer Voices with Nathan Sobo. Thanks for listening.