Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Designing The Lustre Web Framework (with Hayleigh Thompson)
3.1k Plays2 months ago
Lustre is a web framework that takes a lot of inspiration from Elm, some from React, and a surprising amount from Erlang’s actor model, to provide a library that blurs the lines between executing on the client, or on the server.
Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices
Support Developer Voices on YouTube: https://www.youtube.com/@DeveloperVoices/join
–
Lustre: https://hexdocs.pm/lustre/index.html
Gleam: https://gleam.run/
Join the Gleam Community: https://gleam.run/community/
Processing (AV Framework for Java): https://processing.org/
Vue.js: https://vuejs.org/
Svelte: https://svelte.dev/
Elm: https://elm-lang.org/
Elm Table: https://package.elm-lang.org/packages/gribouille/elm-table/5.3.0/
Hayleigh on Twitter: https://x.com/hayleighdotdev
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Kris on Twitter: https://twitter.com/krisajenkins
Recommended
Transcript
Introduction to Web App Architecture
00:00:00
Speaker
Web app architecture is still an evolving field. I'd say the front-end user experience, that feels like it's stabilized. We all know roughly what a web UI is supposed to look and feel like. But how it's architected under the hood, that's definitely still in flux. Should we be using REST or WebSockets or GraphQL? How much rendering are you supposed to do on the browser and how much should be back on the server?
00:00:28
Speaker
How do you manage the state of a web page when these days a web page state can be so huge that it definitely needs some management? Web pages these days are complicated beasts and so we need good architectural practices to handle them.
00:00:45
Speaker
but Joining
Meet Hayley Thompson
00:00:46
Speaker
me with some possible next steps in the evolution of web app architecture is Hayley Thompson. She's a React developer and an Elm developer. Two quite different frameworks in two very different languages. And she's currently synthesising her expertise into a new framework called Lustre. And Lustre adds into that mix some inspiration from the actor model.
00:01:09
Speaker
It has connections to a line and there are parts of luster that look less like web pages and more like location agnostic actors. How on earth does that work and why do it what's the upside. The
Chris Jenkins' Programming Journey
00:01:23
Speaker
upside is all in pursuit of the perfect architecture so let's see where she's taking this i'm your host chris jenkins this is developer voices and today's voice is haley thompson.
00:01:46
Speaker
My guest this week is Hailey Thompson. Hailey, how are you? Yeah, really good. Thank you. How are you? I'm very well, very well. ah you've You've got one of my favorite topics that only became my favorite topic about halfway through my career, we can talk about. So
Hayley's Path from Music Computing to Web Development
00:02:03
Speaker
I i got into front end programming probably around the early 2010s, just out of sheer frustration.
00:02:13
Speaker
that the quality of engineering, to my mind, to my mind in my experience, caveats, was pretty terrible. Like, if it worked, you weren't supposed to complain about anything else. And I found Angular 1, which was, in hindsight, not the greatest design, but at least their water design. And that's when I really started getting into, like,
00:02:37
Speaker
not just front-end programming, but how we can make the engineering of it sane. That was my starting point, but it's about you. So what's your starting point in the front-end world? Well, actually, my background is in music computing and creative coding. So I actually got into web development through the Web Audio API.
Discovering JavaScript and Frameworks
00:03:01
Speaker
Right.
00:03:02
Speaker
Yeah, like I was teaching myself programming. And so the idea of learning C was kind of off the table. It was a bit too complicated for me. So when I when i worked out that JavaScript had this API for synthesis in the browser, I kind of just ran with that.
00:03:21
Speaker
um So probably in maybe 2016 is when I started. Back then I was doing a lot of C++ and open frameworks and Java doing processing, creative visual coding. Processing being that's an audio framework in Java, right? It's it's more of a visual visual framework actually. there is ah There's like an audio module, but at the time I was doing sort of generative art.
00:03:50
Speaker
um
00:03:53
Speaker
and was kind of
Elm vs. Vue: A Developer's Perspective
00:03:54
Speaker
concurrently learning web development because it sounded interesting. Then I discovered the Web Audio API and started playing with that. I picked up Vue, maybe in 2018.
00:04:07
Speaker
and I was doing that for a while. I leapfrogged over React and TypeScript and all of that. like At the time, um NPM and Node and all of this was like super intimidating to me. Like I really liked that I could just drop view in a script tag and just work because I didn't really understand package management or anything like that at the time. right ah And then in 2019, I came across Elm. ah yeah I did like a functional programming class at university that taught me Haskell.
00:04:51
Speaker
yeah and I had seen Elm like previously, maybe a year before, and bounced immediately off it because it looked super weird, was very strange. After I did this class, I kind of came back to Elm. There was this big push at the time, it was like maybe 2019, and there was this big push from a lot of the big names in the Elm community. They were giving a lot of like conference talks. I was getting a lot of recommendations on YouTube about how Elm was solving all of these big problems of like Elm as TypeScript and React and Redux and Immutable.js and Webpack all into one thing. um and I just picked that up. and The rest is kind of history. i've I've been doing Elm now for five years, and then for about a year or so now, I've been developing
00:05:47
Speaker
In my mind, what is Elm 2 in Gleam? Right, which is Lustre, right? Exactly. OK, so we're going to get into Lustre, but the fact that you created Lustre puts you forward as um as an architect of front-end design patterns, right? So I'll give you the interview question. How would you contrast, how would you describe and contrast the way Vue does it to the way Elm does it?
00:06:17
Speaker
Yes, that is a very good interview question. Sorry. This is an interview, but not that kind of interview. I think what a lot of current front end frameworks come down to is this very kind of component driven architecture where in many ways you're thinking at quite a small scale, and you're encouraged to say, for like, in View viewue is a really good example, actually, because you know they have these single file components, SF seeds, they call them, right? And so you have the markup for your component, and then you have a script tag, which encapsulates all of the logic, and then you have a style tag, which and you know starts the component. And the idea then is that by focusing at the component level,
00:07:10
Speaker
You have all of these kind of rich, interactive, sophisticated experiences, and then you know you compose them and you piece them together into something much larger. um I think that is in stark contrast to Elm, where you have this kind of very holistic view of your application, where you think, okay, what is what are like the main domain types in the application line building, for example. like What does my model represent? Your model being your entire application state. And then from there, your UI, these are pure functions, right? Now you have
Transitioning to Functional Programming
00:07:50
Speaker
um no component state, no local component state. Instead, you have just simple building blocks to piece together your UI, which is largely divorced from your application logic, your business logic. I think these days two approaches are very, very different.
00:08:09
Speaker
it's It is that dividing line almost between OO and FP, right? In one, you think of the world as broken up into into little islands of objects. And in the other, you think of it as data and stuff you do to data. It's interesting because in the Elm Guide, there's actually, there's like a footnote written by Evan that Clearly, a lot of people ask about, oh, where are our components in L? So there's this little footnote that's like, components are objects, and we don't have objects in functional programming, and blah, blah, blah, blah, blah. Yeah. And how did you find it going from one model, which is probably a more popular model, the OO-ish model, going to the FP model?
00:08:59
Speaker
Yeah, actually, the transition was
00:09:04
Speaker
in some ways, predictably difficult, because it's a very big shake up of your worldview, I would say. But I was early enough in my
Scaling with Elm and React
00:09:15
Speaker
journey in general that the kind of scale of things that I was building and working on was small enough to actually the transition to this kind of Elm approach where you have a model that describes your whole application. Actually, that worked really well for me. And I actually found that transition very smooth in a lot of ways, and now feels very, very natural. Okay, okay. It sounds like you found it easier than I did, but I was more baked in the whole objects are everything worldview. Yeah. Okay, so, so that I
00:09:55
Speaker
I think these things are less starkly different at small scales. You say you're dealing with small applications. Yes, they're different, but the differences don't really show themselves until things get large. Have you worked on large helm projects and large view projects?
00:10:14
Speaker
I've not worked in large V projects, but I am also a React developer these days. okay okay So I have worked in both large Elm and React code bases. Do you want to, without being too biased, ah compare and contrast? Actually, I don't think I am that biased. I'm ah i'm a really big fan of React, actually. um I think especially after doing Elm for four years before I even looked at React, now Now i come when I came to React, it felt like React was now giving me lots of things that I wished I had in Elm. So I think both approaches do scale, I would say. In my experience, I've worked in big code bases in both, and both of them have been pleasant enough. I think they both have their own pain points. But I think there's a big shortcoming in Elm's model
00:11:15
Speaker
Which is this like complete rejection to encapsulated state. This like, I would say very dogmatic approach to encapsulated states as objects and objects is OOP and OOP is bad. Which, which I think breaks down very quickly when you think about the DOM, when you think about the HTML that you're already rendering. The example, the example I always give always give is the input element, the native HTML input element. This thing has so much state that you don't ever interact with. right like We set the value of it, but think about managing focus like if you tab into it and now it's focused. right this is This is internal state. Well, think about handling keyboard interaction or just
00:12:16
Speaker
Even something is linking exactly placement of the cursor, like the curse of blink. Like when you make a selection, like all of this stuff is handled by the browser, but it's handled as an encapsulated stateful thing. And in
Components vs. Holistic Approaches
00:12:31
Speaker
our L maps, we go, well, we don't worry about that. We're going to render the marker.
00:12:40
Speaker
And I think actually there are times, especially now in kind of modern web development, where we're really crafting like quite sophisticated applications, right? We've moved beyond documents, or at least yeah the industry that I'm in has moved beyond documents into applications. Yeah, and here yeah you have very, very rich, interactive elements on a page.
00:13:06
Speaker
And if you really want to craft like a a native experience, I think it very, very quickly falls down in Elm's model where you have to store all of that somewhere centrally.
00:13:21
Speaker
Yes. Okay. I can buy ah one place I've experienced that is trying to do maps in Elm and they are sufficiently rich that it does start to become very difficult, but Okay, so you're not you're not supplying enough controversy, so I'll do it, right? but the the div divide the The polarizing line, and all programmers try and be polarizing at some point.
00:13:48
Speaker
is you Elm tries to reject the existence of state except for one big global state. And I think that works quite well generally, highlighting some problems. Then you've got the other end where every tiny little thing becomes a component. And I find the problem there is every individual unit works well, but you've got this hidden graph of interactions which becomes impossible to debug.
00:14:15
Speaker
If you're saying components are sometimes good, I think that's what you're saying, how do we know where to divide up those lines? I think a lot of it comes down to culture. I mean, so I'm going to keep hammering on about Elm because it's the thing I'm most familiar with and and also the thing I have the most opinions about. can And so like as a as a culture, as Elm developers, we we have this rejection of encapsulated state.
00:14:43
Speaker
But if you look at big Elm code bases, or if you look at kind of complicated packages, there is this implicit recognition of, well, I have some behavior that I want encapsulated. So I'm going to put it in an opaque type, and I'm going to tell you to store it in your model, and I'm going to give you some function start data,
Elm's Unique Component Handling
00:15:02
Speaker
and I'm going to give you a function to render it. And what we've done there is we've gone This is sufficiently complicated that I want to hide how this works to my users, to my consumers. But Elm is not expressive enough to do that in a way that doesn't involve you breaking it into three isolated pieces and asking your yeah users to wire things together. um And as Elm developers, I think we have a pretty good intuition of when we should do that.
00:15:37
Speaker
Because the language and the framework itself makes it quite tedious, it's not an approach we reach for immediately. Whereas in React or Vue or Angular or Svelte, it's very easy to create a component with encapsulated state. And so the culture there is about creating many encapsulated components.
00:16:01
Speaker
um So actually I think it's down, I think it's quite obvious thing to say out loud, but it's down to what the language makes easy. And if you make something hard, your users will only do it when they need to.
00:16:17
Speaker
I'm going to play devil's advocate for the Elm team here and say the reason it makes making components hard is they're a bad idea.
00:16:29
Speaker
But it's I would say that it's clear. I think it's the Fair Elm line, but I think if you just take half an hour to look at things being built with Elm, they are still making components. They're just doing them they' they're just doing them in a way that that is technically not a component, but is functionally identical to one.
00:16:52
Speaker
right If you get publish a package or or if you're working internally, if you have a module and it has its own model type and it has a message type and it has an update function and it has a view function, you have in essence created a component, but you can't put those pieces together yourself. So you you hand the constituent parts to your users and you say, why are this up? But it's a black box as far as you're concerned.
00:17:24
Speaker
Yeah. Yeah. Okay. Yeah. i'm I'm now thinking of a table library I've used in Elm, which is not as customizable as I'd like, but it's great when you just want to throw up a table with like sortable columns. And it's exactly that. I always have to double check and triple check how to install it because I'm installing it in three different places. Exactly. The community actually has a very very big infatuation with web components as well, which is somewhat uncommon among other front-end frameworks. is I think largely as a community, we as a front-end community, we kind of landed on the idea that these um that could anymore. But the AMP community in particular is actually very bullish on the idea of when you do have some encapsulated state, just put them in a web component and render them in your view.
00:18:21
Speaker
And I've seen people experiment with building our maps as web components. And I think that tells you something. That tells you that there are people that recognize when you have some sophisticated logic combined with view that actually there's there's something to the idea of being able to package them up in a black box. Because this is a web component is very much like the input tag, right?
00:18:51
Speaker
in the US treat it like HTML, but it actually hides a whole bunch of interesting state. I can see why philosophically Elm people would be attracted to that idea because they're already using that idea in an even simple form fields. But you seem to think it's not such a great idea. I think it's an excellent idea. Oh, okay. I have a misinterpreted you. Yeah, yeah and i'm I'm a big fan of that idea. And in fact,
00:19:22
Speaker
This is how components work in Lustre. So just
Introducing Lustre: A New Framework
00:19:29
Speaker
just to give a quick TRDR, Lustre is this kind of same model view update idea as Elm. So you have like a model of your application state, a message type that represents all the ways the world can speak to you. You update that state and you have a kind of pure view function.
00:19:49
Speaker
And kind of born out of this frustration of Elm from Elm, actually what you can do in Lustre is you can take those building blocks, that model view update package, and say, okay, treat this as a web component. And it registers as a real web component that then in your like application you render as any normal HTML element. And you communicate to it. You communicate the same way that you would like native HTML, so you set attributes and the component sends you events. um So there's there isn't this split idea of, well, I render HTML this way and I pass data into HTML this way versus um my framework has a component and I have a props object and I pass in. yeah It's quite common these days where the idea of component is a framework specific thing
00:20:46
Speaker
that is detached from the idea of HTML. And in Lustre, these two things are operationally equivalent at the point of consumption. Does that bring back the problem of once you've decided, because there is a nice thing separating out like the behavior from the HTML in that you get me The obvious one being you can render it multiple ways. but You can have the same underlying logic for multiple presentation types. And it sounds like you're losing that as you quite tightly wire the HTML tag to what it does. I think it's always going to come down to API design. And and and that is unavoidable.
00:21:36
Speaker
um
00:21:38
Speaker
So I would say that yes, if you have committed to treating something as a component, then you have lost the ability to treat it as constituent parts. But if that functionality was valuable to you, there you didn't have to make a component, right? Yeah. okay um i'm I mentioned and mentioned previously that it's about culture and it's about what the framework makes easy. And so there is a bit of There's a bit of overhead to creating a component in Lustre because you do need to build like a full application. but You do need that model type. You do need that init function, that message type. but You need all of those bits. And only once you have those bits, can you make them into a component, which is different to pretty much conventional front-end frameworks where you might initially start off with
00:22:38
Speaker
a function that just returns some markup. And then when you decide, actually I need state, it's very easy to add state, right? It's very easy to to turn it into a component. But the fact that Lustre kind of adds a not insurmountable, but obvious hurdle means that you are you're discouraged from making this approach the default. Like the default approach is still always to write pure viewer functions.
00:23:08
Speaker
View functions, sorry. OK. So you're saying if I come into Lustre, I will get largely the Elm experience, which is write some data describing my idea of the world and the way it can change, and then write some rendering code. But the option to turn it into a web component, which is like a live HTML tag, a smart HTML tag, is very much open to me.
00:23:38
Speaker
Yes. That's pretty much the that's the ideal scenario. Okay. it's there Is that novel? I mean, I've never actually tried in anger to create web components in other frameworks. Is that common? That it's well supported? Do you think you do it better than anyone else? I think the best example I can think of is Svelte.
00:24:04
Speaker
So that okay you can compile any Svelte component to a web component, at which point Svelte has disappeared and you could use it anywhere, which is the same in Lustre. and
00:24:17
Speaker
i don't think I don't think that idea is novel. I think the the novelty for Lustre is actually, we have this idea of universal components,
00:24:31
Speaker
Ooh, yes, I want to get into that, but first we've got to have a quick sojourn into Gleam and what that offers to build it up, right? So um Lust is written in Gleam, which is a language I know you love. Why Gleam and what does it offer as a front-end developer first? Yeah,
Gleam for Frontend Development
00:24:52
Speaker
it's actually, I think that's a great question and I think my answer will be somewhat dissatisfying. Okay.
00:25:02
Speaker
I think there's there's a lot of exciting, mature things. As a front-end developer, like you are spoilt for choice. In my mind, the strength of Gleam is is actually if you're a back-end developer, you get access to the front-end, which is less common. You get access to the front-end.
00:25:28
Speaker
you like via Gleam's JavaScript target. So for for people that don't know, Gleam compiles to both Erlang and runs on the Beam, the Erlang virtual machine, and it also compiles to JavaScript. And it's not exactly novel to have a full stack language, but I do think if you are like a front-end developer,
00:25:51
Speaker
You're spoilt for choice. and and And so there are reasons to choose Gleam, I would say. like the big The big one for me is that Gleam is just much semantically simpler than TypeScript. like Just the experience of using it is much lower complexity. There's a lower cognitive burden, I would say.
00:26:12
Speaker
But actually, I think the superpower is if you're a back-end person and you don't really care about this front-end thing, but you need a front-end because everything needs a front-end now.
00:26:24
Speaker
yeah I think the selling point then is that you get access to a front-end framework written by like an excited front-end developer that hopefully I know what I'm talking about. and You get to use the same language and the same concepts and the same primitives.
00:26:42
Speaker
So it is mostly about bringing good front-end ideas into this world of people that don't necessarily have access to many front-end ideas? That's definitely been my strategy for now.
00:26:57
Speaker
like Interestingly, front-end people are a naturally curious bunch, I think. If you're into front-end, you have almost definitely explored 10 dozen compiler JavaScript languages and you've tried 100 different frameworks. and so actually Actually, the number of people using Lustre right now, especially especially say four months ago when Gleam had only just become version one.
00:27:27
Speaker
the people using Lustre then were like front end people. And so actually there's already like a community of knowledgeable people. But my focus as a kind of brand strategy, if you want to call it that, is is really questioning pushing to particularly like developers on the beam. So like not just backend developers, but specifically like Alang and Alexa developers.
00:27:57
Speaker
and And the reason why I think that is a viable and interesting and exciting strategy is that this i like Elm's idea of how front-end should work, this model view-up should update architecture, is just so strikingly similar to what Alang and Elixir developers are already doing on the beam with actors.
00:28:21
Speaker
Yes. Yeah, there is a parallel. There's a huge parallel there, isn't there? Because it's like, yes it's all let's treat the world as a state machine. Exactly. It's it's it's entirely message-based functional updates of state. It's just that the the Elm and the Lustre thing that we just add a view function to that building block that you already have. if you kind Add a pretty way to look at your actor.
00:28:50
Speaker
Exactly. I've mentioned in talks I've given in the past, I ah just frame it as your front end is another actor, um but not an actor in the beam. right Now this actor lives in the browser, but it's the same. You still send messages to it. It still has an update loop. I think for for a lot of those kinds of developers, that's a real light bulb moment because it's like, hey,
00:29:19
Speaker
this kind of approach to building applications that I'm already familiar with. like I don't need to learn a new idea to build the front end. I still need to learn how to put HTML together. But yeah in
Integrating Frontend and Backend Architectures
00:29:32
Speaker
terms of creating an application that has behavior, you're already doing what what you need to do.
00:29:40
Speaker
Yes. And it's interesting because that's exactly the flip side of the coin of where I started out, where people knew how to generate pretty looking HTML, but there was no fundamental underlying architectural pattern to make it sane. You're turning that on its head and saying, here's the architectural pattern. Now all you need to do is to make it look good, which I always outsourced to a designer with like giving me Photoshop documents anyway.
00:30:09
Speaker
And then you're just left having to center a div, but you know, you can make that happen eventually. Well, actually I have a companion library. but just I i have a i have but have a UI library that is, at the moment it's a bit in flux, but the idea is that like, it's just a UI kit if you don't know or care about front end.
00:30:32
Speaker
You just want something consistent and pretty. Is that something like Bootstrap or Tailwind or something like that? Conceptually? ah Yeah, I suppose so. Conceptually, yeah. Conceptually like Bootstrap and the idea that if you just need something that looks good and a way to put it all together. Yeah, yeah. It's absolutely great for the prototype and will hire designers if anyone complains.
00:31:02
Speaker
Yeah, pretty much. OK, so the I can certainly see that sell, especially for people who are on a back end like Beam, which there is no obvious front end framework to go to as far as I'm aware. right the The major architectural difference there is if I'm trying to think like an Erlang Beam back end clean developer,
00:31:30
Speaker
I'm expecting to have lots of different actors running my application. And it seems like the front end is going to largely be a single actor. Is this where the idea of making web components, how does that play out, the the proliferation of actors for tasks? I mean,
Lustre's Flexibility in Component Deployment
00:31:51
Speaker
that is largely it.
00:31:53
Speaker
say
00:31:56
Speaker
you would expect to have many actors in your application for sure. and in most cases, your front end, your view, that will be one actor, right the browser. ah But this is this is the draw of Lustre's approach to components, I think, is that actually if you do need more, then then you're still proliferating that same i idea, that same like update loop idea into components now, and they do compose.
00:32:27
Speaker
so like in I would say in your underlying applications, your actors form a graph. right like yeah can have you can have you need You have supervision trees, but in terms of communication, you have essentially a graph of actors that can send messages to one another. The only difference with the front end then is that rather than a graph, you have a tree right because you're ultimately rendering HTML. Any of those components might be.
00:32:58
Speaker
But but the the building blocks are the same. and
00:33:04
Speaker
I'm going to bring it up again because I think it's i think it's the most exciting bit is that actually, well, if you have this application type, which just bundles up the idea of an actor with a view function,
00:33:24
Speaker
And you can use it as your application, and then also you can embed them, nest to them as components. And this idea is already kind of very familiar to the Beam. Why can't you just move that whole bundle as an actor on the Beam? Which is exactly what Lustre lets you do. Take me through that.
00:33:49
Speaker
in
00:33:51
Speaker
Yeah, we need to dive into that. because So you've got an actor running on the front end generating HTML from a certain point of view. You want to move that to the back end. Why? I think the most compelling argument is if you need or want direct access to some sort of backend service, right?
00:34:16
Speaker
like Maybe you're doing real-time data visualization. but you have like ah To me, having a real-time aspect is the real draw to this ability. But also also, this ability unlocks the fact that you just don't have to worry about network transport anymore. right like You don't have to craft an API layer on top of your database and then do some JSON serialization and then deserialize it and then turn it into HTML.
00:34:46
Speaker
right you can run your component next to your database, conceptually, and just make database queries and then just turn it to HTML. say Before we get too deep into that rabbit hole, let me give a little bit of grounding or anchor it to something else that already exists.
00:35:09
Speaker
ah sorry The Elixir community has Live View, which is this idea of what if we ran the entire front end as an actor on the beam. But this is for entire applications. So you would have a Live View application. And then what you do is you send like a ah minimal runtime to the client, which sets up a WebSocket connection. And then your entire front end runs
00:35:41
Speaker
on your backend and just sends DOM patches, HTML diffs, essentially, to get live on data. Right, so the frontend is just saying, user click this button and the backend is sending back. Exactly. You need to stick in a new div here. Exactly. Right, yeah. And so this is how Live View works. And really, Live View wants you to build your entire application this way. And that's a very popular approach for Elixir developers because they specifically don't want to deal with the front end, right? this This lets them reuse everything that they already know about Elixir to make interactive front end experiences, right which is great. I think it's really, really cool tech and it's very, very exciting and kind of confusing that it never really caught on outside of Elixir. I think there's like a PHP library that does something similar.
00:36:38
Speaker
But I think that has a really glaring problem that now your entire front-end interactivity is mediated over a WebSocket connection, which as a front-end developer seems absolutely wild to me. As ah as people in the UK that have talked to services that just exist in California, the latency on that too is suboptimal the further away you get. Exactly.
00:37:06
Speaker
yeah I think this approach is very popular in places where you have excellent internet and also for internal tools, right? well Yes, and the internet, you wouldn't notice the difference. yeah Yeah, exactly. So, Lustre essentially has looked at that and gone, well, that is an excellent idea. We already have this idea of components that is conceptually compatible with how things already work on the beam. So instead of shipping our entire front end to the back end and running it over a web socket, why don't we just take the idea of a component and say, well, you can render it on the c client as a web component, or you can run it on the back end as a server component.
00:37:54
Speaker
And you're seamlessly doing this thing where sometimes it's actually running in the browser, and sometimes it looks like it's running in the browser, but its DOM patch is going over the wire. Exactly. The the idea the idea is essentially, in cases where you aren't doing platform-specific code, I mean, obviously, if you write some code that uses an Erlang library, you can't run that in the browser.
00:38:19
Speaker
but I'm not going to solve that. But if you if you write your application in a platform-agnostic way, so for example, if you did use an Erlang library, if you provided a JavaScript implementation, like an alternative implementation for the JavaScript target, the idea then is that you can tape take that app, this gleam type that encapsulates your application update function view,
00:38:48
Speaker
You can just take that. And the only thing that needs to change then is is how you start it. So as ah you can start it as a single page application, a spam, just calling luster.start in the browser. You can register it as a web component, or you can register it on the backend as a server component.
00:39:09
Speaker
And the idea then, yeah, the idea is that your application code never changes. Like you could put that in a separate project or publish it as a package and say, Hey, you can choose where this runs. Okay. Would you also be able to start off saying, Oh, I'm trying to think of a scenario here. So I start off with something that's a local intra internet project.
00:39:35
Speaker
and I do it all rendered on the server side. Eventually, I decide to spin out some part of it to the public. Can I then, 18 months down the line, start to push some, if not all of the application out to it to be genuinely on the front end? Sure.
Setting Up Lustre in a Monorepo
00:39:57
Speaker
This is when you would lean into components.
00:40:00
Speaker
so
00:40:02
Speaker
We had that discussion, right kind of debating whether this idea of encapsulation of state is a good or a bad thing, whether this is a naughty or a dirty thing. But actually, when you draw that hard component boundary, right you say, this is a black box that is encapsulated. Well, the benefit of making it a black box is that from the parents' perspective, it doesn't need to know.
00:40:31
Speaker
So if you wanted to start with your application fully rendered on the client, for example, as a single page application, you do that and then you go, okay, actually, I have some complicated piece of functionality. So I'm going to make that into a component because there are like engineering benefits to doing that, right? It's easier for me to think about this as an encapsulated thing.
00:40:59
Speaker
so You do that, and then nine months down the line, you think, oh actually, this would be way better if I could just run it next to my database. Because you already drew that component boundary, the parent already doesn't care.
00:41:14
Speaker
like it's it's it's not it It's not the parent's concern where the component is rendered anymore. like It just says, I want to render my counter component.
00:41:27
Speaker
Counter's a really rubbish example because you would never make ah make a counter component. But you get the idea. yeah yeah And so then then then at that point, it's just an implementation detail of the counter where it renders. So you just say, OK, well, now I'm going to swap out that for a server component. And I'll move my counter to the back end. What does that actually involve in practice?
00:41:56
Speaker
i mean what What things am I typing to make it now appear on the backend instead? So in Gleam projects, full stack Gleam projects are essentially organised as a monorepo. So you have a backend Gleam app, you have a front-end Gleam app. And typically I suggest people have a third Gleam project for their shared code.
00:42:22
Speaker
for types and serialization and that kind of stuff that you want to share between both projects. okay And so let's say you have this counter component, you've made it, it's in your front end project yeah because i so that's where it was. So you have a module, it's got the init, the update, the view function, so neatly bundled up into its own thing. Currently in the client,
00:42:49
Speaker
kind of like main setup function, you have a call to luster register and you say, call it my component and you give it your component app. And now that's registered as a custom element. So the process of moving that to the backend is removing that, that call, because you don't need to register it as a custom element anymore. Copying that counter probably to your shared project now, or you could move it straight to the backend link.
00:43:16
Speaker
you just you would move the code somewhere else so that you have access to it from your from your backend. right yeah yeah and Then from your backend, you set up a WebSocket handler.
00:43:31
Speaker
and when you and when When that WebSocket initializes, you call luster start server component. and Now that's been an actor that speaks over a WebSocket. Then the only thing that changes in the client then,
00:43:46
Speaker
is you render a custom element called Luster server component and you point it to your WebSocket endpoint. okay
00:43:58
Speaker
and so like if you If you organize this properly, essentially that would be hidden through like hidden a function call. So you would have a function that
WebSockets and Rendering with Lustre
00:44:07
Speaker
is like counter.view. Just from the counter module, you would call that or counter.element or whatever you call it.
00:44:18
Speaker
And you would just swap out the implementation there so the parent doesn't even need to change at that point. OK, so let me ask you a couple of quick practical questions on that. can i Do I need one WebSocket per component, or am I reusing it a single WebSocket across the whole project? Currently, well, so actually compared to LiveView,
00:44:44
Speaker
and This whole approach in Lustre takes a step back from being tied to any particular web server or transport layer or protocol. And so okay you could actually use the same web socket for as for forever for anything because it's your responsibility to essentially hand the data from the web socket to the server component runtime. And this is this is a negative in the sense that it's a little bit more boilerplate and so as an author, but it's a positive because now you are in complete control over how that works. so You could swap out the WebSocket for HTTP long calling or server-sent events. or I mean, if you were crazy, you could you could wire the whole thing up as a REST API. right like The server component runtime doesn't actually care. okay I'm going to run it over a Kafka topic then.
00:45:45
Speaker
rabbique something like that something great yeah Typically, you would use one socket per component, but only because that would be the easiest or least complicated approach. But there's nothing stopping you, technically, from using one socket for all components on a page. okay Okay, so multiplexing is my job, but it doesn't sound like that hard of a job.
00:46:07
Speaker
The other question is, um sessions, are you automatically taking care of the fact that 50 front-end users are running 50 components on your server? No. okay At this point, it basically, um what Lustre gives you is an actor that represents your your application runtime and then the rest of the job is you. Like, Lustre's job is purely about rendering, essentially. OK. So I'd expect to use a back-end author. Sorry. you Exactly. Yeah, yeah. Lustre, really, the bit that I'm knowledgeable about is the front-end application bit. And so the fact that it's running on the back-end is kind of
00:46:58
Speaker
an implementation detail. like as You as an author, you you maybe don't know so much about front-end rendering, so you use Lustre, but you know more than me about building your backend. so I'm not going to have opinions on that.
00:47:17
Speaker
okay i think this is i think this is a benefit like me claims Gleam's general approach in both the language and as a community is to minimize magic. um And so that often manifests as slightly more work as a user, slightly more boilerplate.
00:47:39
Speaker
um But if you can get over that initial gut reaction of boilerplate is the worst, like I want all the fancy things to do it for me. If you can just get over that, actually what you get is a very simple experience and one that you have a lot more control over. Generally in the long run, I favor that approach. Yeah. Okay. So what about, um, I should do a couple more checks on the practicalities of this. So what about.
00:48:09
Speaker
continuously nesting. It sounds like I'm okay if I take a client-side component that contains a server-side component that contains a client-side component. Is it just going to work? Exactly. It will just work. Okay. Okay. Cool. which i think is like like like That is so magical. like I think that is the really really cool part. the um The ideal scenario that I have in my head that I always always tell people is you have like an admin dashboard or some sort of like, yeah, like admin dashboard. And you you have that as a single page application because you want like snap navigation and it makes sense to render this on the client first. But then inside that you have, say, a really data intensive table. Like maybe you you have a table for your million users registered to your startup, your SaaS app.
00:49:05
Speaker
and And so you make that into a component because you want to handle like filtering and search and ordering and that kind of thing. You just want to encapsulate that. So you make that component. And then inside that table you render like a search input that handles like focus trapping and autocomplete and suggestions. And so you make that a component itself, right? So right now you have your single page app, a table component, and inside you can autocomplete search.
00:49:35
Speaker
The really, really neat thing is that you can go, well, this table, you know, like I'm spending a lot of time making a whole backend service before like querying my users and I'm adding pagination and filtering. I have this whole rest API and I'm doing like Jason encoding and then decoding. So you go, okay, well, why don't I take this table and render it as a server component? Because then, you know, I skip out that whole,
00:50:03
Speaker
layer and I can just make database queries and I can just speak directly to the database. yeah yeah yeah But the table is still rendering a client component, right? It's still rendering that autocomplete input. And so you have all of the benefits of client rendering there, you know, like you're not worrying about WebSocket latency at that point or anything like that. While getting all of the benefits of your ah the renderer being next to your database, while getting all the benefits of your application still running on the client. Yes. You might have just found you might have just found a solution to the age-old war of thin client thick client. Because each has
Strategic Component Use in Lustre
00:50:48
Speaker
pros and cons, and you're saying you can pick and choose which pros and cons you want on a component level. Yes, and because components
00:50:58
Speaker
um Because components are conceptually larger than they are in React at all, um you don't get stuck in the same sort of debates or stuck in the same sort of pitfalls that you might do if you had this ability in, say, Vue. It would be crazy if you could turn every single React component into a real-time thing that ran on the server and they could arbitrarily nest. like That would just be like an absolute explosion in complexity and engineering effort. right right because Because components are working on a far too granular scale in those frameworks, but because Lustre components, are they encapsulate a lot more.
00:51:47
Speaker
right i Then actually, some of the operational downsides to doing this. like There is a bit of setup. That cost feels a lot lower because you're getting a lot more out of one one sort of little component. Are you saying there's a natural push away from components, which isn't like a a law, like it almost is an Elm, but it's just, like when you don't make it the easiest path, then you don't get the same proliferation of components. But the option's still there if you need the power. Yeah. That's exactly it. And yeah honestly, I find that when I'm making things in Lustre, I'm um using components sparingly.
00:52:41
Speaker
like then There's a bit of setup to them and also you just kind of don't need it in say 80% of scenarios. like The Elm approach works super nicely for like 90-95% of things. So like don't don't fix what isn't broken. and The bit that's broken is that the 5% where you really do want a component, you can't have an Elm.
00:53:06
Speaker
Yes. OK. So do you have any guidance for where those boundaries lie? Because that's a question that faces every framework. like What is the component boundary?
00:53:20
Speaker
I'm not sure I have any real real rules right now. um the The framework is still young. And and also, this this idea of components is still young.
00:53:33
Speaker
um when i was When I first started on Lustre, we didn't have components. like It was a pretty faithful elm implementation. and so The framework now is maybe technically a year old, probably practically more like six to eight months old. and so I think there's there's a lot of learning like as a community working out where those boundaries are.
00:53:57
Speaker
Because there is this Overheads to setting setting a component up, right you do need a message type. like I think the message type is is the bit, the bit that tells you whether this really needs to be a component or not. you know like if If you only have, say, two messages, you're not really encapsulating a lot of complicated interaction here.
00:54:23
Speaker
and so The benefits of setting this hole up this whole thing up as a self-contained application is quite slim. You've gone through all that work just to hide the fact that this component responds to button clicks or something.
00:54:40
Speaker
yeah but i saying as like like I don't really want to give a precise number because ah I don't think that's the kind of heuristic that is practical. but As soon as you get to like say five messages, five, seven, 10 messages, like at this point, you're clearly dealing with something that operates in its own world and receives messages from the outside world specific to it. and Then I think it starts feeling a bit more natural to turn this into a component.
00:55:14
Speaker
Yeah, which is to that a very unsatisfying way of saying I'm not entirely sure what that boundary is. But I do know that Lustre pushes that boundary further away than React or Vue or Svelte. And I
Creating Web Components with Lustre
00:55:30
Speaker
think that's a good thing. Yeah, puts it on a high shelf that you can still reach, but you won't have it. It's like that trick where you put the biscuits, you put the cookies on the shelf you can't quite reach.
00:55:44
Speaker
Yeah, exactly. If you really, really want the cookie, yeah if you really want the cookie, you can get it. Yeah, but you won't do it unthinkingly. um So I have one more question about this in practice, which is, if I've not really had a great experience trying to write web components,
00:56:06
Speaker
And if I just thought, okay, I'm going to get into this just for the one feature of hopefully it's an easy way to make web components and publish them in a language of my choice for other people who don't care what language it's written in to use. Is Lustre on Gleam a good choice? It's certainly, certainly a choice. I would say that but this,
00:56:34
Speaker
This functionality or this this particular scenario is something that I ah did have in mind when I was working on this feature. So to rewind a little bit, Lustre has a companion DevTools package that runs on Erlang. The full process here being that like most of these developers like not needing node, not needing MPM, that's a selling feature. So there's a DevTools package for Lustre that includes a bundler and a dev server and some other stuff, but but relevant to this conversation is it includes a bundler and specifically it includes a command to bundle a web component. So you can actually take any Lustre component and say, hey, build this as a component, and then you get a JavaScript bundle that you can just include on any web page.
00:57:25
Speaker
And then I just write a regular old HTML tag and it's magic. Exactly. OK. So there there there are some caveats. There are some caveats to that and some some problems inherent to web components, I would say. So Lustre puts in a bit of work to make it so that you as an author don't need to know that a component is a web component.
00:57:55
Speaker
in so much as
Lustre DevTools and Learning Resources
00:57:56
Speaker
styling web components is difficult in in in the web component world. okay And what I mean by that is the phrase web component refers to more than one specification. So there's the custom element spec, which is having these custom HTML tags with their own logic, et cetera. It also refers to this idea of the shadow DOM.
00:58:25
Speaker
which is this way of hiding what the component renders to the parent so that you can't reach in and manipulate it using like a query selector, for example. And Shadow DOM is excellent. It's really, really good.
00:58:44
Speaker
But when you register a Shadow DOM, what you actually have also done is isolated the styles of the component from the parent. And this is both a blessing and a curse. This is excellent because it means that you can create a component with its own like styles, and then you could drop it in a different web page and guarantee that it's not affected by that page's styles. like It looks how you authored it.
00:59:09
Speaker
Yes. But the problem, of course, is if you want to use Shadow DOM and you want to use it in your own web application so you're not actually worried about conflicting styles, now, typically, as an author, if you are building these in JavaScript, for example, you have to pay attention to how they're styled. And there are ways to to to have styles pierce the Shadow DOM. So typically, you would use CSS variables, but there's also and like a shadow root CSS selector. but But the point I'm making here is that you have to be aware of this and then you have to essentially work around it. So Lustre does some work to work around this for you so that you never need to understand or even know. um And the way that manifests is essentially where we can, we adopt style sheets from the parent document, and where we can't, we clone them into the component.
01:00:07
Speaker
so For example, for example like if you have like an external link tag for some stylesheet hosted somewhere else, that gets cloned into the component so that the component has access to all of those styles as well, which works great if you're building a Lustre app and you want components. um This works less great if you're building a component without an app,
01:00:35
Speaker
and you want it to be styled in a self-contained way. um So I would put back to you and anyone else who's interested in this scenario is please try it and then please tell me what you need to make it work nice because I don't think anyone is doing this right now. like The functionality is there and the DevTools have had the forethought to include the way to bundle them, but I don't think anyone is actually doing it right now. Right. Need to use a testing. Yeah. Okay. Then that brings us to what must be the last and inevitable question. If someone wants to get started with Lustre, where do we start?
01:01:19
Speaker
The documentation, I put a lot of effort into this. like it's It's still a work in progress, but I put a lot of effort into the docs. So if you go to luster dot.build, that will just redirect you to the the package API on hex. And there, there's like a guide, like a long-form written guide that will kind of take you from zero to like a complete app that does side effects, HTTP requests, all of that. There's also a
Current Usage of Lustre
01:01:46
Speaker
quick start guide. There's also an ordered number of examples that each build on complexity. If you don't want to do a lot of reading and you just want to look at code, there's a bunch of examples as well. And there are cheat sheets if you're coming from React or Live View or Elm. Oh, wow. OK, you have done your homework. Yeah, I really want people to use this thing, you know.
01:02:09
Speaker
Cool. So one last question then, one last question. a What are you using it for? Are you using it in Angular in anything in production? Or is it just like, you're the library author sending it out into the world to be used? Right now, and i am I'm actually, I don't have the time to be used it myself. Like, yeah outside of work, I'm spending all of my time like developing the library. But I do know a handful of of people that have got this into production now. I know some people are putting their startups around it. and
01:02:43
Speaker
yeah like i'm not I'm not using it that much right now, which is why any more people I need more people, more experience reports, more users. Right. Well, then hopefully yeah this little conversation will inspire a few people to go and give it a try and then track you down with feedback.
01:03:01
Speaker
Yeah, I hope so. So at which point we should probably let you get back to writing the library and let them get started. Hailey, thank you very much for your time. Thank you very much for having me. Absolute pleasure. Cheers. Thank you, Hailey. If you want to give Lustre a try, there are links in the show notes, and if you're googling for it, it's L-U-S-T-R-E.
01:03:25
Speaker
If you want to give Gleam a try, Lustre is a very nice gateway drug into Gleam's world and thereby into the world of Erlang, so give that a try. And if you just want a nice place to hang out, the Gleam Discord has some great people, including Hailey and more.
01:03:42
Speaker
As this episode goes live, I will be at the go-to conference in Copenhagen. So just on the off chance, if you happen to be there, come and say hi. And if not, I have every intention of coming back with some new guests so you won't miss out. So stay tuned for that. Make sure you're subscribed and we'll be back soon. I've been your host, Chris Jenkins. This has been Developer Voices with Haley Thompson. Thanks for listening.