Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#35 Thomas Heller aka @thheller
58 Plays7 years ago
Heller woke podcast where Thomas cuts a sharp silhouette while discussing his work on the awesome shadow-cljs
Recommended
Transcript
Introduction and Guest Welcome
00:00:15
Speaker
Welcome to DEFIN, episode number 35. Today, we are going to speak with our very special guest from Germany, right? Yes.
Host and Guest Introductions
00:00:28
Speaker
So we have Thomas Heller from Germany. Hello. Welcome to the show. Thanks for having me. So quick introductions. This is Vijay from Holland and Brian from sunny Belgium.
00:00:42
Speaker
And Thomas? From Oldenburg, Germany. Okay, which city? Sorry, which city is that again? Oldenburg. Oldenburg. It's like Old Bridge. Old Castle, if you translate it. Old Castle? Oh, Burgh. Okay, yeah, yeah, yeah. It's next to Bremen. So it's northern Germany. Okay. I think I was only in bigger cities like Frankfurt. Bremen, Hamburg.
00:01:10
Speaker
Yeah, Hamburg. Yeah, yeah, cool. So this is episode number 35. And thanks a lot for joining us. So let's get started.
Thomas Heller's Programming Journey
00:01:18
Speaker
Can you give us some introduction about yourself? Like, how did you get into closure? And what are you doing these days? And we'll get into other projects soon. Sure. Well, I'm safe deployed, independent programmer have been for
00:01:36
Speaker
20 years about that, have been doing web development since
00:01:43
Speaker
Can't remember when 1998 or something. Internet Explorer four days. And yeah, I usually jump between languages every four years. And then I think six years ago, I looked at Closer and experimented with Closer and really liked it. So I stuck with it, I guess.
00:02:11
Speaker
But Closure, as in a Closure backend, or how did you get started? I mean, what kind of applications were you building in Closure? I looked at it when Closure skip was announced, the initial announcement. And I never used a lisp before, so it looked totally alien. I couldn't make sense of any of it. But the idea sounded good. And I have been a Java developer for many years.
00:02:41
Speaker
So I knew the JVM and I like the JVM and I was doing Ruby at the time and I hated Ruby because scaling was really annoying and I always wanted to go back to the JVM. So Closure was nice and the promise of Closure Script, the promise of having the same language on the server and as you have on the client seemed too good to be true.
00:03:09
Speaker
And so I tried it several times. I failed for a year, I think, until it finally clicked and made sense so I could use it. And I had several projects at the time. And the way I do experiment with the languages is I have a running project that was a Ruby project.
00:03:32
Speaker
And I wrote it in Clojure just to have a comparison. And at the time, I only did the server side, so no Clojure script yet. Just to compare how much code it is, how easy it was to maintain, how it worked. Was it faster? Was it slower?
00:03:52
Speaker
And yeah, that's how I got into Clojure. And ClojureScript was only the initial push, I guess. Just before you go on, Thomas, what was your conclusions when you were doing the server-side stuff with Clojure and Ruby? How did that shootout work out for you?
00:04:10
Speaker
All the headaches I had with Ruby went away. I was back on the JVM so I liked that and it was quite a bit faster and much less pain involved because at the time the Ruby project
00:04:26
Speaker
was running into scaling issues. And since Ruby doesn't do threading very well, you do the usual, or back in the day, you had unicorn and many, very many workers that could share memory. So you had to talk to Redis or memcache or a database. And so,
00:04:48
Speaker
Yeah, constantly going between all the workers and never having shared memory and all that stuff made it really annoying to work with. Was this with Rails or with Sinatra?
00:05:01
Speaker
That was a rate, a pure rate. I can't remember which version, but an old one. But yeah, it was just annoying and seemed unnecessary. Since I knew the JVM, I knew Java, I knew what you can do with one machine with the JVM, how much faster this used to be. And then Ruby was just slow. And I always liked optimizing things. So Ruby just got in the way.
00:05:31
Speaker
But it was a nice language compared to Java Ruby, of course.
00:05:36
Speaker
I don't hate Java. I hate the Java enterprise world. So all the patterns and factories and whatever, but you don't have to do that. You can write pretty simple Java code that still works and looks okay. And what I liked about
00:06:02
Speaker
Ruby was easier templating. Since the language was much more dynamic, you could do templates way easier. And I never liked the template solutions for Java. They all sucked. Yeah, I think we had like free marker and velocity, a couple of other things. Is that kind of templates that you're talking about? Yes. Like ERB and Ruby and yeah, yeah.
00:06:27
Speaker
Yeah, that's true. And then, I guess the first template, any language that I used was ERB, but then you got Hamel, which had indentation-based syntax. And that just felt nice since you just had to type less. So on the closure side, we are using Hiccup or was it something else back in the day? Hiccup, yeah.
00:06:55
Speaker
So you entered Closure World right when Closure Script was announced.
00:07:04
Speaker
No, a year after. So I looked at it when ClosureScript was announced, and I tried using it. I think it's several times. Because I never used this before. I couldn't get used to the parents. And editing was just a nightmare. The balancing the parentheses was annoying. And I was using
00:07:31
Speaker
Textmate at the time. Textmate, yes. I mean, maybe people are textmate, you know, it's like they're... Yeah, I used BIM before that. And it doesn't have good closure or list support. And I had to learn Emacs because of PILOT, because everyone was saying, learn PILOT, learn PILOT.
00:07:54
Speaker
And I failed so many times. It was just frustrating. And there's he pinging at me. And yeah, then after several attempts and several failures, I finally understood part of it. And then I learned enough of the key bindings to make it work. And then everything clicked. And then I just can't go back. I can't do it without anyone.
00:08:22
Speaker
So obviously, emacs is the thing. And then no, no, no. I never could get get you three. One for the one for the cursive team. Yeah. But he's a very smart guy. It's on the section. Yeah.
00:08:45
Speaker
Anyway, yeah, so just to be quick, so going back, so the nice thing about Ruby on Rails, obviously, was that Ruby made web applications quite easy, add all the kind of database integration, etc. Difficult to scale, as you said, but there's plenty to like about it.
00:09:06
Speaker
And when you came to the closure side, after all the parens or balances, et cetera, what were the kind of things that you liked about the closure side that kind of kept you there? What pushed me towards closure was ClosureScript. Because having the same language on the client as on the server was the ultimate goal. I have been doing JavaScript since 1998 or something.
00:09:34
Speaker
So very long time. And I always had two languages. I never did JavaScript on the server. So I always had per, PHP, Ruby, Java, Python, Erlang, very many different languages. But the thing that stuck with me was JavaScript.
00:09:56
Speaker
I always had to do JavaScript, always had a second language. So you always had to do these mental context switches between the languages and the translation of the data. So you always had to, in Java or in Ruby, converting to JSON,
00:10:19
Speaker
the Ruby objects, so JSON, and then inventing special syntax for dates and all that stuff was just annoying. And when I read about ClosureScript, or tried ClosureScript, I saw that it is possible to use the same language on the server as on the client and just transfer data around it without thinking about it.
00:10:45
Speaker
You get the same data you use on the server, you get it on the client, and you don't have all this conversion crap going on. And that was just the ultimate selling point for me. What is an example of a typical application that you are building?
00:11:07
Speaker
varies, but typical websites for a small to medium sized website, where so CMS systems, content management, with shop systems, all this, okay, anything web really.
00:11:27
Speaker
Okay, nice. So you learn different kinds of languages, but obviously, I mean, if you see your experience, you work from mostly non-functional languages, if I understand correctly. Before I get closer, I looked at Erlang pretty extensively.
00:11:45
Speaker
And I absolutely love it, but it's not usable for the stuff I'm doing. But yeah, function was a language first and then closer. Okay. And did you take it, take a look at Phoenix or Elixir Phoenix things as well? I did, but I like, I liked Erlang already. Why would I use them?
00:12:11
Speaker
Because that was the idea, right? I mean, at least there was the push for, because there is some meta-programming stuff in Elixir that is not there in Erlang probably. You could do it in Erlang as pass transforms and whatever, but Erlang has the...
00:12:27
Speaker
The issue is that it's built for something very specific. And if you do just HTML generation, it was never built for this. So it's not specifically fast for this. You can get it fast enough with Biola and whatever, but just reading stuff off the database, generating some HTML string is not Erlang's strength.
00:12:54
Speaker
It was never built for us. I think it's good for chat applications and things like that. If you need high parallelism with high concurrency, then Erlang is perfect. I would do all kinds of chats and servers that really need this level of concurrency. I would use Erlang, but I usually don't build those applications.
00:13:19
Speaker
So and then comes closure script. And so if you, what is your typical stack with closure script closure things?
Shadow CLJS: Creation and Motivation
00:13:28
Speaker
I mean, what kind of libraries that did you standardize on that kind of stuff or use, I don't know, luminous or something?
00:13:36
Speaker
given that I started pretty early after the announcement, there wasn't much around. So I wrote pretty much all my own stuff, which I was used to since I have had done it so many times before. And just seemed normal to me to do that. And I closure was around a little bit longer. So there was
00:14:01
Speaker
hiccup, so I used that to do the HTML templating for my client-side stuff. And it's not hard to write, so React wasn't around, so I did my own stuff, just the usual DOM manipulation stuff. And libraries, I think I didn't use any in the beginning.
00:14:28
Speaker
Did you use much kind of like javascript interop or did you.
00:14:34
Speaker
When I started the app, I first ported from Ruby to Closure, still had a pretty significant CoffeeScript client side. So at first I used the client side CoffeeScript, and then I started porting more and more parts of it. So initially I did a lot of JavaScript interop, and then eventually it just got to be all ClosureScript.
00:15:03
Speaker
Okay. And then on the server side, like a database related stuff, what kind of. Just on the Java stuff. So JDBC, Jetty has a server, HTTP server. Yeah. And was it mostly kind of like, for the JavaScript, was it mostly like form filling and animations and stuff like that? Or were you doing these single page applications? You know, the rich internet applications, whatever.
00:15:30
Speaker
Both yeah i had the the websites of a traditional website not seeing a page applications which did the usual jquery style dominant manipulations and then the the there was a pretty significant admin site for the cms and the shops and
00:15:53
Speaker
And those who are built in a single page application style of way. So pretty big ball of muds. So how did you come to Shadow CLJS? So what is the history and why did you build this thing? And maybe a quick intro about what Shadow CLJS is. Yeah, let's go through what it is.
00:16:17
Speaker
Yeah, yeah. So, Shadow CLJS is a build tool for Cloroscript. So, it's just about compiling and bundling the code, more or less. And it started out as Shadow build. And just because the more code I had ported, the more my project grew and the
00:16:45
Speaker
slower it got to compile and
00:16:50
Speaker
I came to the point that I wanted to do stuff that the closed script compiler couldn't do. I knew that the Google Closure Compiler could do it, but closed script at the time had no support for modules, the code splitting. And I wanted to use that. So I had no other option than looking into how it worked. And it's not that hard to make work.
00:17:20
Speaker
And so, yeah, I started writing specifically for modules since I didn't since close scripts itself had no support for it. And that's how it started, basically. So it started in 2013. So it's basically a kind of closure script webpack initially.
00:17:44
Speaker
Initially, it was just about modules. It was just taking... So Closerscript has two parts to its compilation. First, you need to compile the Closerscript to JavaScript, and then Closerscript is basically done.
00:18:03
Speaker
And then the Google Closure Compiler is used for bundling stuff together. And modules basically has nothing to do with ClosureScript. It is not involved in that part. It's just about taking the JavaScript that the Closure Compiler produced
00:18:23
Speaker
And then feeding it in the correct order to the Closure Compiler. So it was just the bridge between the two compilers that I was doing. And so couldn't really compare to Webpack more.
00:18:44
Speaker
You could you could have done it with the with the shell script at the time because the closure compiler has a executable that you can use and you could can do it with that alone so what was the advantage of modules then what was the thing that motivated you to do. Why why since no one cares about modules in javascript why did you care about it.
00:19:06
Speaker
My app was getting too big. Because one thing I didn't realize at the time, but I had all my closed script code in one folder. So the admin stuff was on source, CIJS, the client side stuff was in there, the shared stuff, everything was in there. And the closed script compiler, to this day,
00:19:34
Speaker
compiles everything in that folder. Then advanced optimization is supposed to kick the stuff out you are not using, but it doesn't kick out everything. The client side app, the thing it's got shipped to the user,
00:19:55
Speaker
included most of the admin stuff although they never used it. And at the time I thought modules would be nice for this because I just make a user module and an admin module and the admin module never gets loaded by the user so the JavaScript says more.
00:20:13
Speaker
And of course, nowadays, I just have two different bits. But at the time, I thought modules would be the solution. And I still use them for other stuff. But in that particular case, it was not a good idea. But it sounded like a good idea at the time. So what would be the steps that, if you want to integrate this into the ClojureScript compiler chain itself?
00:20:41
Speaker
Do you think that is going to be valuable to include shadow CLGS functionality into closer script compiler steps? I think pretty much everything is supported nowadays. David Nolan added support for modules. Yeah, yeah. Some years or two later. Yeah. And I think pretty much
00:21:02
Speaker
well, shadow cjs does a lot of stuff very differently than the standard close script compiler. But for the most part, it's just about this bridging between close script and the Google closure. It's not about it doesn't do too many modifications to closer to close scripts, closer script itself.
00:21:24
Speaker
Yeah. So what would be an example of, like you said, some things that are done in Clojuskipt are a bit done differently. So what would be the example of it? What will shadow CLJS do differently?
00:21:38
Speaker
Lots of stuff. I guess the biggest difference is the way the namespace forms. So NS and every closed script file is handled because I do two pass compiles. This means that I first read the file, only the namespace,
00:21:59
Speaker
and then resolve according to what the namespaces are, sort all the files and then compile with parallel compile as opposed to the standard closed-script compile which does one form at a time basically.
00:22:20
Speaker
So, Closure Script is supposed to work like Closure. And Closure works by reading one form at a time, compiling it and executing it. And in Closure Script, this is not super useful because you want things to be more static because of the Closure Compiler.
00:22:38
Speaker
And yeah, that's the big difference is how the compiler actually does stuff. And the result is similar. But since I control how dependencies are resolved, I could do the
00:22:57
Speaker
JavaScript integration completely without the modifying anything in closed script itself. So the support for using NPM directly was done without the compiler knowing.
00:23:13
Speaker
So given that, so you said you're one of the primary motivations to pick up closure, closure script is that using the same language across server and under client. Apart from the data, so how do you take advantage of it? I mean, do you write everything as CLJC and then which kind of features do you share with the common code?
00:23:38
Speaker
Where it makes sense, I do use CLJC or CLJX before that. And not all code needs to be shared. The code serving the HTTP requests, so when you're talking to the database, that doesn't need to be shared. So it's just a closure for those. And the client side with all the React or DOM stuff is just CLJX.
00:24:06
Speaker
Since I can never get used to CLJC because of all the switching you have to do, whether you are in the JVM and can use threads and exceptions and all this stuff, or JavaScript, which has different semantics and everything is async and
00:24:28
Speaker
One of the things, so maybe staying on the Shadow CLJS, take your point about the motivations. What do you think, I mean, what would you say makes life easier for developers using Shadow?
00:24:43
Speaker
Or for like standard vanilla c.l.j.s. So the big thing, the big differentiator at this point is the NPM integration. I just opted to use NPM directly. So if you want to use React or
00:25:03
Speaker
Anything else you find on NPM, you just NPM install the package and use it directly. You don't have to go through CLJS.js, which is basically just a rebundled NPM package. You don't need to use Webpack or anything, you just use the files directly from NPM.
00:25:27
Speaker
So you get this kind of like in the require, you can just say, you just do npm install and you can say require, you know, whatever, react or. Exactly. Or some kind of request library. Yeah. On the node side. Yeah. Okay. Um, the other thing is that, that, that I kind of, I know that people are impressed with on the project is the documentation stuff that you've done. Yeah. Um, because that meant to me.
00:25:55
Speaker
I've got to be honest that like you said at the very beginning of this conversation, what is the barrier to using ClosureScript? It has been, I think it's getting better on the ClosureScript side, but the entry-level documentation and getting started part of ClosureScript was absolute torture at the beginning. It's definitely getting better. I mean, I think we can all agree on that. But your documentation, and I think it's yourself and Tony K who are doing this, if I'm not wrong, but you've really done a great job, so kudos to you on that.
00:26:25
Speaker
Yeah, thanks. So the big push. So the project is old. I have been using this for four, four and a half years now. And I never wrote any documentation. I basically just wrote it for myself since I had had these issues I wanted to solve. And then last year in August, I think,
00:26:50
Speaker
I started noticing that other people were interested and I could do stuff that's currently hard to do in closed script. And I just need to write documentation. So I started writing some wiki entries, some examples, and it was just all over the place. I never quite organized it.
00:27:16
Speaker
And then Tony K, at one point, became interested in ShadowC edges and wanted to try it for himself. And he's for full crew. He wrote the documentation, which is fabulous.
00:27:35
Speaker
He just took everything I wrote from all the different places and put it in one document. So, and structured it like he thought would be all right. And I just kept working on that after that, but he definitely did the initial push. And
00:27:57
Speaker
Organize the stuff since i just wrote something in wiki and something on my blog and something in an example was horrible and the documentation now it's still struggle it's still a lot left to be documented but it's much better now so now it's actually getting usable. It's very usable yeah no it's one of the things i'm really personally very impressed by yeah.
00:28:22
Speaker
So if you compare, because you're not building a lot of Closure Script and Closure Replications, if you compare the similar kind of experience with Ruby on Rails, so what would you miss right now? Or do you miss something from Ruby on Rails? Of course, you can say the mongrel and whatever. I used to do that shit whenever it was mongrel, so that was a long time ago. So you don't miss anything from Ruby at all. Absolutely nothing. Nothing at all.
00:28:49
Speaker
Because there is always this friction online whenever you speak to somebody, there is no framework in closure script. I don't care. That's what I like about it. I have been doing this since forever, especially JavaScript.
00:29:07
Speaker
When I started with JavaScript, there was no jQuery. There was no Webpack. There were no tools. So I always had to know the basics. I had to know how everything fit together, how they worked. And so I always try to pick my own stuff. And in Ruby on Rails, it was nice to not have to pick your own stuff. So they gave you everything.
00:29:35
Speaker
until it didn't work anymore. Until you wanted to do something that they didn't support out of the box, like talking to two different databases. They added it eventually, but it was pretty damn annoying to do in the beginning. And when I came to Java, it was the same thing. You could use Spring, or you could do your own stuff and write
00:30:03
Speaker
I have the code or so. And yeah, when I came to Clojure, I just looked at the libraries and I knew, okay, I'm going to use Hiccup. I'm going to use Jetty, since I knew that there was a ring which looked like, what is it? Rec in Ruby. And so that made sense.
00:30:28
Speaker
And yeah, I could just use, Closure was great because I could use Java libraries that I knew from before. And it worked well enough, or there were wrappers for them. And yeah, I could just pick what I wanted, because I have been doing this for so long, I know which parts I need.
00:30:52
Speaker
I'm just conscious of the fact that, you know, we really want, I personally would like to talk quite a lot about shadow CLJS. So we got back to that. I think that's, I mean, it's all like the bullshit about Ruby and blah, blah, blah. Okay. It's great. We loved it back in the day. Yeah. But we throw it in a bin now.
00:31:14
Speaker
Yeah, so just going back to ShadowCLGS, because I think probably the people want to hear a bit more about that. So dependencies, one of the things that's moving on a little bit with Closure Script and Closure in general is that there's kind of a new style of doing dependencies, because you're seeing NPM and stuff like that, which is all great. But obviously, now with CLGS and CLJ, we've got this depth.eden.
00:31:43
Speaker
Now, that's just a way of expressing the dependencies. Is that something which you support in Shadow CLGS? How is that kind of working there? First of all, NPM is horrible. I don't want to use it, but the JavaScript world is using it, so we kind of have to. But depth EDN or tool steps is supported. You can just use it.
00:32:13
Speaker
The only part that ShadowCLJS needs from that is constructing a class pass. You just need to know which just to load, and you can do this with lightning in, with two steps, or with ShadowCLJS itself, since all it needs is the class pass. It is basically just a closure library. You can use it with lightning in, you can use it boot, but it's a little bit more complicated.
00:32:40
Speaker
So the only thing it needs really is the class pass creation. And you can use it completely without any tools and just to Java, class pass, whatever. Okay. So in terms of like,
00:32:55
Speaker
the features that are being added to Closure Script over the recent months. You know, all of those features are kind of available with Shadow CLGS. You know, I just want to make sure that all the people in the audience kind of know where you are in that respect. You know, how closely are you tracking the features that are available in the core project?
00:33:15
Speaker
I personally don't think that the new stuff, the cages.main, I think you're talking about, is very useful for shadow cages because shadow cages wants to control or wants to help you write less. So the configuration part
00:33:34
Speaker
takes care of most of the stuff you need. So you don't have to invoke it via the command line with some weird, huge config. So the big idea behind Shadow CLJS is making the configuration easier. So that the browser configs look...
00:33:59
Speaker
looks different from node configs. So if you're writing a node script, which target node script, that's just a script that's going to be executed by node, it has totally different requirements than a browser build. And so the target abstraction in the context is about capturing the
00:34:24
Speaker
least amount required for each environment, so that you don't have to write too many config instructions. And the big thing that Shadow CIJS does is it unifies the development config with the release config.
00:34:47
Speaker
So when you are writing, when you're using lean CIJS build or CIJS.main, you always have to write two configs. One you are using with development, with FigVeal, and then one for advanced compilation. And what I learned is that about 80% of the configuration is identical. So why do I have to repeat it twice? So...
00:35:16
Speaker
That's the big idea behind the config. So all of the tooling, like you said, things like things that are built around Lightning and like Fig Wheel, et cetera, they all work nicely with Shadow CLGS? No. It does those things itself. Right. So Fig Wheel is just for reloading code. Reloading code is not so hard to do. So I've just wrote the code to do it directly built into Shadow CLGS so you don't need anything else. Okay.
00:35:45
Speaker
and one big idea is so that you can do Closerscript without having to learn Liningen, without having to learn anything else but Shadow CLJS. Nowadays, it does pretty much everything I think that Figurator does, the wrapper, the library load, and it also does everything that Lean CLJS build does.
00:36:13
Speaker
If you are doing, uh, figure you need to adjust, but as well, so you need to learn to do two different things and they happen to share the same config syntax, but you still need to learn two things. Uh, and shadow series is just one solution for everything for the entire pipeline. Okay.
00:36:34
Speaker
So you can,
Tool Interoperability and Recommendations
00:36:36
Speaker
so technically you can just build your entire application in CLDS, like backend and frontend as well. Node.js things. Okay. But would you recommend that? I mean, building the whole backend in Node? No. That's disclosure.
00:36:52
Speaker
Yeah, that sounds like a reasonable start. But I think, I mean, I'm using Node.js with CL.js for a command line interface. And that really is very nice because with shadow CL.js, you know, I can just use these like, you have these watching to file watching capabilities on you so that
00:37:14
Speaker
if you can be writing your code and you can be running tests and everything in the background as files change. And I found that very, very compelling to be honest, as an experience. Yeah, there is. So, if you use Lightning, or rather the ClosureScript compiler itself, or the part that packages the code together, the compiler is pretty much identical.
00:37:42
Speaker
the code, outputting the code in the correct order and the correct organization of files is very different depending on the environment you're running in. So if you're running in the browser, you have very different requirements than when running a node. So what Shell is here just does is the targets express
00:38:05
Speaker
how the code should be packaged together. For running the tests, I can package the code differently and use my knowledge about the compiler and the files that go into it to feed the test framework.
00:38:22
Speaker
Since otherwise with this do, I think, in lining and to our enclosed group test, you always have to create this runner namespace where you list all the namespaces of your tests. I just compiled them. I know what they are. So I can just inject that code. I basically just generated and then you don't have to write that code. And very many tools in closed script try to
00:38:53
Speaker
do things that the build tool should do, and that ShadowCLJS does. So you have to write less, and it's just taken care of by the config. Okay, so you mentioned that there are different stories for the browser, for Node, for CLI, and you have different targets for that. So is that the fundamental kind of configuration process with ShadowCLJS?
00:39:21
Speaker
Yeah, the basic idea is that the config should express the, or should require the least amount of information to make your project work. So for the target node script, you just specify main, which is just a fully quite simple to your main function. And you don't need anything else. The compiler knows everything from there, or you need output too, so to know which file to write.
00:39:48
Speaker
After that, the compiler can do the rest and it can inject or call the function for you directly. Currently, in ClosureScript, if you're using standard ClosureScript, you have to import, I think, cijs.nodejs and you have to set main clifn and invoke that somehow. And the compiler just takes care of that in the config. You don't have to do it in the code.
00:40:17
Speaker
And as just turns out to be much less code to write.
00:40:23
Speaker
So do you think, I'm just a kind of like, the other thing is like, what, apart from this target thing, what do you consider to be the kind of like, let's say the lock-ins or the kind of like, what, if you're moving project, let's say we're in a project where the people at that company or that project have chosen shadow CLGS, and then you move to another project or another company and then they're using another tools, what kind of things we've got, or do the way around, you know, what kind of things we've got to get used to going from one environment to the other?
00:40:52
Speaker
config obviously, but from close script it's safe, basically nothing. It's identical, it's just a standard close script compiler, it's using the latest version, it does some modifications to make the external inference a bit better.
00:41:09
Speaker
But I hope to write a patch for that so it ends up in core eventually. And it's basically the same stuff. The big difference currently is NPM integration. Since if you start using many NPM packages, this stuff still doesn't work very well in standard ClojureScript. So you will get into trouble there. But the syntax is identical. So the syntax doesn't need to change at all.
00:41:39
Speaker
And yeah, you can basically just use whatever. The language itself stays identical. That doesn't change at all. So that's the big point is that it's just like the configuration tools, which are kind of essentially 10 lines of code on your side or 20 lines of code on the standard side. So it's just a matter of getting used to that or getting
00:42:09
Speaker
The other thing is the two things, like you say, about the fig wheel versus the shadow tools. And your repl, I guess, is a bit different from the other repls.
00:42:23
Speaker
It still behaves the same, but most tools are used to lining in and expect you to use lining in, which I expect is going to change due to its steps. But currently, most tools are built for lining in, so you have to jump through a few hoops to make everything work, but after that, it's the same.
00:42:47
Speaker
Well, the big differences, I guess, that ShadowCL just supports running multiple builds in parallel, and you can switch between them in the wrapper without stopping the build. And I think Fig will do with it. I'm not sure. But beyond that, it's just a normal N wrapper server. Right. But when would you not recommend using ShadowCL? Yes.
00:43:15
Speaker
This is the only thing that you need. That's the whole idea. Yeah, of course. So you say you're making some patches to certain things to go back into the core. I mean, what's your kind of, I mean, I guess that some of the core people, you know, look at the tools that you're doing and, you know, the sort of changes that you're making and
00:43:37
Speaker
Would like it to contribute is that i mean what's your history with the court team i mean are you have you tried to contribute things before or what's the story.
00:43:46
Speaker
I have tried very many times. Actually, some of the features you see today are from me, but the big thing for me is I wrote the tool for me in the beginning. I had problems I needed to solve, and getting things into core takes time. Writing the patches takes time, and not everything I thought about
00:44:16
Speaker
It turned out to be a good idea. So it's good that those things did not end up in core. But it's basically just a time question. Do I have the time to create the patch and get the thing in? Most of the time, it's simply not required because it's just about stitching things together with a Closure Compiler, a Google Closure Compiler. But yeah, the extents inference work I did.
00:44:46
Speaker
I think it's a useful thing to do and it makes it easier to use. Since currently you have to type hint with an object type or with a class. So you need to know that the class of JavaScript objects, which is when you are trying to type and react or any other NPN library is pretty tedious.
00:45:13
Speaker
and all you need is a gs type and you don't need to know the class and so so the excellence inference is much easier easier to use i think and much more reliable as well and that's i yeah it's a pretty big chain pretty big chains
00:45:32
Speaker
in the core algorithm for this. So writing a patch and the proposal why I'm doing that is taking time. That's why it takes time. But several features. So the first big thing I wanted or I implemented in shadow build
00:45:51
Speaker
or shadows here it is, was the way you use macros today. Previously, you always had to do require macros or refer macros. This was pretty annoying. I found a way or implemented a way so that you can just use refer. You don't need to know if you are using a macro or a function.
00:46:15
Speaker
and you just use refer and the compiler can figure out the rest. This trick was the CLJS namespace requiring itself via require macros so that the compiler can extract which macros the namespace provides so that it can do the right thing at the right time.
00:46:40
Speaker
But that is one patch that I never wrote. I wrote the proposal, I documented how I did it, but I did it in Shadow build and Shadow build was very different at the time. So writing a patch from my implementation was pretty much impossible.
00:46:59
Speaker
And I just wrote down how I did it. And I think Mike Fikes finally implemented it, or David Nolan, both. But it took a year or so longer than I already had it. And yeah, several other things.
00:47:16
Speaker
One of the things that you talk about is the warning tracking. What exactly is that one? It's a better warning tracker. So it's mostly about cache invalidation, I guess. In lean CRJS build or in FigQL, they don't
00:47:32
Speaker
they don't cache very much, but when they do, they don't show warnings again. If you compile a file that has warnings and use cache, if you start the compile again, you won't see the warning since you used the cache. And shall we say it just doesn't cache files which had warnings. So it just prohibits the cache, so you always see the warning, and I remind it to fix it.
00:47:57
Speaker
So if you're looking at the closure script development, I mean, what are the pain points that you see? Because you extended the compiler, of course, there are all these things that you wanted to have integration with NPM modules, etc. But in normal development, I mean, what kind of pain points that you still think are there?
00:48:18
Speaker
I guess a fan favorite of error messages, which are bad, could be better. I tried to do my best to fix most of them, but it's really hard writing good error messages.
00:48:36
Speaker
not everyone requires the same error message. A total beginner needs more guidance than someone more experienced. And someone more experienced might even be more distracted by all the extra information. And it's really hard finding the balance, but it's also a whole lot of work actually doing them since you basically have to do the work for every single error. And
00:49:05
Speaker
sometimes beginners make mistakes that I never made. So, so I want one thing is, when I use the wrapper, I always use cursive. And it's pretty much since I'm using part it, it's pretty much impossible to have imbalanced parents. Yeah. And then someone started using shadow stages and complained that the wrapper crashed every time we tried to use it. But it was just about unbalanced parentheses. So so he just
00:49:32
Speaker
I never noticed this because it just doesn't allow me to send that. And so I didn't anticipate that error. So the error was pretty horrible. And that's the deal with error messages. You have to account for pretty much everything the user is trying to do. And for every level. Yeah. And that's hard. It's a lot of work and not so easy to fix.
00:49:59
Speaker
Yeah, I mean, this is this kind of, as you said, like one of the hot button topics, like, you know, talking about error messages. And I think, you know, as you have plenty of experience in Java as well, we are used to stack traces. And the thing is they look ugly, but they're very...
00:50:16
Speaker
I think usable, okay, that's a strange word to use, but at least you know how to dig through this thing. They look very ugly, of course. They're not beginner friendly or anything, but they give enough information for the advanced practitioner or whatever, user, to dig through the whole chain and then see where exactly the error is. I think sometimes you become very blind to, oh, this is ugly, or that doesn't even come to you.
00:50:45
Speaker
Yeah, but but if you're a beginner, and you see this 200 line stack trace dump, they are intimidating. You have no idea what any of it means. Yeah. And so I try my hardest to never show them. Since most of the time, they are not useful in the error itself. The message is enough. Yeah, or showing just showing the location of the error. You're just underlying what's wrong is most
00:51:10
Speaker
most of the time it's good enough. But sometimes the stack race is required and useful, but it's hard. Yeah, but it is the same. I know it can be applied to every sort of programming issue, right? Because it's not just one dimension that we're looking at, whether they're looking good or whether they're informative or not. It's also, as you pointed out, the user experience, how much experience the user has.
00:51:36
Speaker
Like, if I'm a beginner, then I wouldn't want the whole thing. I would like, OK, go and fix this sort of message. That's easy for me. But as an advanced user, I'll be like, OK, I want to dig into every possible thread that is running right now. And I want to see which one is, you know, that's an interesting idea. So what are your favorite closure features coming from, you know, from ClosureScript or Closure?
00:52:04
Speaker
Uh, the syntax itself. So initially is that syntax was the most tricky point for you to get it. After it finally clicked and made sense. It's just, I can't write code without it anymore. When I write Java.
00:52:21
Speaker
I always place the parents on the wrong side. And the indentation if you write an if or anything deeply nested is horrible in Java or any other language. And so I guess the main selling point for me is just the syntax.
00:52:39
Speaker
And of course the data structures and all of that. But the syntax itself, I can write HTML via Hiccup, I can write CSS, I can write pretty much everything fits this syntax. That's just a killer feature for me.
00:53:05
Speaker
Yeah, I think I've been playing a bit with Python recently for the data science stuff. I mean, every time I type Python code, I feel like I need to wash my hands. I mean, there's so much of the mutability everywhere and then just assigning variables and changing them left and right.
00:53:20
Speaker
Like, God damn it, what the fuck am I doing? It's so, it feels dirty. If you look at what the JavaScript world is doing with React and JSX, it just looks horrible. Yeah. Whenever I look at the NPM library and try to figure out what's going on, I see this garbage syntax, and it just doesn't make sense. And it's so much less code to write if you just use Closure or Sx versions. That's true.
00:53:49
Speaker
Also, it grows consistently as well, whereas the way that JavaScript grows is wildly different from version to version, I think. There's suddenly all these arrow functions and all these dot-dot-dots and all these other wacky bits and pieces that are coming into the language.
00:54:07
Speaker
I think someone did a spoof, didn't they, of saying, oh, look at this functional programming language stuff. And all the JavaScript developers were like, oh, god. Oh, my god. No wonder we love JavaScript. Look at this shit. It's terrible. No, no, no. That is JavaScript.
00:54:30
Speaker
But that's one of the advantages of the syntax itself is that you can write micros to do all the stuff that JavaScript needs to do and needs to introduce new syntax for. You can just do it in a macro. So what are your plans for Shouda CLGS next?
00:54:55
Speaker
Basically everything you can think of. I want to definitely build a React Native target support. Since that's a bit too... It seems harder than it should be currently to use React Native. Then I really want to build a UI so that you don't have to do everything at the command line. Command line is great if you want automation.
00:55:24
Speaker
to script your release tasks or whatever. But for development, it's horrible. If you want to start two builds, run the wrapper into one thing and then run your tests and all the commands you have to memorize for this. This is just annoying.
00:55:45
Speaker
If you just can have UI, click a button and it runs for you. First of all, you get way more information because displaying it in your terminal in a text is just limited. And when you have UI or HTML UI, so single page app, you just get way more information faster and way more visually
00:56:10
Speaker
But is it something you want to integrate into your HUD, the HUD that you have in Shadow CLDS, like Het's a display, or is it something you're going to be a separate tool? I think it's going to be standalone. I thought about, though, it's pretty simple to inject a wrapper into your page so that you have a prompt at the bottom and just type and you got it directly without having to go to the terminal. That's pretty easy to do. The problem is,
00:56:36
Speaker
you injected into your page so your code suddenly has to run whatever technology you used to do to do that so even if you don't really use react the injector thing would you strike react so i really don't want to do that and it's also not very
00:56:53
Speaker
user-friendly if you suddenly compile thousands of files extra that your build is not using at all. And so the one idea I have is for doing Chrome extension. So you have it side by side.
00:57:09
Speaker
That's nice. Or just have a standard web interface where you have two tabs open next to each other and just can see the compiler, compile errors, basic information, documentation, lookups,
00:57:26
Speaker
or all this kind of stuff. You can't really display in the terminal, but you can display in a browser. And I think there would be many useful things that you can display. And the one thing I really want, what is a lot of work,
00:57:43
Speaker
that would be to have a configuration editor so that you don't have to memorize all the stuff, what the keys were and in which place. So you just click, I want to build for the browser. I want to build this output file in this directory and this is my namespace. Since it can read the file system,
00:58:04
Speaker
It just knows the namespaces and just can auto complete it for you and that's just not stuff you can do in the standard. That would be like an app served by shadow CLGS back end.
00:58:19
Speaker
then you would do an API call to shadow.clgs to update the config file, for example. Currently, if you run shadow.clgs server, it already starts a web interface, which doesn't do much currently. But the idea is that you have a web UI you can open and click around and instead of doing everything at the terminal,
00:58:46
Speaker
There are useful things in the terminal for the released automation for continuous integration. You definitely want the command line. But during development, we can do way better than that. Like you said, you need all of the bits and pieces so that you can persist it and have the CLI run it. But you shouldn't have to interact with all that kind of stuff through some crappy editor.
00:59:16
Speaker
that people like.
00:59:19
Speaker
You can totally do it. And if you know what you're doing, that's probably faster than clicking through the UI. But for a beginner that doesn't know which key goes where, which all this stuff, the web interface could just show all the information. Just show which namespaces are there, even let you inspect the namespaces, browse the documentation, all this stuff. So you can display just way more information that way.
00:59:49
Speaker
And not everything has to be done at the rapid. So a final question about Shadow CLJS. Maybe this should have been the first question. What is the name? Why did you pick Shadow CLJS? Naming is hard.
01:00:09
Speaker
When I came to Closure, the very first namespace I needed on the very first namespace I wrote was DOM stuff. Since I used to use jQuery and did a lot of DOM pre-react days. At the time, the new standard that was announced was Shadow DOM.
01:00:36
Speaker
So very creatively, I named the namespace shadow.dom. And that's where the name stuck. So the library was called shadow. And it had a namespace shadow.dom. And the shadow just stuck around. So initially it was called shadow build because it was meant to build my shadow build.
01:01:01
Speaker
I was trying really hard trying to understand what exactly shadow is because I remember in my native tongue there is like a second grade detective novels and the guy was named by shadow. It's like a totally James Bond like character or something.
01:01:19
Speaker
It's really just about naming is hard. Seeing my own name and every namespace was annoying me. So the library I wrote, since I wanted to use it in a different project, needed a common name. And the thing was, the big thing that everyone thought was to be the future was web components with Shadow DOM. And yeah, that never happened, but the name stuck around.
01:01:49
Speaker
But now you have an opportunity to come up with a very cool backstory and just rewrite the history and then come up with an amazing story. I thought it was because you were shadowing CLGS. That's what I thought.
01:02:03
Speaker
You can see that, right? In a way it is. That's the backstory that you want to give out. We'll help you with your marketing. It follows the CLJS like a shadow. Sweet. OK, so I think before we go offline, maybe a quick round up on what is happening around, because there is a lot of noise about GraalVM and people building native images and all that stuff. So what is your opinion about that one?
GraalVM and Conference Reflections
01:02:34
Speaker
I like it, but for closed script itself, it's basically not very useful yet. The big thing that's interesting is compiling the JavaScript to run in Graal instead of Node, which gives you access to all of the JVM. So if you want to use threads in JavaScript, you can do that with Graal.
01:03:02
Speaker
That's nice, but I'm not, when I'm at that point, I might as well write closure. So what's the point of using closed script then? The idea that's interesting for, or would be interesting for shadows the edges itself is I do a lot of interop with a node. So if we are a barbell or any other,
01:03:27
Speaker
node library. And currently I have to start a node process for that and talk to it via TCP IP. And that's pretty annoying. So it would be nice if I could just launch a node context in the JVM and have access to all the code still without having to leave the JVM. That would be nice. But beyond that, it's not super exciting.
01:03:54
Speaker
It's interesting from a language design standpoint, but it doesn't provide anything I don't, I already need, currently. I'm actually curious about the polyglotness of it because there is no, you know, you can use the same, not the same, but you can use multiple languages in them.
01:04:13
Speaker
share the code pretty quickly or in a nice way or something. But this is the same shit that .NET was talking about. You can write everything in one language and then you can write something else in a different language. But one of the things that I think for the closure side would be the native image thing, I think, to make the startup time way less when you're running in a production mode. I think that is one thing that I could see as a potentially interesting side, I think.
01:04:42
Speaker
But usually that's not a big problem because you just started up and you're done and once it warms up and you don't need to care about it, I think. Right. So the people always complain about startup time. It's always too slow. And if I start something once a month or so, I don't care about startup time. Even if I only started once per day or so, I don't care. It takes 10 seconds or one.
01:05:07
Speaker
This is not relevant. If you write command line applications, like the Node World does a lot, you want those to start fast. But I already can use ClosureScript for that. And a lot of the dynamic stuff that Closure does won't work with a native image. So if you have any dynamic require or
01:05:35
Speaker
Any of the dynamic stuff basically, anything that makes Closure great, it won't work with a native image. So if you want to heavily optimize your code that basically has the same limitations that ClosureScript has due to the Google Closure Compiler, you can use native image.
01:05:55
Speaker
Why would I give up that power? It seems to me as well. On a personal level, I would much rather go from a much more live coding environment than a fast startup time. Of the two things, I don't give a shit about startup time, but I do care about the live programming environment. That's something which closure and functional programming should be all about.
01:06:19
Speaker
Because it's a killer feature of what we do, really. Start of time is meh. And like you said, if we're using ClosureScript for CLIs, they go super fast anywhere. 100 milliseconds is slow for those people. And it's fast for the JVM. So who cares? I'm sorry. I'm really struggling to get excited about this.
01:06:43
Speaker
I mean, I see that people made these native images and they go fast, but I'm like, well, yeah, but we could do that three years ago with Node.
01:06:54
Speaker
The native images are linked to your libraries in your system. So you can't just copy it to another system and expect it to run. So it has all kinds of limitations. It is super interesting from a language standpoint, the implementation they did. It's super interesting.
01:07:16
Speaker
For me, I'm only writing Clojure and some Java and ClojureScript, so I don't want to write JavaScript. Why do I need support for using it directly? I already can use JavaScript directly via NPM or whatever. The interop between ClojureScript and JavaScript is great. Same for Java and Clojure.
01:07:42
Speaker
And sure, if you have something machine learning or some other toolkits where Python is very popular, then that might be useful for you. But for the stuff I'm writing, it's not interesting.
01:08:03
Speaker
I think there's, like you said, Vijay, it's a bit of a CLR play by Oracle. To me, there's a bunch of things interesting about it from an academic perspective in terms of what does it do for hotspots and the infrastructure of the JVM. But that's kind of like, it's super academic at this point for us. But also, this whole notion though that you get the performance you pay for,
01:08:32
Speaker
sucks ass. I think the community edition and the enterprise edition for a VM are like, come on, this is just bullshit. It sucks. If you want it 24 times faster, just pay us more money and we'll make it faster. Sure. Sounds like an interesting idea. This should burn in hell for that. Anyway, on that bombshell,
01:08:57
Speaker
So we're almost at the end of the episode. Thanks a lot, Thomas, for joining us and taking us through the shadows. The dark side of closure script. The summarize old man.
01:09:12
Speaker
Exactly. And as you know, you know, we've been doing this as a live as well. And a couple of comments from just sec. Hey, I apologize for not pronouncing your name correctly. And then he says, great error messages in Shadow CLJS. So, you know, that's a big, I think a nice recognition of your struggle to find the right error messages for people. And so to wrap up, maybe a quick review of what happened. So we had
01:09:41
Speaker
Dutch closure day, I think a couple of weeks ago. So it was, I think, if I'm a bit biased, well, I'm a bit biased. It's been a really nice conference. We had a lot of fun. All the videos are up. We tweeted, I think, at some point. We hopefully want to run this thing again next year as well. Couldn't just interrupt there for a second. Sure. You're a very modest person and you do a lot of work for this conference and it's all like, you know,
01:10:10
Speaker
Voluntarily, time is all completely voluntary. And everyone at that conference had a superb time. The food was great, the sessions were great, speakers were great, the organization was awesome. And I think, you know, we as a community, I think you and your colleagues did an absolutely fantastic job for us. So round of applause.
01:10:29
Speaker
We get wowed at the sticks of applause at this point. It's obviously not just my work. But you did a lot of the announcing and I know that you did a lot of the backroom work. So kudos to you. I was the show money. So that was really fun. And of course, we're on Patreon, so if you think we're doing some good stuff, send us money so we can retire soon.
01:10:58
Speaker
And I think that's it from us.
Episode Conclusion and Farewell
01:11:01
Speaker
And again, once again, thanks a lot, Thomas, for taking your time on Sunday and then explaining a lot of things about the project. And I hope, I know, you'll continue to do all the great work and release your fancy UI for ShadowSeal. Yes, I'm looking forward to it. Yeah, thanks for having me. It was brilliant. Thanks, Thomas. It was a lot of fun. Bye bye. Bye.
01:11:49
Speaker
you