Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
# 36 The Tao of Peter aka @ptaoussanis
15 Plays6 years ago
# 36 The Tao of Peter aka @ptaoussanis by defn
Recommended
Transcript
Introduction and Fan Acknowledgment
00:00:15
Speaker
Welcome to Deaf on episode number 36. Are you going on live yet? Are you going on air yet? Yeah, we are. Screw it. You know what you're doing. Look, there is like, at least, I told you, right? There's too dedicated, too many fans, exactly two. Yeah. So they are online now. Do a hardcore fan. Yeah, let's get the hard call. They're knocking on the pub door. Let's open up, you know? They're eager.
Technical Hiccups and Humor
00:00:40
Speaker
Okay, click on broadcast. So it's going to start. Yeah, I think it should be on now. Yeah. So we live. Yes. We're live. Okay, so this is like dramatic. Oh, hold on. Now we'll see. Sorry, I just had to mute the YouTube for a second. No, no way. This is an anticlimax. We were already
00:01:10
Speaker
The girl with the line. Well, this is what we were talking about, wasn't it? You know, it's like state again, you know, it's a total nightmare. The feedback loops. I mean, you know, I got some kind of like, yeah, feedback on a separate state that was being recorded separately, you know, like multi-threading, you know, myself. I mean, how weird is that? You know, you can't make this shit.
00:01:36
Speaker
I think we should have our podcast recording bingo and they're like, can you hear me? No. I don't know. I like that idea. Check, check, check, check, check. Everything is checked. Yeah. I mean, chaos at the beginning, you know, that's, that's, you can't get any, you can't get any worse then, you know, that's the idea, you know, that we, we kind of open up with total chaos and then everything gradually settles down and we get more chaos
Guest Introduction: Peter Tosonis
00:02:00
Speaker
at the end. So, you know, there's a kind of,
00:02:02
Speaker
There's a kind of arc to the story, you know? We come full circle in the end. I know exactly. We come back to chaos. I appreciate it. Always, always, always. Entropy is our middle name. We end on a cliffhanger. I think it's even worse. I mean, we start with a lot of chaos and then finally when we're trying to like, oh, this makes some sense. Oh, we're done. So we can't record anymore. So we go back to the same thing again. So every episode we start with this kind of bullshit and then it goes,
00:02:32
Speaker
Oh, finally, we're talking about something reasonable. You know, it's two hours already, so let's stop. It's a beautiful solution in the margin. Exactly. So, episode number 36 it is. Welcome to the show, Peter. Thank you. Thank you very much for having me. I would like to pronounce your last name, though. Yes. How do I, Tosanis? Oh, wow. That's very good. Yes, exactly that.
00:02:58
Speaker
But that sounds Greek though. That is Greek. Yes. So you're from Greece. My father was from Greece. So I am officially part Greek. Wow. Yes. OK.
Cultural Heritage and Personal Journey
00:03:12
Speaker
So that's the reason for the whole tau and everything. OK, now everything is. We're already having insights at this point. That's not right. Part of the reason for the tau, but also other significance in the name for me. So from the mathematical constant, tau, and also from info, with one as sort of the Japanese concept of circle and so on. So it has some significance for me, and I like the sound of the name.
00:03:43
Speaker
Right. It's a bit of a problem now. I was just going to say it's a bit of a problem with the open source from Peter now, because you should always be aware of Greeks bearing gifts. Yeah, Greeks bearing gifts exactly. Well, I feel like there needs to be at least some hurdle. You see, you can tell somebody really wants the library at that point, because they've gone through the effort of actually figuring out how to type it in. So if they're not committed enough at that point, then
00:04:11
Speaker
You know, it's a, it's a good primary level. So, um, let's, let's get to the, um, I think, so you're from, well, you're not from Greece, but you're originally from Greece, but right now you're in Berlin, right? Yeah. So, so I was born in, um, in Johannesburg in South Africa. Um, and I was there for, for many years. Then I moved to Thailand and I was living in the north of Thailand in a city called Shanghai.
00:04:39
Speaker
for several years and most recently in the last year or so in Berlin. So, newly in Berlin, especially in Berlin time, one year is not much. So, what made you move to these different places, Peter? Was it just like an itch to travel? So, Thailand was a bit random at that time. I just sold my first company and
00:05:06
Speaker
I wanted to travel and I kind of picked somewhere that I heard good things about, but it was very, very random. I wasn't planning on actually moving there. Then I went there on vacation and the intention was to kind of hop around a little bit and explore the region. But oddly enough, first place I ended up, which was in Chiang Mai, I just immediately fell in love with the place. And I was so happy and comfortable there that I didn't move again. I ended up there and I ended up staying for five, six years.
00:05:35
Speaker
And I made that my home for a while. And that was a really, really wonderful, wonderful place. And I'd like to go back. As for the jump from Thailand to Berlin, that was a little bit more also, I'd say, spontaneous. My partner at the time was interested in coming to Berlin for her work. And I had traveled here once before briefly. And I liked to look at the place quite a bit. And I for a long time wanted to get into Europe.
00:06:06
Speaker
And there were also some, let's say family things going on, and some personal reasons as well for wanting to make a change. And at that time, again, it was more or less just jumping in with both feet and then giving it a try. So relatively spontaneous.
00:06:24
Speaker
And are you doing closure in Berlin? Because we've been to Berlin recently as we have a closure hotbed. So is that kind of another small magnet for you as well, the big closure community in Berlin? So let's say
00:06:42
Speaker
sort of is the answer to
Programming Languages and Closure
00:06:44
Speaker
that. So definitely there is, as you put it, I think there's a bit of a hotbed of closure here. I am working in closure in Berlin, but that that wasn't the primary draw, I think. No, I think that I'm unfortunate enough. And I think a lot of closure developers are that they they can kind of work remotely very often as well. I think the market for closure developers is relatively small. Generally speaking, closure roles are
00:07:10
Speaker
say disproportionately often remote, so most of the consulting that I did, for example, was remote. It just happened to be in this case that the city coincided with, let's say, the closure in this case. But like in the case of Shanghai, that wasn't a primary factor, let's say, bringing me here. Right, right, right.
00:07:33
Speaker
Okay, so how did you get into closure? Let's talk about your journey. My journey into closure. Yes. So let me think. So I had, this was, I don't know, a little bit before closure, closure version one. So way, way back when, when most of the information was just on the Google, the Google group, and I finished, I think I just pulled my, my first business, which wasn't,
00:08:03
Speaker
wasn't terribly engineering related. And I started working on a project, a fashion project of mine, which I've been sort of kicking around ideas for for a very, very long time. And I knew that it was going to involve a lot of programming. And at that time, I kind of started just researching what programming tools were available. At that time, I had to draw two lists. So I wanted to try, in particular, sort of move towards the list language or at least evaluate those.
00:08:33
Speaker
And I ended up evaluating, I think I looked quite seriously at common list at the time. And I looked at a language which Paul Graham was working on at the time called ARC, which didn't really materialize much. And then there was this closure thing, which I had heard nothing about. At that point, I'd heard zero word of mouth about it whatsoever. It just happens to pop up when I was searching for lists of variants.
00:09:00
Speaker
What initially drew me was that it ran on the JVM. And I started reading, I think, some of the original information that was out there from Rich on some of his views on things like state and concurrency and some of the interesting, let's say, experience and philosophy that went into the language. And it started attracting me more and more. The more I dug into it, the more it appealed to me.
00:09:30
Speaker
And anyway, I started actually developing a prototype of the product that I was working on in Closure. And the more I used it, the more it occurred to me that this was the right fit. And in particular, that it was a really good fit for sort of prototyping. Shoot, great. So you said you were looking for a lisp, but what made you look for a lisp? Why were you looking for a lisp? OK, good question. So I think
00:10:00
Speaker
It's a little bit difficult to turn back. I think in general, I've been looking for an excuse to try a list because I've read much about them and I'd read a few books on lists and I've kind of tinkered around with lists a little bit, but there's a very, very big difference between reading about something or kind of playing around with it in the employee capacity and actually using it in a production environment.
00:10:23
Speaker
Ultimately, I just wanted, I think that the dominant factor was I just wanted to have an excuse to actually use a listing production to get a feel for what that is like. And this was also at a time. What specifically about the lisp was drawing you? I mean, because we're kind of surfing around there a little bit.
00:10:40
Speaker
What particular aspects of Lisp or of, forget closure for a moment, but what particular aspects of the thought around Lisp was making you think, yes, I really want to try that concept rather than Java or whatever else we're using before? Sure, sure. So a few things. Lisp, I think a lot of people are drawn to Lisp because of the
00:11:05
Speaker
Let's say the era of mystery that it tends to have a little bit. I think it's very difficult to confuse the culture around Lisp with the culture around Java, let's say. Lisp is always, at least the way I perceive it, has always been seen as sort of a hacker's language. And a lot of the old McCarthy papers and things like this, Paul Graham articles on Lisp, a lot of the materials on Lisp always spoke about Lisp with this kind of reverence, which
00:11:33
Speaker
I don't, at that time anyway, I don't take particularly literally because people often speak about all sorts of things in very glowing terms. And I think that's one of the reasons I wanted to actually kind of get my hands dirty with it and see what it was like in practice. But there seems to be this promise behind Lisp, right, that there were some kind of
00:11:56
Speaker
There was some kind of enlightenment waiting on the other side of sufficient time with list, right? XQCD style. It's a Jedi language. Exactly. Exactly. And I think for many programmers, that can already be enough to kind of spark their interest because when you're comparing it to alternatives and you see Java or you see other languages that maybe were available at the time, it appealed to me. And I like this.
00:12:20
Speaker
the foundations on which it was built. Again, like the lumber calculus, things like this, the concept of macros appeal to me. And in general, philosophically, I like the concept of something like a list, which is this idea that you take very basic primitive concepts that can kind of compose in interesting ways, right? So it's not complex.
00:12:44
Speaker
I guess it's a weird analogy, it's something like the game of Go, right? Which you were like, you have a simple set of rules underneath and from that can arise sort of organically, a deep and interesting set of logical constructs. And for me, that was something that I had definitely gotten the impression that it was offered. And it was enough to get me to sort of dip my toe in it, as it were. And the experience, let's say, reinforced
00:13:12
Speaker
quickly reinforce and kind of close the loop quite quickly and maybe understand. Yes. Okay. This is actually, you get what's on the box. Um, and it is interesting and it is worthwhile and like anything that's frozen tongue, but definitely there, there are some profound and important ideas here, which are, um, which are worth some time. And, um, which I think if used in the correct way with the correct understanding of the trade-offs can,
00:13:37
Speaker
can be important leverage, let's say, if you're trying to get a product developed or you're trying to do something interesting. So how did that, what were you doing before that? I mean, because you mentioned, you know, various languages, but what, because some, I mean, I take your point that it was a kind of spiritual journey to some extent, you know, that you were looking for a, you know, the Jedi sort of, the
00:13:57
Speaker
the language of the, you know, the ancients, let's say. But also, I guess, to some extent, you must have been feeling some pain or, you know, feeling like there was something that the current languages or the current environment that we're using couldn't quite deliver. There was some unrest in your mind, which drove you to this other place, you know? Yes. So I think that's a good question. The answer is,
00:14:23
Speaker
I actually tend to be quite practical about these things. I think ultimately you can more or less build anything with anything, right, if you're sufficiently dedicated slash stubborn. The thing maybe that was missing for me in other languages was something that is in some way not important and is in some way really, really important, and that was joy.
00:14:48
Speaker
Um, so I started programming when I was, when I was really young and I started with a Borland turbo Pascal, um, I think 7.0, something like that as a blue IDE and the help file and the rest of it. And, um, I, at that time, the first time he was exposed to programming and I, I kind of taught myself and I just used to write little games and things like this. I, I had a particular fascination in the learning process of that. That's one side.
00:15:14
Speaker
started using programming more kind of as a, as a, in a professional capacity, that joy had kind of gone away. And I started seeing the tools and the languages more as a means to an end. And closure and lists and these things, they, they, they kind of reignite that a little bit for me, the, some of the passion that I felt the first time I was introduced to programming as a child. And the first time kind of the life went on. And I thought, wow, this is really neat. I can, I can,
00:15:42
Speaker
I can express ideas here in a really interesting way. And I feel like I have power and I feel like I'm not constrained by the language. I'm constrained by my own capacity to kind of, let's say, push against the boundaries of what, let's say, what my creativity allows. And I guess that's maybe a long-winded way of saying,
00:16:06
Speaker
The joy was something that was missing in a lot of the other things. And it's something that I suspect that I might find in the direction of lists. And I did. And ultimately, closure, I think, for me offered something even more than a list. Sort of a list was what got me in the door with closure. That was kind of like I would put it was sort of one of the things on the box that kind of attracted me. That actually, ultimately, the thing that ended up paying dividends
00:16:33
Speaker
and kept my interest in closure specifically over so many years was more than just that it's a list, but actually some of the specific design aesthetics and semantics in the language that I think were largely result of Rich and his particular views on things and the way that these views have kind of been distilled into the essence of the language in a way that you can actually feel. And that sometimes, like when somebody approaches closure for the first time,
00:17:03
Speaker
They're often approaching it because of the SGM or because of the JVM or because of the list, things like this, right? For things that are easy to understand, but actually the thing that for me makes the language so powerful and so elegant and so, so lastingly beautiful and interesting is actually a lot of the more nuanced stuff underneath, like I say, pointing to sort of riches, which is design and conceptual aesthetic, which kind of permeates the language. And those things I think are almost,
00:17:34
Speaker
maybe uniquely to be having closure or at least aren't often so easily apparent in other languages. Does that kind of answer the question?
00:17:47
Speaker
It definitely answers the question. Pretty comprehensively, I think we can come on to like some of the more detailed stuff and some of the kind of the I mean, I think you're right there. I mean, we wouldn't be both here. I think if we wouldn't all still be here if we didn't think there was some pleasure to be had by using the language, which is a kind of weird thing because I think you're right. There is some
00:18:13
Speaker
There is some pleasure for sure in using it. It can be like a professional pleasure, but also a pure pleasure. And I think that's definitely unusual. I've not really felt that in other programming languages also. But there's also a few rough corners as well. Let's not be- It would be a nice advertisement for closure. Closure gives you pleasure.
00:18:38
Speaker
But it's actually true though, I mean, you know, I'm sure I guess people who are doing Haskell, when they're, you know, when their types compile and all this kind of stuff, I think they're probably very pleased, you know, I mean, I'm sure there's some, there's definitely some pleasure in other languages. I'm not denying that, you know, that's true. I think it's it's it.
00:18:55
Speaker
It comes from the small surprises or some things that you that you thought like they were really difficult and then you get into this new language and then suddenly they click and then you feel really nice because I haven't I think that I agree with you very completely that you know there aren't enough languages that give you this kind of
00:19:13
Speaker
feeling because recently I was playing with Rust and of course you know the first three four chapters are exactly like C and I was like what the hell am I writing you know this is ridiculous and then and then once you get into the concurrency thing you're like wow you know C level concurrency and then it can catch like race conditions and everything at the compile time there's like this is nice so there is just one selling point
00:19:35
Speaker
But then again, I feel really dirty writing that kind of code without any parentheses. Now, I have this buffer around me in my code now. I feel safe and secure and nice around the list now. And if I have to write the free floating C shit, I was like, this is weird. One thing which I do want to say, which I feel kind of compelled to add, is that I think that, especially in terms of things like pleasure,
00:20:04
Speaker
it's inherently subjective. And I think in general, we have a tendency in our industry for getting into this mindset that my language is better than your language, or you're using your language, therefore, you're stupid, or how could you possibly be using this language? It's shit. And that's kind of a human thing. It's sort of a natural thing to do. But I think I just want to make the point that
00:20:29
Speaker
you know, like you mentioned yourself, right? Closure itself has pros and cons from my perspective, even when I'm using it, there's aspects of it that I'm not too happy with. But ultimately, I think the tactile field that someone gets from a language and what they're particularly looking for from a language and what ends up stimulating that sense of joy or not, I think is ultimately subjective. And I think it's ultimately also a question of fit, right? What is the particular problem you're trying to solve? And how close
00:20:57
Speaker
is the language to that particular, to that model, right? And I think in general, certain languages maybe have more of a flexibility to conform in an organic way to more different kinds of models. So some languages might be quite restricted in the sets of things in which they're well applied. But definitely they're subjectivity and definitely there's a matter of shift. And I think ultimately what you do with the particular tools you have matter a lot more than the tools that you're using.
00:21:24
Speaker
I think it's interesting, isn't it, that Matt's, for instance, specifically designed Ruby to be pleasure, to be a programmer's joy. And that was his goal, actually. And I think many people, I've never been to a Ruby conference, but I've seen some shows. And people seem very happy there. And I've definitely read enough blog posts where people are very positive.
00:21:48
Speaker
And I've been to a few JavaScript conferences where people are incredibly happy with the language, you know? So I totally agree, Peter. I mean, it's definitely very subjective, and it's not about, you know, my language is better than yours. It's what you do with it, and you can get a lot of pleasure out of weird languages. That's for sure. Yeah. Anyway, yeah.
00:22:11
Speaker
I think as long as you're writing in brainfuck or what is the other one called Malboglio or something, Malboglio, something like that. Those are specifically made to give you pain. So there were a couple of languages which are specifically for the
Open Source Contributions
00:22:25
Speaker
pain. Well, some people might enjoy that too. People write poetry in Flemish, you know, so... There you go. I didn't pull. Is that pain for the author, for the reader? I'm not sure.
00:22:40
Speaker
I think it's like Lisp. It's an acquired test. OK. Good answer. OK. So let's get into some of the, because you're a prolific open source contributor. And you have so many libraries available, and especially some of them are probably used everywhere in every language, maybe. So of course, I don't think we have enough time to get into every of these things. But there are three particular things I think we'd like to highlight.
00:23:09
Speaker
One is Timber, of course, the logging library. And second one is Carmine. I'm really interested in your design decisions behind Carmine, because I think this is one of the first libraries where I saw, wow, macros make sense here. This is wonderfully written library, and I learned a lot by just reading your code. And then, of course, the center, which is like the whole front-end, back-end, real-time stuff. So which one should we start with? Maybe Timber?
00:23:39
Speaker
truth as you like. So can you give us a, because usually we, we go through different libraries and some of the people who are listening, they're not familiar with them. So maybe if you give a quick idea about what this library about and what is the problem that it's trying to solve, that'll be nice. So timber is a logging library, logging library for closure. And it is all closure, meaning it doesn't lean, lean against any particular Java logging framework of any kind. So out the box you get,
00:24:09
Speaker
full-featured logging, including things like namespace filtering and logging levels and support for different appenders and middleware and all sorts of things through pure closure without interrupt with any particular Java logging or Java logging configuration in particular.
00:24:25
Speaker
Yeah, but this is also for closure script, right? Because you made it using CLJX. And so any particular reason why you're still with CLJX but not moving to the official CLJCE stuff? So largely just the case of being a little bit slow to make breaking changes, right? So the motivation to switch to CLJCE is not
00:24:55
Speaker
uh, extremely, it's not very strong. Um, basically it's the motivation to switch to closure closure or one seven. Um, and at the time, and just just some time ago at the time, um, the last time I evaluated this, my general feeling was what we get an exchange for switching to closure, uh, one seven as minimum dependency was not worth the, um, making that a requirement from the library consumers. And in general, one of the things that I really appreciate about closure and that I, I try usually unsuccessfully to
00:25:25
Speaker
kind of offering in my library code is backwards compatibility when possible and in general a sense of stability because usually my libraries I'm using them in production applications and I know a lot of other people are too and if you're logging library breaks because you know somebody decided to improve some feature or bump some dependency and you're just trying to get your job done or trying to make your project work it's a lot of fun.
00:25:49
Speaker
So in general, I think there's an economic trade-off figuring out when to do that. And at the time, last I evaluated, there wasn't a strong incentive to bump. And I polled on Twitter, and there were a number of people who were still using older versions of Closure. So I stuck at the CLJX. But almost certainly, it's probably about time to make the jump to CLJX. It's about time now, because I think most people have migrated to 1.7, there's 1.8, there's 1.9.
00:26:21
Speaker
Because this is, yeah, of course. Can I just ask one question? It was a bit confusing to me, actually, in your answer there, was about versioning. If you make a version with CLJC, I don't know what your current version is, but let's call it 8.3. You make a version 9 with CLJC, how are you going to break people? I'm a little bit confused there.
00:26:44
Speaker
Um, so if I switch to, let me think about this. If I switch to CLJC, if I switch to closure one seven as a minimum dependency, you're talking about making a new, a new version of timber. I assume you would do. Ah, okay. So not breaking in the sense that, um, people's current version is going to break, but generally speaking, I have a limited amount of time.
00:27:06
Speaker
So if I switch the master branch to Closure 1.7 or whatever it is, most of the work is going there. Hotfixes is going there. My main focus is going there, which means essentially it makes the previous version end of life. So then I'm forced into the situation either, if important hotfixes come out, either to maintain both branches.
00:27:30
Speaker
which is not really something that I can reasonably do, because honestly, if I had that kind of time, I'd rather publish more code. So it kind of just pushes me then into a situation where either I'm leaving a branch to kind of end of life, or I'm being forced to put more time into maintaining two branches. And like I say, I've got something against that if there's a strong motivation to do it. And sometimes that might be the case. In this case, last time I checked, there weren't any particular benefits from switching to 1.74 timber.
00:27:59
Speaker
So people wouldn't have experienced any particular clear benefit to the upgrade there. So there wasn't a strong incentive to incur any costs at that time. Yeah. So I was about to ask, because this is, how did the library evolve? Because was it plainly closure initially? And then later, you started it as a closure script. And the other question is basically a two-part question, obviously.
00:28:28
Speaker
The other question is because logging doesn't come to mind as the first thing when you're trying to do cross-platform on JVM and on JavaScript. So what kind of challenges did you face during this one? So first question, if I understood correctly, you were asking like how the library came to be? Yeah.
00:28:53
Speaker
When I started, like I mentioned, when I started using Closure with kind of pre-Closure 1.0 days, and at that time, I think it's possible I probably have some of the largest Closure production applications in production. This was like way back when, and I had some large applications at that time. And I was constantly running into all sorts of little odds and ends that weren't quite
Redis and Carmine Library
00:29:16
Speaker
working in Closure or like rough, rough edges in Closure that maybe other people haven't encountered so much because
00:29:21
Speaker
I don't know, it might be people weren't working at that scale or they didn't have production applications at that point. One of the things that I ran into at that time was just logging. Obviously, you can log with Java and that's perfectly idiomatic, I think, and a lot of people actually prefer to do that. Certainly, if you're working in some sort of an environment where you're going to be interrupting the Java logging anyway, that might still make sense to do. In any case, at that time, a lot of the products that I was building were pure closure and
00:29:51
Speaker
Um, developing some kind of little logging library that was just pure closure at that time was, you know, like 120 lines of code for what my needs were. And it was really simple and writing 120 lines of code was massively less effort and complexity, actually, then trying to figure out the methods of the Java logging stuff at that point, which I was not familiar with. Um, so it was literally just, uh, a case of reaching for.
00:30:16
Speaker
by Rich's definition, in this case, what was easier for me, it was easier for me to implement the Java log, it was easier for me to implement logging and closure than to figure out the Java logging stuff at that point. And then as these things go, it kind of grew organically. And you mentioned actually, so, so timber, timber, which is on version four, and tarmine, which is in version three, I think both were some of the first sort of
00:30:43
Speaker
at least their original versions were some of the first closure libraries that I've written. And they kind of grew organically as my needs grew as I was working on my own projects or for a client project. And at some point I kind of reached out into front-end stuff or cross-platform stuff or closure scripts. And it occurred to me that in fact most of the interesting work is inherently cross-platform.
00:31:09
Speaker
So for logging on JVM or in JavaScript, you ask like, was there a lot of effort or something where there are a lot of differences? Well, the actual surface area for the interop that you're concerned with is quite small. The surface area for interop is basically the stuff that actually puts stuff out, right? So your panders that are liking something. Most of the rest of it is actually inherently cross-platform, right? Because you're dealing at the compiler level to stuff like
00:31:39
Speaker
um, uh, illusion for, for logging calls that you don't want to run or it's, it's, it's, it's, it's pure closure code. Right. So, um, making the jump to two JavaScript was not, was not terribly difficult. Um, one could argue, let's say, and I probably would argue this, um, that if I'd originally have had the idea from day one to actually make a cross platform, I may have made sort of slight tweaks in the design here and there.
00:32:07
Speaker
Just to make the shared API slightly more polished for the cross-platform application, but in this case, I think it was good enough. Like I said, we kind of grew into it organically and it serves the purpose.
00:32:28
Speaker
So moving on to the next one, of course, this is Carmine's Redis client, client for Redis, I think, yeah. So how did you design this thing? Can you explain how it works? Because if I understand correctly, Redis guys publish their protocol, their whole thing as an adjacent file, then you take it, and then so I think it's better you explain it how it works, actually. So Redis.
00:32:57
Speaker
Redis is awesome. I like Redis a lot, actually. Redis is one of my favorite pieces of technology overall. And again, it's not perfect. There's trade-offs in it, like everything else. But I really, really, really, it has an important place in my heart. I'm very fond of it. And I like the author a lot. He's actually one of the programmers that I take a fair bit of inspiration from. Anyway, so I have a lot of
00:33:28
Speaker
a lot of applications where Redis was a good fit. I took a look at some of the other libraries that we had at the time. I don't actually recall very concretely because there were several other closure Redis clients at that time. I don't actually exactly recall what the issues were. Some of them, I remember for sure, didn't work well in production, so they had issues with
00:33:49
Speaker
strange things like memory leaks because they were using the Java client underneath or some of them would break with large payloads or under high load in general or there were things with connection issues. There were minor little odds and ends that I was running into issues with and at first I was trying to patch the problems with these other clients and
00:34:10
Speaker
Again, it just turned out at the time, my motivation was very specific. I had products that I wanted to develop as quickly as possible, and I was looking for the quickest ways to actually get to production on those products. And it happened to be, again, that sometimes trying to figure out what the issues were with the other clients and fix them in practice was taking longer than just writing something myself for the client. And so I just went sort of with the path of least resistance, which in this case was a new client.
00:34:38
Speaker
Um, so you ask how, how the client works. So this is something that's quite neat. Um, but it's, it actually wasn't my idea originally. There was, there was another gentleman. I can't recall his name, but it's in the commit history. Um, and I think it's in the doc string and the, the, the namespace who copied on this idea of taking the Redis, um, Jason command list and parsing the command list and from the parse command list. So this is, it's for people who are not familiar, basically Redis has, has.
00:35:06
Speaker
a really nicely structured API, and you can kind of see all the command documentation online. And the command documentation is populated by this JSON file. So this JSON file thing, like these are the different Redis commands. These are the number of arguments it takes. These are the optional arguments. This is the doc string. This is the version it was added into. So all of this is actually under revision control as part of the Redis project, the commands for JSON coming like that. Anyway, so somebody with another closure of clients have this idea of,
00:35:37
Speaker
basically downloading the JSON, parsing the JSON, and with that, then defining all of the public API and doing it in such a way where you actually get the benefits of the structural information. So for example, if you're in Carmine and you look at whatever, the ping command, right? And you do docstring on ping, you get the docstring from Reddit. And that's not actually, that's not implemented somewhere, somebody typing out, you know, def and ping, whatever. It's done automatically based on the command spec.
00:36:04
Speaker
And it's a fairly simple, it's a fairly simple, small application of macros, but it's a nice example of where something like a macro can be handy because it makes the whole thing very easy to maintain. If there's a new command that's published, you just update the command JSON and you run the thing and it spits it out. In this case, the current version of spits it out into Eden.
00:36:26
Speaker
and with the even if we'll define all of the functions and things. Yeah, I think this is one of the neatest uses of code producing code sort of thing, like code generating code, code writing code sort of thing. Because I was looking through different libraries, and I was like, now I'm like, OK, you know what? Every database people or everybody should publish their spec like this. Then we can generate APIs so quickly.
00:36:53
Speaker
So elegantly, not quick, but much more, you know. High fidelity as well, the high fidelity. Exactly. So it's really beautiful. It happens that it's a good fit as well for Redis because Redis happens to be implemented in this way where there is a very, very large API and the commands are all more or less
00:37:15
Speaker
I don't want to say I'm not going to exactly that they're all just independently implemented like this, right? So if you were going to implement a client library for some other database, it's very likely that probably if you were doing a closure one, you would want to kind of fine tune the API enclosure for some reason, right? So like, for example, there's a Datomic DynamoDB library as well, which isn't used very much called Faraday, which I authored. And there's a number of tweaks on top of the underlying library because it made sense to make those kinds of tweaks
00:37:46
Speaker
for the fact that this is social. In Redis's case, it's an unusually easy and sort of natural fit for just, you know, define this function, define this function, define this function, because that's the only way that you would want to do it, if this makes sense.
00:38:01
Speaker
Just going back originally, I know you kind of skipped over it, but you said that you really liked Redis and I appreciate that too. I was wondering if there was something specific about Redis versus Memcache, for instance, that made you think that this was special or different. What is it about Redis that inspired you? So Redis,
00:38:29
Speaker
There's a lot that I like about Redis. Some of it is quite subjective. So some of it is just, I like the aesthetic of Redis. So what I mean by that is... Subjective is good, Peter. It's an opinion show, it's okay. I'll give you both. I'll give you the subjective and I'll give you what I think is slightly more objective. So subjectively, like I said, there's just some aspects of the design philosophy that I enjoy. I like how Salvatore runs the product. I like how he does design.
00:38:58
Speaker
I find that his manner of thinking, like Rich's manner of thinking does with closure, his kind of percolates through the design of the software in a way that you as a user can actually kind of tangibly feel and that I appreciate. Objectively, I would say I really appreciate how close to the metal lettuce is.
00:39:20
Speaker
I like thinking in terms of data structures, right? And this is something that I think closure tends to encourage as well, is just thinking in terms of data and thinking in terms of data structures. And in general, I find one of the things that I dislike about some databases in general is it's often quite opaque, right? You send your data to the database and it's up to the database to figure out how to structure this stuff and how to index it and exactly what data structures are underneath. Maybe you know, maybe you don't, maybe it's part of a public API, maybe it isn't.
00:39:49
Speaker
Um, but ultimately you're, you're, you're leaning a lot from the database to kind of make reasonable decisions about how to store and represent and access stuff that you need. And one of the things that I like about Redis a lot, so I often use it in cases, not just as, um, so like you can use it as an alternative to memcache, something like that. But I think that where it's more interesting is just as a, as a data structure server, right?
00:40:16
Speaker
If you need a list, you need a list of certain specific semantics. You need a sorted set, right. It's got these really, really interesting data structures with an API that's very, very transparent about the cost of operations against the particular data structures that by itself. I think already actually puts it in a unique position where
00:40:37
Speaker
When you want a specific data structure, and in my line of work and all other stuff I've done, I often wanted a specific data structure. Nothing beats that. It was just amazing to be able to say, I want this and I want that, and I want to hash over here, and I want the sorted sense, and I have very particular reasons for wanting these particular things. And it's allowed me, in a lot of cases, to sort of refactor, let's say, applications that were based on other databases and dramatically improve performance.
00:41:02
Speaker
Not because Redis was fast, right, which is often kind of thrown around, but because Redis allowed the conceptual remodeling of the idea in terms of the actual underlying data structures that were necessary, and that allowed sort of the algorithmic complexity to be drastically reduced. And I've had cases where, again, we have production applications that needed to be
00:41:21
Speaker
Um, the performance they get to be improved and just switching from this concept of kind of, I'm throwing my blogs at the database and it's going to figure out how to create this stuff back to me. Um, to look what actual data structures do we need here and maintaining them ourselves and taking responsibility for that and improving performance in the system, several orders of magnitude, because just having that, that ability to think in those, those terms and to be very specific about it. Um, so that's, that's what originally got me like with us. Then at some point there were some.
00:41:49
Speaker
editions, including the edition of the Louis scripting in Redis. And for me, that took it to another level once already, it was very useful. And again, in the line circumstances, I don't want to suggest it's a magic bullet for anything, but in the uses where it's a fit Redis was very useful, then the addition of the Louis scripting, I think took it to another level because it allows it. So for people who are not familiar, sorry, basically, Redis allows you to
00:42:17
Speaker
those right little newest scripts on top of its core data structures and actually send scripts to the client, excuse me, send scripts from the client to the server and execute atomically the script. And this allows you to do all kinds of interesting things in a really, really performant way in a relatively simple language, but again, really close to the metal in a way that
00:42:42
Speaker
when it makes sense can be awesome. And again, this often might not make sense. Then in those cases where you want to be able to send something to a server and say, look, I'm sending you this thing and I want you to perform a hundred operations against these different data sets. And please take an intersection of this and this and put it there in this map only for it is amazing. And again, I've had applications where massive amounts of logic that involves locking and all kinds of coordination between different databases and things could be dramatically simplified through the use of something like this blue engine in Reddit.
00:43:12
Speaker
Um, and the results were just so much faster and so much cleaner and so much more tangible. Um, because again, it just, it gives you transparency into what's happening. So I guess for people who have, um, like I do sort of an aesthetic in general that they like to understand what's going on and kind of miss the days where you're sort of kind of doing your assembler and you know, what's in your buffers and then, you know, where your cash line is. Um, I, I, I kind of for languages, I feel like the value of that is, is.
00:43:42
Speaker
in general, decreasing, right? Because serious reasons is one conversation that in terms of data storage, I actually sense a lean more in that direction, still of liking, liking to understand exactly how the stuff is, is stored. Then more most recently, Redis has also introduced things like the module feature, which now gives you kind of like a binary binary level way of doing doing some of these like extension. And again, when it's the fifth boy, it's
00:44:07
Speaker
When that's what you want, it's just, it's great. It's unbeatable. So do you think this is like the revenge of stored procedures? Because stored procedures kind of like were a dirty word 10 years ago, I guess, you know, but with Oracle and a lot of the databases because they kind of tied you into a specific
00:44:29
Speaker
language for that database. And I guess people like Datomic and, you know, as you say, Redis and other people now, you know, CouchDB and MongoDB, they're all kind of back to stored procedures, big time. Yeah. Yeah. So in general, this concept of dirty word, I think, is interesting, because I think that it's something that, again, our industry, let's say, has some growing up to do.
00:44:59
Speaker
in terms of just evaluating things on a case-by-case basis. We have a tendency as an industry as a whole to say, okay, this is currently the mindset. This is what's good. Object-oriented programming is fantastic. Everything that's not object-oriented is terrible and you're an idiot for thinking it. Or go-to is great. No, go-to is terrible. You shouldn't have go-to. Functional programming is the best thing ever if you're not using functional programming.
00:45:20
Speaker
you're stupid. And I think that but that is true. On this podcast. Yes. I'm joking. I'm joking. Of course, Peter. So like I think in general, I can't have an automatic sort of suspicion when people say things strongly, they have done with things like stored procedures.
00:45:44
Speaker
I think that very often criticism of things comes from fashion more than a deep understanding of the semantic pros and cons that the thing offers. And I think that you can do good programming with PHP or you can do shitty programming with closure. And I think that you can make stored procedures work and I think you can make them dreadful. It depends on the context. It depends what your objectives are. It depends what the economic factors are and the constraints. Like I say, Redis is
00:46:12
Speaker
Um, is dear to my heart because it's so transparently what it is, right? Like in some cases it's absolutely not a good fit. Um, but you always relatively transparent to know what those, what the cases are where it's a good fit because it just gives you exactly what it says it's going to give you. Um, and I think that when it makes sense, I think it's a very good fit. Um, but I think that it's sort of a deep question or a deeper question. Maybe we don't have time for right now, but it's certainly interesting one.
00:46:40
Speaker
is how one actually evaluates in general what kinds of ideas make sense logically. Like something conceptually like a stored procedure, what actually the pros and cons of that from like a semantic, from like a conceptual semantics point of view.
00:46:52
Speaker
And I think that these are actually really interesting questions that sometimes get skimmed over because people have a knee-jerk reaction to say, I read that article that says it's bad, or I read that article that says microservices is good, so we should use microservices. Well, I think we've got five more minutes. I mean, you know, we maybe spend an hour on it, but I think let's not stop there. I mean, you know,
00:47:13
Speaker
Because i think one of the reasons why people were the tip of me okay i'll just kind of give you the i'm sure you're aware of them but i'll give you kind of like a for everyone isn't i'll give a quick rundown of some of the negatives of stored procedures and the typical reasons behind stored procedures negativity is there's some sort of limit on the language.
00:47:34
Speaker
I'm in the words to some DSL by database vendor and people didn't like that for whatever reason i'm so the limitations of expressivity and stuff like that i think you said i can be still good reasons to choose that the bigger reason probably is the lack of transparency in the tooling and all those kind of things where,
00:47:54
Speaker
You have to buy into the tooling of a particular vendor to get the advantages or to have visibility on the processing of a particular stored procedure. And then, of course, you're getting a lock-in aspect, but let's ignore that because that's obvious. So I think those are the two big things, expressivity and kind of lack of visibility. So those are obviously very important things.
00:48:24
Speaker
And I can't say that there are, how do I put this? So I think that the important thing in general when evaluating any particular thing is trying to understand what are the actual constraints of the particular problem that you're working with and what are the hard parts, right? So it happens, for example, sometimes that
00:48:46
Speaker
You would hypothetically acknowledge, let's say that the expressivity of the language that you're being offered in your database is not particularly powerful or it might happen that there is vendor lock in or it might happen that there's there's other I think valid valid criticisms also for for store procedures that one one could level and
00:49:04
Speaker
Ultimately, it's up to the people involved, right? Engineers and business, both to try to figure out actually, what are the pros and cons that we get from this in this particular case? And what actually makes sense? Because sometimes it will happen that you say, look, these are particular kinds of death lists that we don't want to take on, or it's something actually where the overall set of trade-offs is, is positive compared to other reasonable alternatives that are available at the moment. And again, just giving sort of some anecdotes,
00:49:32
Speaker
So definitely Redis scripting, you have some of these same kinds of problems. For example, if you're writing your script now in Lua, right? We're choosing to use Closure and not Lua. Closure is more expressive, Closure is more powerful. Also, let's say you've got some issues of, let's say your tooling, like you were saying. You've got this code and where is it being set up? What kinds of tools do you have available for testing and debugging things like this?
00:50:01
Speaker
So all of these are costs. The question is, what are you getting in exchange for it? And whether that is worth paying or not, again, is purely a function of what are the bottlenecks in your case and what are the viable alternatives? So I've seen cases where someone was using Redis and they were implementing a lot of important logic in the application. But because the application was distributed and
00:50:26
Speaker
So closure applications, because the application was distributed, trying to implement this logic on the client side, on the closure side, involved a significant amount of concurrency management, right? So it was figuring out sort of locking and distribution of information and timing things. It was a reasonable solution, and it allowed, let's say, the bulk of the problem to be expressed in closure.
00:50:56
Speaker
because of the location of where it's being expressed, right? In this case, in the client, instead of the server, you sometimes incur other kinds of costs. For example, in this case, conceptual costs, due to the fact that the distributed architecture, at least some parts of it now need to be dealt with, as the application layer, the client layer, where moving some of the stuff to a Lua script, even though the language wasn't as expressive, and even though the tooling was not as nice, allowed
00:51:24
Speaker
let's say, massively complex, like logically complex distributed code to be squashed into something that was actually relatively short and simple for Lua. And in that case, one of the, let's say, the economic constraints as well was the objective was performance. And switching from this kind of multi back and forth
00:51:49
Speaker
solution with the distributed thing from, from client stomach, from closure to something that could be done just on the server side directly within the server. Again, resulted in, in maybe an order of magnitude, better performance. And ultimately that's what the customer in that case cared about. And the complexity, uh, let's say the cost, the other costs that were incurred were incurred happily. So I think it's, it's always a matter of, of trade offs and what you're actually trying to achieve. I don't think that there's a right or wrong solution. I think it's very much a case of,
00:52:18
Speaker
What are the actual objectives and what are the viable alternatives, but in general, I think that it's an interesting option to have right the concept of being able to run things on the DB on the service side, I think.
00:52:30
Speaker
certainly opens the door for some kind of interesting options. And like I said, sometimes it's a dreadful idea, but it's one of the things that I like about Redis that it gives you that option and in a particularly nice way against a particularly powerful set of basic primitives. In this case, it's basic data structures. I think it's a good fit because there's a lot that you can do with these data structures.
00:52:51
Speaker
By the way, do you use Redis just like a memcache sort of replacement, or do you actually use it like a full-fledged database where, I don't know, stored to the disk when she hits the ceiling sort of thing? Primarily, the first case is usually what's recommended, so as some sort of temporary thing. Just because the defaults are
00:53:13
Speaker
So again, the design of Redis is very specific, acknowledging that there's different trade-offs to be made in the ways that we write software, and Redis more or less has taken a clear direction, which is to generally de-emphasize things like the persistence, the reliability, right? You wouldn't put very valuable data in Redis, usually, unless you've got it very specifically configured, because the default list, they are not
00:53:43
Speaker
are not slanted towards giving you very high reliability in sort of the database sense. However, again, it depends on your use case. I think that saying that you shouldn't use Redis ever as a primary data store, I think is just as silly as saying anything else without condition, right? It depends very much on the circumstances. And I've seen plenty of cases where, again, there were projects I was involved with, where the
00:54:08
Speaker
constraints were things like time to market or cost or performance, and the data was not valuable. And the thing is, if you have data from your kitten photo application that's going back and forth, and you need it to be really fast as some of the people are using it, but if you lose
00:54:26
Speaker
information on 10 kittens is not the end of the world. Something like Redis as a primary data store can actually work. You just need to understand exactly what the trade-offs are, what semantics you need, and how are you trying to mitigate some of the downsides, right? Because you would need to be very clear on exactly what are the risks to your data, at what time do you incur those risks, and more or less what are your options for trying to mitigate them, assuming you need to.
Real-Time Communication with Sente
00:54:46
Speaker
So it very much depends.
00:54:49
Speaker
But I wouldn't rule it out. I wouldn't rule it out. Of course. Before we turn this into Redis podcast, you should really go into one of these MongoDB podcasts and then talk about Redis. I have used MongoDB before.
00:55:07
Speaker
So let's talk about one of the biggest libraries that you released, Cente. I don't know how you pronounce it. Can you give us some idea about what that library is about and its use case and how you built it? Yep. So for folks that aren't aware, Cente is basically just, if I kind of describe it as a real-time, how do I describe it?
00:55:34
Speaker
real-time communications, web communications, foreclosure and closure scripts. So essentially it's web sockets, client server, foreclosure, that falls back to AJAX and has a number of other, let's say, quality of life things that I found useful in real production applications. So for example, these days I had a question, an interview with someone just the other day and the question came up,
00:56:03
Speaker
Why would you use something like Sensei today instead of just working directly at WebSocket? And the honest answer is so when I wrote Sensei, at that time I had a product that I was working on where I had measured that a significant portion of my user base was not WebSocket ready, right? At least not practically, through proxies or whatever there were issues and WebSockets weren't available.
00:56:25
Speaker
So in my case, it made sense to spend some effort to make sure that I could support those people in a graceful way. Now, is that still the case today? Probably not, right? The number of people who are not successfully able to use WebSockets at all is probably a certainly decreasing number at the moment. But some of the reasons why something like Sente might still be interesting anyway is because there's more to actually building a successful production
00:56:50
Speaker
let's say real-time communication system client server than just the city underlying protocol for sending a message of a web socket or whatever. There's a bunch of other little things. Sometimes it's things like time-ups, it's things like your general API, things like keep alive. It's unsexy stuff that sometimes these things are important, right? If somebody's driving through a tunnel and they leave a message, what happens with that message? Or what happens if I'm sending an important message and it's big and I
00:57:18
Speaker
whatever. So there's all kinds of things that you can already do at a library site to try and let's say maximize the use of bandwidth, which can sometimes be important on mobile by like, packing things together, or doing things in such a way where I'm going to retry certain things in certain cases when it's logically logically sensible to do so. So basically, it gave me a layer to operate
00:57:42
Speaker
One higher than sort of the web the web protocol layer, but just to say I have actual applications that I want to build and for these applications. I would like to be thinking at the application level because it's already hard enough to build a real time application even using something like sensei. There's a lot of
00:57:57
Speaker
There's a lot of things you have to be aware of, and you definitely have to design for that at the application level. And so it forces a shape on your application a little bit, sort of the notion that you're going to be working with this real-time stuff. But I wanted to minimize the amount that that intrusion into the application space happened. And so doing something like Sensei allowed me to capture some of the complexity, let's say, at a middle layer, which was a little bit higher than the protocol, but a little bit below the application. And so a lot of the stuff between applications
00:58:27
Speaker
kind of similar. And again, a lot of the work that I've done is like prototyping a lot of different applications or doing like lots of different, especially web applications for clients and having just a reasonable default platform on which I could build a wide range of different applications was useful. So that's why I built Sante and then at some point I just released it because I figured it might be useful for others.
00:58:56
Speaker
Does that answer the question? Yeah, of course. So it's basically sort of a middleware, right? Because it's not forcing anything onto the front end for you. It's not forcing anything onto the back end for you, like forcing you to use Ring or whatever, or any specific kind of thing, right? Good question, now.
00:59:16
Speaker
Actually, funny enough, I haven't looked at Sensei in long enough, but I can't even tell you whether there's officially a requirement in Ring or not. I don't recall. You would need to check the documentation. And certainly, it may... There is a requirement, yeah. There you go, Ray knows. But you do support various web servers, so that's good. It should be kit or...
00:59:41
Speaker
Yeah. So I'm guessing it's probably, if I recall correctly, probably be something like a ring compatible web server. And then there's a protocol that the servers can implement to kind of plug into that. And in that case, let's say maybe that could even be abstracted away so that you, it's not dependent on ring exactly. But I think in Clojure anyway, that the ship has sailed ring is more or less one. I'm not sure if there's any really, really viable
01:00:06
Speaker
widely deployed alternatives to at the ring level. So again, it's just in my case, it wasn't worth time exploring, because I didn't care. But if somebody else is interested in that, theoretically, maybe it's something that could be explored, I can't think of any logical reason why, why it wouldn't be possible to work on something else. But yes, sorry, what was the question?
01:00:25
Speaker
and one for the front-end as well, because we were talking about, is there any specific dependencies that are here? It may force certain shapes on you a little bit. I can't think of anything too concrete. In terms of the API, as far as any API, of course, there's anything on you. If you're going to be using this as a core part of your application, and usually something like Sentai is used as a core part of the application, then you will find yourself naturally conforming a little bit to the API that
01:00:55
Speaker
to the shape that the API is kind of encouraging you to take on. Like I say, more importantly, the thing that often bites people is the stuff that's not immediately apparent. So again, if you're building a real time web application, you need to be designing it for that purpose in mind. And you need to be thinking from day one, or at least thinking very logically at some point about
01:01:17
Speaker
What are the different things that can happen in a real time web application in terms of your state and in terms of the business logic or the behavior of your application? And that's actually kind of a hard problem. I mean, it's not, it's not classically hard, but it's something that a lot of people aren't familiar with. And if it's your first real time web application or real time application, where kind of messages are going like this and there's, there's often a UI involved, let's say, um, it requires some deliberate thought because there's some.
01:01:45
Speaker
decisions that you're going to have to make as an application developer that the library can't make for you because it's going to be entirely application dependent. And you need to be aware that messages can arise out of order. They can arrive. Sometimes this one will arrive and then it won't arrive. And then, you know, message three out of a sequence of five will arrive because, you know, somebody was driving through a tunnel or their wifi was bad or they switched. There's all of these different kinds of things. And you need to make sure depending on your particular application that
01:02:11
Speaker
You've more or less thought through the semantics of what it means to be doing real-time message processing in your particular application. And for some applications, it's pretty simple. For some applications, it looks deceptively simple in that it ends up actually being quite complicated to make sure that you've
01:02:28
Speaker
kind of caught all the edge cases, but it depends as well what the risk factors are, right? Like if it's just an online game or something where it doesn't matter, then maybe some sort of semantic breakage is not that big of a deal. If it's something where you're dealing with important information and important decisions are being made based off of the common infrastructure, then you need to have your semantics, right?
01:02:47
Speaker
I think to me what's interesting, because I mentioned to you before that I'm using Centa for this distributed shared REPL that I'm building, is that in terms of API, the API is quite small. In fact, there isn't really an API as far as I'm concerned because you just plug things in.
01:03:08
Speaker
And then you have some kind of like model multi-methods and then you write your own, you know, you as a library, as a tool author or as an application developer, you basically write your own multi-methods on either side of the socket connection and just route your data that way. So it seems to be more about, almost more about like the data design
01:03:30
Speaker
in that respect. At least that's how it feels to me that it's more about the data that's going over the wire and coming back than following a particular API that you've imposed on us, which I really like, by the way, because I think that's using the power of it's not constraining me to say, OK, well, you must use these sockets in a certain way. You must get your data like this. You must get your data like that. No. I write the API essentially.
01:04:01
Speaker
Yeah, so I think that was a nice observation about about it being mostly data centric. And let's say I think that the API is what there is of it is there to kind of support the conveyance of data in a way that's that's useful. Like I said, the particular the particular application design that you ultimately end up having or needing or wanting for a real time webcom.
01:04:29
Speaker
That's something that you would need to think about from an application to application basis. And it may have interactions with how the Sentai API works. So the API, as small as it is, let's say it gives you certain basic things like I'm going to send this and not care about the results or I'm going to send this and I want to look at the results and there's a timeout. And so it's very basic solving blocks that it gives you.
01:04:53
Speaker
And the intention is that these building blocks kind of pave over the differences between the particular protocols, or in this case Ajax and WebSockets. But I guess it's a little bit subjective to say, is that much of a... Is that...
01:05:11
Speaker
forcing much of a shape on you or not, I guess that's sort of debatable. Some of the aspects like how identification, user identification work and things like how the keys work for identifying messages back and forth, you may find cases where, let's say, Sente can be a little bit opinionated. And for example, I've had cases before where people wanted to use it in applications where they wanted the user ID to be based off of
01:05:40
Speaker
something that didn't particularly fit well with the model that that sent a was originally designed to support. And there can be times where you will sort of buff heads with it a little bit because it's it's fundamentally going up against the direction of of the kinds of applications that I was intending to build with sent a it can happen. So you won't encounter it. Let's say on the on the simple on the basic stuff of, you know, this is the send API that you may encounter it on on
01:06:06
Speaker
in deeper ways of how things like identity work and how the application in general should be structured in order to make, like I said, sort of a conceptually sound real-time application. So there may be pushing from the library to conform to a certain kind of shape, but in my case,
01:06:25
Speaker
I would argue that most of that pushing is actually healthy, because most of the time when that pushing isn't happening, people haven't even realized they need to be in a particular shape. Because it's not apparent at that point that there's a reason they really have a serious think about adopting a shape of some kind or another to deal with things like identity. Because, again, sometimes these things when you get started, they seem kind of simple. But when you actually dig into the details, and you dig into
01:06:51
Speaker
the nuances of how browsers actually communicate state and how it works with tabs and different devices and things like this. Occasionally, it will encourage you to go down a certain road and it will provide a specific path that offers least resistance, if that kind of made sense. It was a bit of a rambling answer. I fell asleep. Sorry, what were you saying?
01:07:18
Speaker
No, you're right. I know the whole business about authentication and logging and all that kind of stuff. I see on the chat that there's a little bit of contention there, let's say.
01:07:37
Speaker
Yeah, I mean, you're going inevitably, you know, and what you're doing, and it's interesting to me what you've done is you've essentially said, look, you can log in via Ring or you can log in via WebSockets. And then if it's WebSockets, you're on your own, make it up using this sort of technique. And if you're using Ring, then it's just a standard middleware.
01:07:58
Speaker
That was the only thing that I thought was a bit odd was like, you essentially hide some of the data under some atoms that only read on the, you know, so like, user membership and stuff like that. That's the, I think in the end, you're right. It was just something which I hadn't expected, that there was sort of hidden behind the API, that it was some data structure that you took care of and I couldn't have an input on. So this is the biggest,
01:08:28
Speaker
Source of contention with centen, I would say that it's one of the bifurcation points where you would ideally decide whether you use something like centen or not. Because again, centen took a very particular direction to solve particular kinds of problems. And by making certain assumptions, it could make lots of simplifications or at least provide lots of support in direction A.
01:08:53
Speaker
That direction is inherently in opposition to some of the other kinds of applications you may want to build in direction B. And for things like, you know, like, for example, a request that I get quite often is I would like to be able to send a message to a specific client ID, not to a user, but to a client.
01:09:13
Speaker
And there's long posts on GitHub about what the pros and cons of that are and why the library has more or less rejected the notion of being able to do that at the client ID level. And I understand that might be frustrating because sometimes people want to send to a client, but in general, again, it's been my experience that when I've seen these things in the wild, nine times out of 10,
01:09:36
Speaker
The desire to want to go in that direction is an indication that you haven't properly sought out how this is actually going to work properly. And you're going to end up regretting this decision at some point. So I would normally try to reverse the kind of mental model and say, look, if you've got something where you want to have a particular response to a particular client, try to initiate the request from the client. Because that way you can use the basic API and get all the sort of, anyway,
01:10:03
Speaker
It's a bit of a long story, but suffice it to say, there is certainly a different design choice that could have been made there, and that might make sense in certain applications. But at least in my experience, I'm not convinced that I've ever actually seen a use case where I've been fully convinced, had a strong argument for wanting to go this direction, or the opposing direction.
01:10:28
Speaker
Every use case I've seen where people had the inclination to want to go this way on sitting down with them and actually working through the design, we both concluded afterwards that the direction that it encourages you to go actually made more sense. It just required a bit of a mental rework of the situation.
01:10:44
Speaker
But it could totally happen. I can't exclude the possibility that there may exist some application that would actually be better fit for sending something to the client ID level. And if that comes, and if somebody can suggest that, then we could maybe open up the API to do that. Or indeed, I think you can have another library do that. I think that it's important that people use something that kind of fits their particular use case.
01:11:05
Speaker
I think an example of that for me was where I'm evaluating something in my REPL locally, but I want other people to be aware of it. So I send it over the network to be evaluated, and then the server-side evaluator basically broadcasts the results. And in theory, it could broadcast it to the other clients and not to me.
01:11:28
Speaker
But actually, in practice, what I did is I just, well, I just ignore it. If I know the producer of that message, I can just drop it because I've already got it. So that turned out to be a much simpler solution, in fact, where I can just say, okay, I'm initiating the action and everyone gets the result. And then if you're not interested in it, you just drop it.
01:11:50
Speaker
And that turned out to be a much simpler model. And I guess that's probably where you're coming from as well. This is a good example. And there's trade-offs. As in the example you mentioned, obviously one of the disadvantages is that you're now broadcasting to clients maybe that are intending to drop it, right? So you're wasting bandwidth, you're wasting time. But I know it's only one. I know it's only one or, you know, out of four or out of 400, it's only one that's going to drop it. So it's okay.
01:12:20
Speaker
Yeah, so in that case, it sounds like that's a pretty close, pretty accurate example of what I had in mind.
01:12:31
Speaker
I mean, if you're going to send it out and then half of the population is going to drop it, then maybe it's a bit more tricky. But then I think the other problem is that how do you register your interest? Then you're putting some filter on the server side then. Stored procedure styling. It all gets a bit funky. Like you said, it's a general purpose library.
01:12:56
Speaker
Yeah. So the honest answer is I actually don't even recall exactly what some of the design, the design traders were from this major because it's been a long time since I've looked at them. I deal with so many different things. I couldn't even remember where the ring was. It was a requirement for the library.
01:13:12
Speaker
So normally what happens in these cases is if somebody if it comes up that it's somehow relevant I'll kind of look at the issue again I'll refresh my memory of sort of what all the trade-offs are and what the what the reasons were. I don't have a cash at the moment But certainly I remember having having had this experience before with with clients where there was a feeling to want to kind of go in a particular direction and I have
01:13:37
Speaker
gone through it several times where, when we actually evaluated what the semantics were, we always found edge cases where there were problems in going in the direction that seemed attractive on the surface. And ultimately, we decided to go with something simpler, which is kind of what Sente was encouraging. But again, I absolutely don't exclude the possibility that there may be some case where that won't make sense, where some other approach would be preferable.
01:14:00
Speaker
I don't have enough of the details in my head at the moment to sort of clearly talk about what I anticipate those cases might be or what the trade-offs would actually be. I need to look at it.
01:14:10
Speaker
I think what's nice about it, though, from my perspective, to get back to the sort of concrete positives, is that you can choose the data structures in a way that's very nice for closure. You can put Eden or transit on a wire. It's very easy to parse that data then. It's easy to sort of, from an API perspective, I'm kind of
01:14:36
Speaker
I kind of spent a long time, like you were saying before, about kind of like absolutes. People went so up for a few years and then, ah, it's terrible. And then it went rest. And now, you know, because Netscape and Facebook, Netscape, Netflix and Facebook don't like Netscape. I was there for a while. But, you know, when Netflix and Facebook said that rest was terrible,
01:15:02
Speaker
Ah, yes, GraphQL, that's what we need to do. But what's interesting, I did rest for a long time. I didn't do much stuff with WebSockets before. So this is my first big WebSocket application. Big, not very big, but you know.
01:15:18
Speaker
first proper investigation into that rather than a theory. And I'm finding WebSockets a real pleasure to use actually with this tech. I think if you're doing kind of bits manipulation, if it's all binary stuff, I imagine WebSockets are horrible, but using it over with center, over closure,
01:15:39
Speaker
It's so easy. It's so much actually nicer than a REST API because you're just talking message passing, which is really nice. I'm not talking about RPC here. Just here's some data, do it the way you want. And then on a reframe side, you just look at the data and say, OK, that's that type of data. That's that type of data. Put it in the database, trigger some event, and the UI does something magical. And it's all good.
01:16:05
Speaker
Yeah, it has been a big pleasure. And I've used it a lot for again, a lot of work I do is prototyping applications. And it's, it's been a joy for that because you can get started so quickly. And like you say, it's a good fit for things like, like react, the reagent, whatever particular library you're using, you're just passing data back and forth like this. And it's, it's, for me, it's been a source of, of
01:16:36
Speaker
significant productivity or some real applications to be able to do message passing like this. I found it quite enjoyable.
Code Invariance and Debugging with Truss
01:16:45
Speaker
OK, so I think we can talk about one more library. I know I said three, but obviously there are plenty of things. One more library. One more thing. Just one more thing. One more thing. We're going to have to have another show in a year or so. Secret announcement.
01:17:06
Speaker
Because I'm really curious about your goals for Truss. It seems like a sort of spec, but not really. So what is it, and what is the idea behind it, and where are you taking this one to? So Truss, I actually think, is probably my most interesting library. And I think it's a bit of a strange one, because I think in some ways, it's one of my favorites.
01:17:35
Speaker
Um, but it's not very well known and it's not very, very widely used. And it came along at a bit of a strange time, at least it was a published published at a bit of a strange time. So trust, um, for anyone who doesn't know is, um, I kind of describe it as, as basically an invariance library for closure. And, um, what I mean by that is essentially it's a way of making, um, making assertions in your code base about invariance in, in your system, in your code at that, at that point in time.
01:18:03
Speaker
Like you mentioned, there's some, let's say, functionality overlap with things like Closure.spec and schema. So it was a quick history. So I mentioned before that I started using Closure way back when, like already version 0.9, something like that, that I already had at that point a large production application going.
01:18:28
Speaker
We were talking earlier about things that we like or dislike about curious tools. One of the things that I found at that time challenging about closure was the dynamic piping. And in general, I actually tend to have a preference towards strong static piping in general. Again, there's trade-offs, and I think if you kind of use them with your eyes open, closure gives you a lot of leverage to, let's say, minimize the pain from this. But anyway,
01:18:54
Speaker
I have some production applications where the dynamic typing enclosure was really hurting me on an ongoing basis. It was causing problems. It was causing lost productivity. And I noticed that when I was consulting, it was occurring very, very often as well in, in, in the wild. So the same kinds of problems that I was running in with my larger applications, I was observing a lot in client applications in places where I've kind of been called into help with the code base or it was starting a project of some kind.
01:19:23
Speaker
And it basically boiled down to a lack of ability to convey, semantically, some of the invariants that we have in our system. So a type system is one obvious way of doing this in one way to say, OK, this argument is an integer, the return type is a whatever. In practice, what I noticed with closure was
01:19:44
Speaker
Most people didn't experience this pain because most people were running small projects. If you have 1,000 lines of code or 4,000 lines of code, the dynamic piping doesn't bother you at all. It's fine. You don't even notice it. Because the program is so small, you have the whole thing in your head. And especially if you're writing closure in, I don't want to say exactly in the idiomatic way, but in a way, the closure encourages you to write where you tend to have a nicely decoupled system. So you can actually get pretty far in building real large closure applications
01:20:13
Speaker
while being able to retain the whole thing in your head. Because even if there's lots of concepts, the way that they kind of compose is nice and clear, so it's easy to stick a lot in there. But ultimately, every closure project, I think, that gets big enough at some point becomes large enough that there's part of the code that you haven't seen in six months or a year. And it's complex. And in particular, when you have applications which have complex business logic,
01:20:39
Speaker
So like a lot of applications which I was working on, which were highly mathematical or which we're dealing with a lot of like complicated algorithmic stuff. It becomes really, really difficult to keep track, right? I'm making the refactoring of this thing over here. And I think that I recalled it this or it's covered in the unit test sort of, but it wasn't exactly completely specific. Or I mean, famously one of the things that closure is a bit of a sore spot for people is its error messages.
01:21:06
Speaker
Right. And this comes again, everything goes in context. Being on the JVM is wonderful and a wonderful source of productivity and leverage, but you know, the fact that closures also implemented on all this book underneath makes your messages. I mean, I'm not sure if that's the only reason, but certainly that's one of the reasons why the error messages are, can be so okay. And, um, one of the things that I noticed was on my first on my own project and then also in, um, in the consulting was.
01:21:36
Speaker
When there's errors in production, in closure, sometimes they are so maddeningly difficult to debug because you get some funny stack trace in some little function in the middle of nowhere and it says it's a null pointer exception or it says there's some whatever ridiculous stack trace that again very often at the times when you're supposed to be debugging these things money is being burnt and there's urgency to try and get a solution and it's incredibly frustrating when the amount of information you get out of the system is so
01:22:04
Speaker
so small and or so, so unhelpful. Anyway, so at that time already, this is way back when this is before schema, this is before anything else, I wrote a little library. So at the time, just a set of utilities, which I started using for myself to cover two use cases, number one, refactoring, and number two, basically, this case I described with like issues in production. So when you're writing some code,
01:22:34
Speaker
What I found was, particularly for large code, while you have it in your head, and it's clear, in particular, it's clear how different pieces connect, you have a certain understanding of how, let's say, assumptions which are implicit to you at the time that you're writing the code but aren't explicit in the code, either because they're minor and they're not a part of the public API or something within a function.
01:22:57
Speaker
or it's something about how this component relates to that component and trying to keep this stuff documented and trying to keep it all sort of in a way, let's say, covered by the unit test or something else which is actually reified in the code so that it's under revision control and it's there as part of the contract of the code is tricky. And like I explained, I've got a video on the Trust Library where I kind of go into some examples of where this comes up, but essentially,
01:23:26
Speaker
Your choices are things like doc strings or something like quarter pipe or go to the spectrum exists at the time. Um, so basically there were various tools for doing this. And for me, none of them kind of felt immediate enough. Um, meaning that I I'm typing some code and in my head at the time that I'm typing the code, I have some particularly invariant in mind. I know, for example, that as a trivial example, this is going to be a vector or more interestingly, let's say,
01:23:52
Speaker
The state is in a particular position. So if we're talking about real time web communications. These are the kinds of like edge cases. I'm getting message X.
01:24:00
Speaker
And my expectation is at the time that I've got the message X, either condition A or condition B are holding or something about the user state or something about the connection state or something about how these components relate. Again, it's a bit difficult to give example in abstract range. So obviously, closure had preconditions for a long time. So what kind of what what what didn't you like about that?
01:24:25
Speaker
Awesome question. So Closure has preconditions and it also has assertions. You can just do a search on any particular state. So there's a few things I dislike about both of those. Number one is that the error messages are useless. So if I have a function and I say the precondition is. Just tell it how it is, OK? Yeah, but I mean, it is the case. Preconditions don't solve this problem. Preconditions give you the ability to throw an error that's just as useless as the other errors that you would get.
01:24:53
Speaker
It will throw something like, what's a function over here with expecting a vector, but it actually got a set. Excuse me. You'll have a precondition like the input to the function is a vector. And it will say, OK, that didn't happen. The precondition failed. At that point, you've got no other information. You don't understand why it failed. You don't understand what the input was for the function. You don't even know what the data type was that wasn't a set or that wasn't a vector. So we're expecting it to be a vector. That wasn't the case. It's through an error. But you've got no other information to work over.
01:25:21
Speaker
And again, if that happens in production or even if that happens in development in a large complex program, you're going to end up spending a bunch of time trying to track down. Yes, the precondition has helped enforce an invariant that it hasn't helped you when that invariant is actually violated. It hasn't helped you very much actually debug or understand what was the state of the program that caused this particular invariant to be violated. So it will say woven a vector. Cool.
01:25:48
Speaker
If that happens in production in a function, and you're looking at the function, and you're looking at the consumers, and it looks like all the consumers are being pulled with a vector, how is this happening? Clearly, there's some sort of an edge case happening somewhere that somebody's calling into something that's not a vector. Is it nil? Is it a set? Is it a string? That would be helpful to debug, because if you know what is the invalid value that's getting into this thing, it gives you an enormous amount of context to try and get where is it actually coming into the system. When you know that the invariant's failing, that you don't know how or why, it's, again,
01:26:17
Speaker
When you're lucky, maybe you might have a hunch when you're not lucky or when you're under time pressure. And in particular, when it's a large program, it's not it's not very useful. So one of the primary motivations behind trust was just to fix this particular issue. And theoretically, this could be fixed and close your core. Actually, there's no reason this can already be done.
01:26:34
Speaker
which is when there's an inviolation, provide as much contextual information as possible, right? Provide line numbers, namespace, provide the forms that are going in, provide the values that are going in. So if, for example, I'm doing an invariant to say this value is a vector, right, and it triggers a warning, it triggers an error in trust. Trust will tell you, no, this failed. It will show the actual predicate function that failed. It will show you the input. It will show you the data type of the input.
01:27:02
Speaker
and will allow you to attach any arbitrary context to that error message. So it gives you a lot of information in the error. And again, this is all sexy stuff. It's not that complicated. It's just attaching a little bit more information. But even basic stuff, like just what's the value that doesn't satisfy the condition is just massive time waste when you don't have that. So providing this information just to things like, let's say, improving the preconditions or improving the post-conditions was already useful. But more importantly,
01:27:32
Speaker
Or let's say to kind of generalize this notion preconditions and post conditions are about the public API function, right? So it's about it's about what's coming in and what's going out. And that stuff is of course, very, very important. And arguably, it's sort of the most important thing. But as a programmer, again, when your programs get really bigger and bigger, especially ones that are logically complex, there are more
01:27:57
Speaker
There are more invariants that you have in mind than the particular API contractor functions right within the function body of a particular thing You may within the function body be making certain assumptions either about the state of the world or about different systems or even about the flow of a particular program For example, right? You're you're working with something and you've got a left binding and over here you're pulling whatever you're pulling something out of a set and three lines later and
01:28:20
Speaker
while you're writing the program, you know that there's no way that this value can be nil, right? Because you know logically you just put it in over there two lines earlier. So between here and here, like if you look at these 10 lines of code, there's no way that this value over here can be nil. Cool. But the thing is now you save this code and it goes into production and six months later, somebody else is working on this code and they're refactoring it. And because you're working in, it's not uncommon, some sort of a situation where let's say the features are changing very often, you've got a prototype and stuff is
01:28:50
Speaker
stuff is quite dynamic. Again, one of the benefits of closure is that the dynamic language allows you to kind of play with your ideas over time. So inevitably you do that. And somebody is refactoring this function or changing this function and they skim this thing or they're reading it and they miss the fact that actually this is supposed to be non, this can't be known at this point. And they end up passing it to another function where that's a problem. Basically,
01:29:19
Speaker
When you first wrote the function, you have an implicit assumption of some kind in your head that you thought was self evident while you were looking at the code that some time. Some, some time passes something changes and somebody else is looking for the refactor it and since it wasn't explicitly documented that this there's this particular invariant on the value.
01:29:37
Speaker
Somebody maybe makes a change now, which seems to work 90% of the time, but has introduced the possible edge case to something, because now when these three conditions hold some weird case that isn't covered by the unit test, there's now a nil value in here, which gets passed to the system, which throws an exception, which says null pointer exception with no other information. So anyway, what Truff allows you to do, and again, just purely dogfooding here with solving issues that I was having and that clients were having,
01:30:05
Speaker
I found it massively valuable to just be able to insert in line and say, I'm asserting in line at this particular point that this form is going to be non-null, or I'm asserting that this particular form is a vector, or I'm asserting that this particular value is bigger than that particular value, or whatever. And again, it's a bit difficult to give realistic examples.
01:30:29
Speaker
Um, in abstract because the place where I find this useful is in real applications, right? Where you have actual significant business logic of some kind. Um, and that doesn't convey well to kind of toy examples. It's a little bit difficult to articulate. Like a lot of people might hear this and think, I don't have a need for that. I've never seen a need for something like this. And that's, that would be completely true because they might not have had these kinds of situations where you have substantial business logic.
01:30:52
Speaker
inside the functions. But when you do, I found it invaluable to be able to make assertion on the state of your program as it's running inside at any particular point. So trust will always return when the invariant passes, it will return the input that you put in. So if you say this is a string and it's true, it just flows through and it gives you the string. So you can kind of just insert these things in line. And what I noticed is since I started doing that in my own programs and in client applications, our
01:31:22
Speaker
Our error rates dropped substantially down, and our time spent maintaining and debugging stuff actually went substantially down. Just this one small thing in a few little places here and there. Because I noticed how much, like disproportionately, how much time debugging was actually spent on these just stupid things. Like sometimes something could have just been codified as part of the code, some sort of an invariant.
01:31:51
Speaker
And if it's finally to just provide a really good error message with as much context as possible, that stuff sounds stupid. It's not sexy, but it saves enormous amounts of time, at least in my experience. And yeah, so this was way back when I started using this. Then after that, at some point, schema came along.
01:32:12
Speaker
Um, and people were using schema. I never published the early version of truck and people were using schema and finding use in schema. Then obviously not closure, uh, closure dot spec, um, has come up. And I think that that's also really interesting. And I think in general, I would sort of encourage people to look in that direction. Um, it's not entirely obvious to me yet that, um,
01:32:35
Speaker
that closure.spec has completely obviated the need, at least for me, or the value of something like trust. I will tend to use trust myself. I use it for different reasons, right? Like spec, I find really good for defining like data structures and API level contracts. And in particular, I love the, I really appreciate the
01:32:59
Speaker
the fact that you kind of get the leverage, the syntax to do several things, including things like the generative, the generating and the docstrings and it's wonderful leverage. And I think it was a really good, a good design and a good idea for core language thing. Like if you were going to design something like this, uh, for the core of the language, I think it was, it was very nicely done and it offers a really, really good, good balance of functionality for, for, um,
01:33:23
Speaker
for the cost as it were. So still the error messages are not ideal. I think this is something that can be fixed, but also people keep saying that it's going to get fixed and ultimately needs to get fixed at some point. And also I never found it quite as comfortable for me for inline assertions, which I find is sort of a bit of a sweet spot for trust. So like trust is a bunch of little things for like the structuring. If you have several, I want to, for example, I'm going to say,
01:33:51
Speaker
I have these values, and they're non-nil. And then you provide three or four different symbols. And if they all pass, it gives you back the thing in a vector. So you can just de-structure them. So there's little syntactic things like this, which, again, syntax is not the most important thing in general. I rarely find it sort of an interesting argument to say, language x is better because the syntax is nicer. Syntax doesn't really matter that much. But in this particular case, I actually find that syntax is important because
01:34:20
Speaker
Arguably, people could just be documenting things, right? And they could just be putting comments in their code. But the trouble with comments is because it's a comment, it's not being run by the compiler. It's not being checked. It tends to go stale. People forget to update. The people are also lazy about it. So what I found very useful with the trust in text was it was terse enough and expressive enough that, in my experience, people would actually use it. And I used it. And people that I worked with actually used it. And they enjoyed using it.
01:34:52
Speaker
I guess they found that the overall net expense of annotating a couple forms here and there was more than easily paid for by the advantages and so they were inclined to do it and they did it happily and it wasn't something that you kind of have to force on people.
01:35:09
Speaker
That's a really long answer. I think the nice thing about it, though, Peter, I mean, is the is that again, it's like conceptual overhead is quite small. You know, you might have something and then you're pretty much done, you know, so you're not buying into a very big API or, you know,
01:35:26
Speaker
and also like a really good DSL it blends perfectly well in and compose as well with all the other predicates which is kind of the thing about spec as well which is you know it's one of the advantages of spec is you just use language predicates and that's one of the advantages of truss as well.
01:35:41
Speaker
So you saw that and I think that's really a great insight because doing something where you have to essentially replicate all of the core predicates
01:35:58
Speaker
like Core Type did, then you're kind of, you know, it's, you're on a losing horse, I think, because it's a different language at that point. Whereas what you're doing is you're just saying, okay, well, there's just one very small thing that can help you to assert stuff. And here it is, which I think is really, I agree. I mean, I think it's a really nice, useful library with a very low
01:36:23
Speaker
conceptual overhead, but quite a lot of payback. Maybe it's not as much leverage as spec, but then the overhead for using it is also incredibly low. Yeah. So in general, you'll notice a theme in a lot of my thinking and a lot of my libraries as well, which is just I tend to approach things
01:36:44
Speaker
Certainly with the published open source, just from a practical point of view, right? Because in general, my interest is not actually in the programming. My interest isn't in making nice tools or hacking on the stuff because I find it enjoyable. I do find it enjoyable, but ultimately I do stuff because I have a project that I want to build, or I'm trying to get something else done as quickly as possible. And the way that I can get back to that product as quickly as possible is kind of my motivation. So for a lot of these things, like with trust,
01:37:10
Speaker
I found a solution that worked for me that let me decrease the time spent in maintenance and the time spent in debugging. And what I can say is ultimately in my case, purely subjectively, but after using something like trust and kind of getting the grips of where sort of developing the static for where to use it, the problems that I had with closure being dynamically typed completely went away.
01:37:34
Speaker
I no longer miss my static typing from closure. And in fact, I actually now feel pretty satisfied at the overall balance that I get between the dynamic typing and the ability to express invariance. And in particular, I like the fact that invariance are a more general concept than just piping, right? Because piping, again, the easiest example is to sort of share our piping, to say this is a vector, this is a string. But those aren't actually the most interesting things I think in practice.
01:38:04
Speaker
The kinds of invariants that I find useful in production are, again, business logic things. And again, providing information to say, all right, this
01:38:13
Speaker
you know, this statistical function, whatever this is supposed to be bigger than this under these, I'm expressing that as an invariant and then attaching, attaching debug information for when that invariant doesn't hold. It's just, I really cannot overstate how much time this has saved me. It's saved so much time to be able to do this. And again, you often, I often never needed it in many places, but in the few places where it's valuable, that the value was just outstanding.
01:38:36
Speaker
Composing predicates like this is so easy compared to the higher kind of typing system that you would need to do a similar thing in a more type-oriented language. You can just express a bunch of predicates in
01:38:56
Speaker
Not even one line is aligned, but it's a short line. Yeah, it's really nice. The best thing with this one is that you can just use as much as you want. It's not forcing the entire program to be following certain design or following certain types.
Clojure's Flexibility and Conclusion
01:39:14
Speaker
So you can just put in, OK, here, this is the most critical part of my code. So I'm going to put in something here that is going to get maximum value out of that one instead of doing a large upfront design.
01:39:24
Speaker
And obviously the thing is you can have spec and you can have trust. You know, the two things are not incompatible. In fact, they're completely composable. They're completely, what's the phrase? Yeah, they can coexist happily. Yeah. And to be clear, absolutely. I think that trust would be a terrible spec. Trust doesn't do what spec does. If you're talking about data structures and things like this, spec is great. And you wouldn't use trust for that.
01:39:55
Speaker
So I think actually for me, one of the nice things, the releases spec helped clarify for me actually what are the boundaries of what I enjoy about trust and where, I guess it helped clarify the boundaries a little bit anyway for what I was actually looking for from it because I noticed in myself, where was I still reaching for it versus other tools. But ultimately something that I really like about Closure,
01:40:19
Speaker
Um, in general, like we were talking earlier on about, you know, it's a list that's cool, but the things about closure also extend a lot. There's, there's more nuances about closure that I think make it really attractive. And one of the things that I really appreciate about closure in general is that it tends to be, uh, a craftsman's language, right? It's practical. Um, it's mostly functional, but it doesn't force you to be functional in all cases. It's mostly immutable, but it acknowledges that sometimes you want to be able to mutate things. It's, you know, it, it.
01:40:46
Speaker
It strikes a practical balance, recognizing that people sometimes need to get stuff done. And we're just trying to achieve a mission here. And the way that I would put that is Jonathan Blow has a nice way of putting this, which is, so he's developing a programming language at the moment, which is really, really interesting. And I'd encourage anyone who's interested in sort of low level languages to check it out. Jonathan Blow is working on a language called Jai, J-A-I. The most interesting programming language product
01:41:14
Speaker
project going on at the moment, in my opinion. Anyway, one of the things that he's doing with his language that I also appreciate is it's not a big idea of language, as he would put it, right? It's not trying to make it impossible to throw, you know, to have a null pointer exception. It's not making it impossible because these super sophisticated pipe systems, right?
01:41:35
Speaker
Like I found it interesting when Rich released transducers and everybody was trying to sort of provide the pipe signature, right? For transducers. And like, I think that it's also possible to go too far. And again, that just is purely subjective. I don't think there's a right or wrong. It's a matter of taste and it's a matter of what you're trying to do. But for me, my particular taste is I'm interested in stuff that gets the job done. And for me, like a full on pipe system,
01:42:04
Speaker
would take away a lot of the joy that I feel in Clojure. And I think actually a lot of the benefit that I get from it is being a dynamically typed language. And I get a lot of leverage from the fact that it's dynamically typed. And again, it's for these reasons that I've done a lot of work in like prototyping in Clojure because it's awesome. But then because also it's a list and you can write macros, you can plug on something like this where you get to do a little bit of just reinforcement at the edges that you care about when you need. And I find that that's a really nice balance.
01:42:35
Speaker
Okay cool I think we are of course I mean there are there are lots of things to talk about obviously but you know we can we should probably make it a two-parter at some point you know we need to come back and then discuss most things and I'm really curious about I mean this has been fantastic I mean we discussed a lot of
01:42:56
Speaker
things. So I think we can call it a wrap. Peter, thanks a lot for joining us and, you know, giving us your time and describing a lot of awesome things that you're working on. I learned a lot of things today. And of course, I'll take a look at the JI language as well. I've never heard of it. So I'm really curious about, you know, I'm trying to follow, like, learn one new language every year sort of shit for some time.
01:43:25
Speaker
So my Rust is a bit rusty right now. So I'm on Rust now. So hopefully next year, I'll pick up something else. But anyway, again, thanks a lot for all of your time. And of course, fantastic libraries. For the people who are listening out there, maybe you're already using his libraries. So go to his website and click on the back button. Not the back button, but
01:43:52
Speaker
Back button. Don't use them. Don't use the library. Don't be angry with me if they're not working correctly. I'm not responsible. Go to go to the website and do not click the back button, but click back me up or how do you call this in English? I mean, Patreon. Yeah, not Patreon. Support me. Yes. Become a backer. Support me.
01:44:13
Speaker
Exactly. Become a backer of his amazing work. Or the back button. That's fine too. Exactly. So I think that's it from us today. Again, it was fantastic to have you online today.
01:44:30
Speaker
Thank you very much. And thank you for being patient with my rambling opinion. No, no, no. No, we love it. This is exactly why we do it. Yeah. Yeah. Awesome. Thank you both. So a quick shout out to the things.
01:44:46
Speaker
the things yes the regular shit i mean this is the weird part i mean the show i keep i keep forgetting what i'm supposed to do um uh all the music coming from pizzeri he's uh p t z e r y um on on soundcloud uh go and uh like his music i'm not sure if he is making any new tracks or anything but um i keep listening
01:45:10
Speaker
And all the hard work of making us sound better is being done by Wouterdallert from Belgium. He's been doing awesome work and helping us a lot. And of course, we are also on Patreon, so if you want us to retire sooner and then buy our own yacht and ship and go there and give us your money, and then we'll put your name on our yacht. Yeah, that would be good. Is that a problem?
01:45:35
Speaker
I'm going to put that on my sponsored buy or at least a paperboard at some point. That's it from us. This is episode number 36 with Peter Tosanis. Thanks a lot. Thank you, Vijay. Thank you, Ray. Absolute pleasure.
01:46:17
Speaker
you