Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#2 - The Reader
107 Plays9 years ago
Defn Episode #02 - The Reader
[ a little news: spec ]
The wonders of the Reader
- Compare to the C pre-processor
- How it underpins the language
- Reader forms
- EDN / Transit
- Tagged literals
- Reader Conditionals (1.7+)
- clojure.tools.reader
- Finally, why no user defined reader macros (as in Common Lisp and Scheme)?
References
The Data-Reader's Guide to the Galaxy (Clojure West, 2013) by Steve Miner
Spyscope by David Greenberg
More information and links at the DEFN website
https://defn.audio/2016/05/30/episode-02-the-reader/
Our thanks to Ptzery for the intro and outro music (Melon Hamburger)
https://soundcloud.com/ptzery/
Recommended
Transcript
Introduction and Audience Feedback
00:00:21
Speaker
Okay, let's begin. All right, okay, so Daffan, episode two, welcome. Hello, this is Vijay. Hi, this is Ray. So apparently we forgot to, at least Ray forgot to mention his name, so I think you should introduce yourself, Ray, again.
00:00:41
Speaker
Okay, yes, that was a bit, I was too excited last week, last time. I was saying last week, I don't know what it is. Anyway, I'll get my name out now. Ray McDermott here in Belgium, Vijay Karan over there in Holland. Definite episode two. So it's been interesting two weeks, hasn't it, Vijay?
00:01:04
Speaker
Yeah, I think it's been very exciting. I mean, I never expected this amount of reaction from the listeners. I mean, it's amazing. It is phenomenal. Yeah. I can't believe how much attention the podcast has got, to be honest.
00:01:22
Speaker
Yeah, I think we had more than 1,000 listeners on Soundcloud or something. Yeah, it was well over 1,000, which is incredible. And lots and lots of positive feedback on Reddit and Twitter, and really, really good. I mean, it shows there's a hunger for information out there in the closure world. And if we can feed it a little bit, then that's great.
00:01:46
Speaker
Yeah, that's the whole goal of our show anyway. So I've been very humbled by a lot of nice feedback we've been getting. And of course, there was a little bit of a technical hiccup in the last recording, but I think we have fixed it now. So the sound quality should be much, much better from this episode.
Technical Improvements and New Spec Discussion
00:02:06
Speaker
Yes. OK. Fantastic. Well, big week this week, eh?
00:02:12
Speaker
Yeah, I think the amazing spec has been released, which is fantastic news for Closure. And because I think most of the guys have been settled on schema one way or the other, but it is very nice to see Richiki taking more hammock time to think about this stuff and then come back with spec into the core. So that's pretty amazing.
00:02:35
Speaker
Yeah, I agree. I mean, what's, by the way, just out of interest, you know, news never happens in Closure. You know, we get announcements so rarely that it just happens that after the first episode, he chooses to release Speck. So, you know, good timing, Rich. Of course, yeah. I'm pretty sure he's listening to this and laughing at Claude all the time.
00:02:58
Speaker
So, I hope, I mean, Rich, if you're listening to this, I'd be happy to have you on the show and tell us that we are, yeah. My headphones just fell off at that point.
00:03:14
Speaker
Okay, I apologize for starting with big jokes. But yeah, I mean, spec has been pretty decent. I mean, this is one of the nicest things, at least if you see schema, but taking that into next level, which is awesome. I think we should do a complete episode on spec and what it is doing. That does settle a little bit on it because I noticed by the way that Rich actually is still in the
00:03:36
Speaker
in the Slack channels and this kind of stuff on the emailing lists, slapping people about using it for parsing text and shit like this. So yeah, the dust is still settling, that's for sure. But let's just have a quick, I mean, you mentioned there about the prismatic stuff, because that has been absolutely a phenomenal thing.
00:03:58
Speaker
We've had a lot of this stuff from Ambrose, the gradual typing stuff, the core type. I think that's been an interesting experiment, but the thing which is really practical, the thing which really the community has got behind is the prismatic. I think it's now plu-matic actually. Yeah, I think it's plu-matic, yeah, true.
00:04:18
Speaker
And I think that's actually, we've got a lot of mileage out of that, so I think kudos to those guys. And I don't think people will drop it straight away. I think that people will still be interested in it. Still got a lot of mileage in it. Still got a lot of value. But he did do some interesting things this time with Speck. He made some interesting reflections, didn't he? Against Prismatic, where I think one of the biggest differences is you don't have to provide the typings for the keys.
00:04:45
Speaker
Yeah, that's true. So there is much more decoupling in terms of what you can specify. And also, I mean, the other nicest thing that I saw is that you get this generative testing for free. That was most exciting part for me. Absolutely. Absolutely incredible. Maybe we should just quickly say what spec is, though, just because I didn't think we did that yet. Oh, yeah. I think it's better to give a quick overview of spec.
00:05:12
Speaker
So spec is essentially something like what we call occurrence typing. So this is because closure has been always what you call like an optional typing friendly language because everybody wants to get into adding types later than specifying types upfront.
00:05:34
Speaker
And if you see, and also it was very difficult to provide or validate the data schema, so to speak. And that's the reason why the name of the library was also schema. And of course you can write documentation and you can specify what it means.
00:05:51
Speaker
it's not good enough. So spec is essentially going to give you a couple of things like how to validate the data and destructuring and you can instrument the stuff that you have and you can have generative test. I think automatically generating generative tests and also error reporting. So you should have all these features using a library like spec.
00:06:20
Speaker
I think that was one of the things to me when we looked at things like core-typed and prismatic schema and the generative testing tools. They looked so close to each other. This is one of those frustrating things actually.
00:06:37
Speaker
was that, you know, they didn't, they were always kind of, they didn't play well in the ecosystem with each other. We're kind of stepping onto this toes. That's the nice thing about this stuff is that, you know, it does harmonize a lot of those, harmonize a lot of those thoughts, you know, makes it practical as well. So I think that's going to be a huge win, I think, for the community.
00:07:02
Speaker
Of course, and also moving it to one of the core libraries makes it makes it available to everybody. So that is much more. I don't want to call it a blessed library, but at least, you know, there is a standard now. People, everybody can use in the community and that everybody can reap benefits out of it. So that's pretty awesome. But I think it's a blessed library, isn't it? I don't think anything that comes out of Cognitec tends to be. That's true.
00:07:27
Speaker
you know, tends to be blessed. I mean, I think there's a few things that are a bit, they're a bit, let's say, you know, un-blessed, like pedestal, perhaps, you know, that's kind of like, it's, yeah, no one's hating on it, but it's, it's not one of the, it's not a core thing where this really is, you know, a core thing, like chorusing and that kind of stuff, isn't it? You know, this is, this is definitely there as something which, you know, Rich and his, uh, and his core team are kind of standing behind.
00:07:56
Speaker
Yeah. But anyway, I think we should move on to the main discussion for this episode. We should certainly do an episode about spec. I mean, we'll discuss more about spec, I think, in the future episode. I think also, please try and schedule your announcements around our show timing. That would be good. Yeah.
00:08:16
Speaker
I think that's our humble request to the closure community. So we'd have a lot of stuff to talk about, obviously. And we'd like to take the false credit of, hey, we poked them to do this thing or something like that.
Exploration of Closure Reader and Its Functions
00:08:32
Speaker
Yeah. But anyway, so to the main discussion for tonight, so we promised the internet that we are going to do an episode about reader this time. So yeah, I think maybe just to back up a little bit on what you were saying there is that I think a lot of the feedback that we got from the first episode was, yes, OK, it's good stuff. It's nice start. And obviously, I think people will try and try again, hopefully, and give us a second chance to stay with us.
00:09:01
Speaker
This is definitely going to be a more technical episode. We're going to be going into stuff which is a bit deeper. Let's try and get a balance between, we're obviously always trying to get a balance between stuff which is good for people that know closure and people that don't know closure at all.
00:09:21
Speaker
So we'll see how we get on, because we've got a list of things to go through on this reader. And some of it's a bit archaic and a bit technical. So hopefully we can keep it accessible. But yeah, like you said, let's crack on. Yeah. So what is the reader? So why reader is so important in Clojure and Lisp? Well, maybe we should start with that part. Yeah. So in my mind, so if you see the compilation phases for any given programming language,
00:09:51
Speaker
You have multiple steps, multiple phases in the compilation. You have the first step as lexical analysis or there is a lexer that is going to... So you're writing code and code is essentially stream of characters. You're typing in with your keyboard frantically and then you save it to a file. That is basically... Do you have to do it frantically?
00:10:10
Speaker
I think so. OK, just checking. Because that's the nice part, right? You spend enough time in the hammock, and then the only delay is your typing speed now. So you have to type very quickly to move your thoughts onto the files. I just went off the topic completely there. Sorry, yes. Oh, that's OK. Yeah.
00:10:36
Speaker
So you have these characters and those are in your files and your program is there and then use lexer and then lexer is the one in the compiler phases or compilation phases that produces what we call tokens and then you have a parser that is going to do some sort of a syntax tree or a parse tree that is going to generate a proper syntax out of it and I think that's called parse tree or something.
00:11:00
Speaker
And then you have the semantic analysis, which is the analysis phase, and you get the AST. And from the AST, the final step is usually either generating the machine code, or in the case of JVM languages, maybe a byte code, or in the case of JavaScript, it's JavaScript. So you have these different phases in the compilation. And an interesting factoid is that the latest Scala compiler has 25 phases. That's fun. 25 phases.
00:11:29
Speaker
Yeah, I mean, I don't want to poke fun at Scarlet Guys, but you know, I mean, it's a fairly complex language, so you need all these phases to generate these things. But coming back to... Yeah, but actually, to your point there, to your point there, that the Scarlet Guys have added, like, some kind of macro, fairly naive macro implementation.
00:11:54
Speaker
Exactly. To their language. So I guess that adds a lot. And obviously they've got all the functional stuff, the O stuff. It's a pretty complicated language. That's true. Yeah. So it is a complex language. Ambicious project, that's for sure. Exactly. So it's a complex beast. So obviously their compiler is much more, it's not that easy. And
00:12:12
Speaker
And coming back to closure and lisp, I mean, essentially, reader, you can treat reader as the combination of lexer and parser because those two steps are essentially the same in closure. So, and that is, I think, as you were pointing out when we were discussing before the show, it's essentially the literally the UI of the language. So, that's where the first part of the compilation step.
00:12:42
Speaker
It also validates. Sorry, go on. Yeah, sorry, sorry. Go ahead.
00:12:46
Speaker
I was going to say, the other interesting thing about the reader is that it does have uses on its own. And we'll see a bit later in our conversation, won't we, about like the Eden and stuff like this, which, you know, so I mean, I know that some people use preprocessors or JSPs or these other type of things as kind of like tools on their own, but yeah,
00:13:14
Speaker
But they're also less powerful, I think, typically than these readers are. So I think what's nice about it, I guess, is at the very beginning of the language, even in the actual tools to understand the language, you have decoupling.
00:13:29
Speaker
between read and eval and print and then loop. So the reader is in itself a tool that has some usage outside of just purely the REPL.
00:13:45
Speaker
Yeah, that's true. So because the reader is one of the, I mean, I think we'll probably discuss a bit what reader provides in terms of those things. Yeah, sure. So that gives us much, much understanding. I think you were pointing out that reader is more like a, you can maybe compare it to the C preprocessor, for example.
00:14:06
Speaker
especially when you have the branching in the code. So with the CLJC things, do you want to explain the comparison a bit? Well, my thinking was actually to compare it negatively. Yeah, of course. Well, mostly the interesting thing about, let's say, the reader is that it's not dumb, whereas the C preprocessor is a pretty dumb
00:14:35
Speaker
just a kind of thing that runs over your file and just cuts and paste symbols so you hash define x to be abc123 and then whenever it finds hash x in any of your C files it just replaces it with abc123
00:14:53
Speaker
And just cutting and pasting on a mass is not the same kind of macros that lisps have. Yeah, of course. I think as you're pointing out, it's essentially operating at the lexical level and is just moving some text from here to there. Yeah, that's what it is essentially doing.
00:15:11
Speaker
Yeah, the other interesting thing is that the C preprocessor doesn't understand C at all. It doesn't need to understand C. There's no bearing to C at all. It's its own little mini-language. Whereas the reader is part of Lisp and it's a complement to the language as well.
00:15:33
Speaker
So, I mean, obviously, yeah, I mean, the kind of if-def things where you want to hide or show certain bits of code, there is a little bit of that with this cross-language stuff between reader conditionals, between closure and closure script, and we'll come on to a bit more detail about that later.
00:15:56
Speaker
So the nicest part is that at every point you're producing in all these compilation phases where you have reader, you have analysis, you have micro expansion, at every point all you're getting is closure data structures.
00:16:12
Speaker
So you can operate on them. That's the whole definition of having access to closure forms everywhere with the closure library. So you can use the language to manipulate how the language is getting compiled. So that's the homo-iconicity that we have. So which is pretty interesting in terms of how readers operate.
00:16:35
Speaker
And I think if we see what reader is doing, there are some fundamental forms that reader can understand. There are symbols, literals. So if you type a form, form is essentially an S expression. So anything that begins with a parenthesis, for example, and the reader understands it and then converts them into
00:16:58
Speaker
Lisp AST or code and code AST, which is essentially Lisp again. And we have different kinds of reader forms. I'm not sure if we can go through all the reader macro characters though, but maybe we should explain a bit what they do. So there are some specific characters in the language that are picked up by the reader and interpreted differently at the reading phase. So I think it would be horrible if I keep pronouncing these symbols though.
00:17:28
Speaker
Yeah, but there's some funky things aren't there like when you first start doing it in the in the repel You know if you if you just type I think if we just type hello world for example as a string then you'll get back hello world as a string in the repel and that's just because well it recognizes that as a literal, you know, exactly and if you type in I don't know something like
00:17:55
Speaker
Plus and then you have one two three. Well, it recognizes that as an s-expression That's all good. And then you can like like you say you can do quotes and you put a quote in front of a list Yeah, then that list is no longer evaluated as a list. Yeah, but it's but it's actually taken as literal exactly so there's some there's some nice interesting things there in this in these macro characters, hmm
00:18:22
Speaker
Yeah, so one of the interesting at least a couple of things that are that stand out for me is the metadata thing. So you can attach additional metadata to a given symbol or given. Yeah, I think essentially a symbol you can attach metadata you can specify. Well, you get some typing hints there, don't you actually, because that's the
00:18:44
Speaker
That's the other thing. And actually bizarrely, I was looking through some code today and I'm ashamed to admit that I didn't, I only discovered today how you make a deaf private and that's through this method. Yeah, that's true. I'm sorry everybody. We're still learning, so that's fair enough.
00:19:06
Speaker
But I think one time I used the metadata or the type hints or I was writing a small numerical library and instead of using, you know, just to prevent reflection essentially, you know, I specified Java long or whatever for the numbers and that was much more useful because I'm also making some constants, you know, those kind of things. It's very useful.
00:19:28
Speaker
And the other stuff that stands out is the syntax code and the uncode and uncode splicing that enable you to write a lot of macros in a cleaner way. So you start with the literal but in the body you want to evaluate again and then in the body you want to go back to the literals again. So you can do all this macrology with these reader macro characters.
00:19:53
Speaker
I think we'll probably, in the course of this podcast life, we'll definitely have to talk a lot more details about macros one of these days. I know that it's one of these things which is, let's say, underplayed enclosure, or downplayed, let's say, rather underplayed, because obviously there's a shitload of macros enclosure, but it's downplayed because people would have data first, then functions, then macros.
00:20:22
Speaker
but I think it's important enough for us eventually to have a longer conversation about them. Yeah, certainly, yeah. I was just gonna say, one of the things about this, all these like inks and lungs and stuff like that though, I don't know about you, but I find it a little bit ugly. I find it a little bit kind of anti-closure. It starts to look a bit yucky. I know you get better performance out of it and that kind of stuff, so I see why people do it, of course, and you have to do it, but it does start to feel like,
00:20:52
Speaker
Well, what the hell? You know, we came here to get away from these types, you know, not exactly. But yeah, you have to put these friggin types everywhere. They look horrible. Yeah, which is why I like spec, actually, because.
00:21:08
Speaker
In the end, you can do both, of course. I think one of the major use cases for specifying the types is to prevent reflection. I think that's what I would think. Because I know every time people talk about closure, then it essentially becomes a war between static typing or optional typing.
00:21:28
Speaker
Yeah. I mean, that's something that I don't want to get into, but you know, sometimes you want to have... We're in there, mate. We're in there. We're in that war. Maybe we should... We've taken a side. It's okay. Exactly. I mean, obviously with the spec thing that, you know, we're...
00:21:44
Speaker
We're offering a little token towards the typing people. That's true. That's true. But we can take that when we want it. That's really good. But we're definitely, we've taken a side. Come on. Sure, sure. I mean, I picked the right tool for the right stuff. So I don't want to be too diplomatic, but I prefer Clojure because it doesn't force me to do things like specifying the types all the time.
00:22:11
Speaker
that's not something, most of the problems that I'm working on are essentially data manipulation. So I don't want to spend additional brain cycles on first thinking about the types and then dealing with the data because in the end I'm throwing the stuff into JSON and what types do you have there? Essentially none. Come on, start in a minute actually because we're just, since we talked about JSON, we should maybe talk about Eden.
Format Comparisons: Eden, JSON, and Transit
00:22:35
Speaker
Yeah, that's true. Because that's also a part of this reader thing, you know.
00:22:40
Speaker
Yeah, so I think we should explain what Eden is. So Eden is essentially EDN, extensible data annotation. So that is one of the underlying. Yeah, I think the background of that is that I think the background of that isn't it is that I think it was the criticism of Jason basically is that it's not extensible. That's really the kind of motivating
00:23:06
Speaker
Force if you like on motivating idea behind Eden and transit and all the kind of closure data formats is You know, we have to keep the the the data format extensible We don't want the horror of XML back, but we want the advantage of XML, which is the extensibility. I
00:23:26
Speaker
But there is an interesting thought experiment. I don't want to call it a thought experiment. But some people claim that XML is essentially lisp in angular brackets. Or maybe I should say XSLT, not XML, but XSLT. It has the similar notions. But I think it's too spiky with the square, sorry, the angle brackets. I think parentheses are much more smoother to look at. I don't know. It's my personal preference.
00:23:53
Speaker
Yeah, you don't have to say, you know, open brackets plus and then have a closed bracket slash plus at the end. Yeah. Yeah, that's that's horrible. Yes. Mental. Yeah. Yeah. But maybe I think I have some some more things to say when when when we want to say something about the reader macros, though, but I'll hold the thought for for until we get there. But we should we should get into Eden a bit. All right. OK, so so yeah. So the main thing is it's it's extensible. That's that's the main point, isn't it?
00:24:20
Speaker
Exactly. And also, it's the essence of Closure syntax. I mean, it's the basis of Closure. So we keep saying, hey, code is data, is code, is data, is code, and recursively, so on, so on. So Eden is essentially the fundamental syntax of Closure. So you have all the list and maps and whatever that you are used to in storing the data in Closure. All those literal formats are in Eden.
00:24:47
Speaker
So, I remember that there was this training by Stuart Halloway and Neil Ford, I think. Oh yeah, yeah, yeah, I've seen that, yeah, on the ORE stuff. Yeah, exactly. I think that they start the course with Eden, not with closure, but with Eden. So, they first explain what Eden is, because that is what closure is, essentially, in terms of how it looks like, and then they move to closure.
00:25:09
Speaker
I'd forgotten actually, yeah, but you're right. Yeah. Yeah. That was a long time ago. Yeah. Yeah. But we had a, I'm not sure if I can say this online, but what the hell.
00:25:23
Speaker
Because as long as it's not one of these kind of gawkers things where I'm going to get sued out of existence. I hope what I leave guys are not going to get pissed off with me. But we had a viewing party of this episodes actually. I mean, we spent like whole day and there were around 10 guys and 10 folks sitting together and then watching the whole closure thing. The whole training by Stuart Holloway and Neil Ford in
00:25:50
Speaker
I don't want to specify the place just to be sure. Exactly, yeah. But that's a hell of a party though, yeah. Okay, anyway, right, back to Eden. Yeah. Yeah, so yeah, you're right, anyway, blah blah blah. So it's all the kind of the formats that you like, like you say, the lists and all that kind of stuff.
00:26:10
Speaker
Yeah, but it didn't pick up, right? I mean, Eden was too closure-specific. And of course, you can have Eden as an interchange format between different languages. That means you had to have Eden parsers in other languages. So that was one of the painful things with Eden. It's a perfectly suitable format. If you have front-end and closure script and back-end and closure, obviously, then it's easy to just shuffle Eden around.
00:26:38
Speaker
but that wasn't the nicest format to be used.
00:26:46
Speaker
Yeah, I think also, I think part of the sort of hampering of the adoption was that people were saying, OK, well, I like the fact that you've got this extensible data data notation. That's all good. But I'll tell you what, if I just put it out into JSON and I read it into my front end browser, it goes about a million times faster than Eden does. Yeah. And I know that was that that hurt, I think.
00:27:11
Speaker
Yeah, that's true. And then we have transit, which is essentially kind of a better format and better transitory properties. Is that a word? I don't know. I don't know.
00:27:26
Speaker
Okay, maybe I should explain what it means in my brain. So you can interoperate with other languages easily. So you can send the format because it can be passed by JSON easily, JSON engines or JavaScript engines. And it's much more compact. I think it is binary.
00:27:45
Speaker
It's much easier to send over the wire and pass it and use it but still I think there are some projects I know that are that were using transit on github, but I didn't see enough Enough things that are getting picked up by by the community a lot
00:28:00
Speaker
Well, again, I think it's one of these things which is not seeing a huge adoption out in the wild. But obviously, I think if you look at the kind of omniX stuff, then I think they're using it as a means to transfer data backwards and forwards.
00:28:17
Speaker
Also, if I remember rightly, the thing about transit is that you can, if it's seen a particular value or a particular key, then it can cache it. So you can speed up quite a lot that way, which actually makes it faster than some JSON engines as well. True, true.
00:28:36
Speaker
So I think we should connect it back why we are talking about Eden because Eden is the stuff that is read by a closure reader. So that is the idea behind having Eden or talking about Eden. That's why I was saying at the beginning that it's one of these things where Eden shows that the reader has got more value than just in the raffle. Yeah, yeah.
00:28:59
Speaker
And obviously, even the whole extensibility is coming from the tag literals. Right literals are essentially the things that in syntax wise, it will basically hash and and a word.
00:29:12
Speaker
And there are some default or a built-in tag literals that you can use. So if you put a literal string for date, for example, then you can use instance tag literal, and that will be read as Java util date, I think, at least in closure. And then CLJS, probably it will become JavaScript date, I think. And you can also have your own custom tag literals. They're called data readers.
00:29:39
Speaker
Yes, it's a thing called again. I think it's called ear muffs. Oh, yeah, the star star plus star star star Yeah, so you can you can have your own data readers and that is the extensibility comes in that is where extensibility comes in So you can you can define your own name spaced tag literals? For example, if I have a map with a person and I can put a tag in front of it and call it I don't know my name space slash person and then put a literal representation of the map and
00:30:08
Speaker
my reader for or the data reader for that map will become will will convert that into a person object or person struct or Record for example. Yeah, these are quite powerful things actually because that's you know for certain I mean I think I don't know I mean I like UID and UUID and time and stuff like that I'm a bit negative on the whole kind of
00:30:33
Speaker
person stuff. I like URLs and, you know, kind of like things which are fairly generic in the, actually I don't know. I know I saw a video of Steve Miner from a few years ago in Closure West talking about his
00:30:52
Speaker
tagged library, which allows to do printing of deaf records, which is interesting, you know, so you can do that at the REPL. But the other thing that I don't know actually, maybe as you know, we had talked about this before actually, but I don't know if there's a kind of library out in the wild there of these generic things like hash URL or hash
00:31:17
Speaker
like hash. I'm struggling now. I'm wondering because in general, hash things. You can definitely have a hash thing. Now I'm trying to think of some generic and generic types of things like obviously UID and time is one, URL is another I can think of immediately.
00:31:41
Speaker
IP address, let's say. There are certain, or telephone numbers, or emails. Yeah, I'm on my floor now. I can do it now. So things like that, like hash email would be a good one. Or hash postcode or something like that. You can have domain specific things like the bank account numbers, you know, IBAN numbers or something. Right, right, right. But my question is, do we have those? I mean, I don't know, maybe it's, actually this could be a call out for the listeners actually.
00:32:10
Speaker
if they know of such libraries, because maybe this is just another form of lazy web here. Yeah, exactly. I'd quite like to know about them. Yeah, and also how pervasive is creating your own tag literals, because one of the suggestions was that, hey, every time you use a tag literal, don't pollute the top level namespace, but introduce tag literals for your own namespace.
00:32:34
Speaker
So these things look a bit like reader macros though, right? I mean, at least, you know, compensating for... It's very limited form, obviously. Very limited form. Exactly. But we'll get into reader macros later. But the tag literals are interesting and I think that is one of those things that give Eden the whole extensibility.
00:32:57
Speaker
I think. So anonymous functions, for example, and the metadata probably as we talked about and the DREF thing whenever they want to DREF and atoms.
Tag Literals and Security Concerns
00:33:08
Speaker
And whenever you're commenting, you're essentially talking to reader in a sense. You're skipping away things. And I think we should talk a bit about the security related issues with reading stuff.
00:33:24
Speaker
Because reader is the one that provides the read function, right? Of course, yes, that's right. But actually, I guess there's two parts to reading, isn't there? Obviously, if you're going to read your code, then that in the end causes eval, doesn't it? So you have to make sure that you're reading from a trusted source.
00:33:44
Speaker
There is a version of read from Eden where you're not allowed to do these evals. So that was a worry when I think it was in 1.5 when it first started where there was a security risk at the beginning. That's true. But that's closed down now.
00:34:01
Speaker
Yeah, but now everybody should use the Eden reader instead of the co-reader, I think. Yes, exactly. It's not discouraged a lot. Yeah. But there is, is this the same? You said there was a talk about Steve Milner. That was the tagged library thing. Okay, yeah. That was in the closure. The closure west. Closure west from a few years ago, yeah.
00:34:26
Speaker
I did see another one by this guy called David Greenberg which was definitely a very wacky kind of way of using or it was to my view it was a bit wacky view of using read tag literals which was this spy scope so you could it was I think was hash it was a hash spy and then slash trace or hash spy and
00:34:51
Speaker
Tag and stuff like this and the idea was that in your apple you could essentially put a little watch on on variables or forms inside of your code so you could embed these these reader these tagged literals inside of your code and of course they would be they would get expanded by his functions So they could be printed out and all these kind of things so it was it was a kind of limited form of debugging
00:35:22
Speaker
Yeah, that's true. I think it's it's I don't want to call it. Well, this is a interesting or quote unquote interesting use of tag literals, I think, because when you pointed out, I was looking at the code. And so it's essentially providing a couple of data readers. So data readers are something that you that is essentially a small map that says, OK, if I see this tag literal, then this is the function that should be called.
00:35:50
Speaker
Yes, yes. So it's basically a mapping between these things. So if you say hash person and then there is a, or maybe I should say hash namespace person or something, then you should invoke a person constructing function, for example. So the spy scope looks very interesting to me, at least for debugging, you know, what is happening in the system.
00:36:12
Speaker
maybe there's not the original intention of the tag literals but still i mean it's a innovative use certainly yeah i mean that that's definitely very close to uh to a reader macro i think that usage yeah um maybe is uh maybe is that would maybe the author would disagree there but that's the way i look at it anyway yeah uh okay so if david greenberg you're listening yeah you know please please find rayon in twitter and then just shout at him
00:36:41
Speaker
I'm sorry David. Tell him you're wrong. If I misunderstood, to be honest I mean I typically yeah I don't always use a fantasy repple from cursive and I think if I was using a more primitive repple and that's all I had available then this spy scope
00:37:00
Speaker
Would actually come in a bit handy, but I think these days because I use cursive all the time. Yeah, that's got awesome debugging Yeah, and I think it probably supersedes most of the stuff that's in there But but like you say it didn't it it's kind of like a proof of concept if nothing else It's an interesting way of using these data these tagged literals
00:37:22
Speaker
That's true. But I should probably clarify to the listeners that when you are saying primitive, you mean the most advanced ripples like insider in email, right? Yeah, I think I think we're gonna have a fight on forecast soon, like cursive versus cider. All right, okay. I'm looking forward to it. Bring it on. So cider is alcoholic and do it. But what the hell?
00:37:47
Speaker
So cursive has curse in it and sire has cider so pick your battles man. Actually I think you win. On that basis alone. Okay no offense to cursive people so I apologize. Yeah but I think the next interesting part in Reader is that the Reader conditionals so we were explaining a bit about the if-def sort of stuff in Closure Preproster.
00:38:12
Speaker
So you can have essentially conditionally compile the stuff. I don't know if, have you ever seen the .h files or the header files in cross-platform C programs? They're horrible. Oh God, yeah. Well, I worked with C professionally for nearly 10 years. So yeah, I'm burnt by that big time. Yeah. Oh God. But there are interesting stuff in... I'm sorry, go on. I don't want to go into my whole history of it.
00:38:38
Speaker
of the horrors of C programming. Yeah, horrible, horrible time. Maybe that that should be another episode that says how I move from C to closure. Oh, God. Many of the languages in between but but yeah, of course, of course, but that's a but but they're still around, right? I mean, if you're writing iOS code, for example, you would check for the screen size and all the crap and what kind of platform you're on and
00:39:02
Speaker
Yeah, anyway, let's not. I don't think you need to do that anymore. In all fairness, I think most of these things are fixed. But the things that I remember from the seed days are actually a bit like this in the sense that, you know, if I'm not rude about it really, is that people tend to use C macros and hash defines
00:39:26
Speaker
to to do platform specific things so you would you know, you'd have particular things for x86 or for spark or for you know risk or whatever chipsets you wanted to do obviously a lower level than Closure enclosure script but yeah, but it's a very similar concept, you know, yeah, that's true you wanted to branch for this particular type of hardware and Run this particular particular bit of code or you'd want to branch for another bit but I think you know
00:39:56
Speaker
It's only like that from a distance really with a squint, I think. But I think it has some equal issues in the sense that it can become a bit difficult to read. And I think if you overuse this kind of stuff, then you will land yourself in difficulties.
00:40:17
Speaker
Yeah, that's true. But it also enables one of the nicest features, right? I mean, you can have the same code which is, or at least non-platform specific code written in a CLJC file, the conditional closure file.
00:40:33
Speaker
and you can use the same code in the front end if you are using a Closure script and you can use the same thing in the back end. So this is like one of the nicest features of having reader conditionals and also the whole CLJC files.
00:40:49
Speaker
I totally agree. And let's be honest, I'm kind of too negative about the confusion because most of our functions that we really write are kind of less than 10 lines. So it's very different from the CUP processes where you'd go like, if Spark, 100,000 lines of code, else. Yeah.
00:41:15
Speaker
You couldn't hold that in your head. This is very different, I agree. Actually, we've got homo-iconicity at the beginning with the reader and now we've got isomorphism. If you're not bewildered already, you are by the end of it. This isomorphism, this ability to run it on the back end and the front end,
00:41:38
Speaker
That's something which the JavaScript guys are also talking about with Node and the browsers. So you take the same code that is running in the browser and then move it into Node.js or something. But the nicest part for me is that I can, I mean one of the simplistic examples is that writing some domain validation code and you can just put it into a CLJC file and you can ship it to the front end where you can use the same validation functions and you can use it in the back
Reader Conditionals and Tools Reader Speculation
00:42:08
Speaker
end.
00:42:08
Speaker
So that is one of the simplest use cases I have in my mind for for for having the code across different places Definitely. That's a huge reuse when that is. Yeah
00:42:17
Speaker
Yeah. And I think one of the, if you, I'm not sure how many, how many people actually dig into the source code of Closure Compiler though. Probably not that many. It could be a fun exercise, but what I was alluding to is that we have this Closure, essentially all the reading is happening in Lisp reader or Java in Closure Compiler. Not too long, fairly standard.
00:42:46
Speaker
parsing and lexing, sorry, lexing and parsing code. But one thing that is much more interesting for me is the closure tools reader, which is actually reader written in closure. And that's really awesome. Yeah, exactly. That's that's really super fun because that is one of the things used in closure script compiler. So they're in compiling closure script, which would mean essentially compiling and parsing closure. Okay, I think I need to explain it a bit more. So
00:43:15
Speaker
You're writing Closure and Closure Script is essentially similar language for all intents and purposes. Exactly. So you would want to use the same code that you're going to parse Closure and Closure Script because they're the same languages. But you don't want to use the whole JVM driven parser. But if you see Closure Tools Reader, which is actually Closure parser or Closure Reader written in Closure,
00:43:45
Speaker
And that's what being used in Closure Script, which is one of the nicest things because I think, I may be wrong here, but this is one of the steps where you can go into Closure and Closure, for example. So bootstrapping Closure and then essentially writing Closure and Closure. I know it has never been one of the goals of the community, though. Well, yeah, I mean, I think like we were talking about this a while ago is that essentially this proves the point, doesn't it?
00:44:14
Speaker
If you can write ClosureScript in Closure, then you've ticked that box, haven't you really? Exactly. I mean, for ClosureScript, it's already the case. But this was a prerequisite, wasn't it? This Closure Tools reader.
00:44:30
Speaker
I think so. Or maybe it made it even easier. I'm lying actually. You're right. It wasn't a prerequisite because this is only 1.7 plus, isn't it? Exactly. And of course, ClosureScript was out before that. But what I mean is it's a prerequisite to bootstrapped ClosureScript. Exactly. Yeah. And also it might be one of the nicest things that you can use to move to Closure Enclosure. So to make Closure self-hosted.
00:44:56
Speaker
Well, I think what you should do is just make a pull request. Of course. All right. Oh yeah, you said just as well, right? Just make a pull request. You've got most of it written, haven't you? I think I need like a couple of years of hammock time before I can get there. I'm probably not at that level yet. Should we just round this discussion off then a bit now?
00:45:20
Speaker
I think because we've been going for quite a while on this. So I just sort of notice the time there. So I think one of the interesting things about this is that in common lisp and scheme and racket and all these other things is if there are these reader macros.
00:45:37
Speaker
So so users themselves can can extend the reader and this is this I think You know was the cause of much gnashing of teeth when when which he at the beginning said well You're not gonna have them. That's true. Yeah, I mean, this is one of the biggest contention points in in enclosure community I think because most of the people who come from lisp are used to writing their own user defined reader macros and I there are some arguments on both sides, but I think I I
00:46:07
Speaker
Obviously, the show is biased. I think the choice made by Riccik makes more sense because if you see a racket, for example, you can create your own language in the middle of the whole racket language. In the same file, you can start with language racket and then switch to language Poland or language whatever, and then essentially, you're introducing multiple languages within the same language because of the rear macros.
00:46:33
Speaker
But that would mean you are, did we say, complex on this podcast already? I think we've said it, you know, you can do it, go for it. But I think we'll keep doing that in every episode then. Yeah, it's a word that's in, I mean, I think we should, you know, just try and throw these words in there.
00:46:49
Speaker
Exactly. But that adds additional complexity. It complexs the language because every library can essentially introduce its own language, which is really not what you want. And it's not common. It prevents you to have a common language in terms of closure. So I think that is one of the major arguments against having reader macros.
00:47:16
Speaker
But yeah, I think you're right. I mean, it's one of these things where, to me, it's definitely, I mean, to some extent, reader macros make the language more powerful. I don't think there's any doubt about that. But I think the problem is, as you say quite rightly, that power will give you the power to essentially make another language. And that makes the whole, I mean, that was one of the reasons
00:47:43
Speaker
There were many reasons obviously why the lisp didn't really catch on. But one of these reasons, I think reading macros was definitely this whole notion where you had all these guys basically writing their own languages and that's almost like the hello world to these guys where it's okay, I'll just rewrite lisp in my own image.
00:48:06
Speaker
And then they just can't share the code. And why is that, by the way, just to sort of finish that point a little bit, is that if I'm going to write a library that I want to share around, then I have to basically rely on the underpinnings of the language, this language substrate that behaves in a certain way. And I can then write these libraries.
00:48:30
Speaker
If all languages are completely different, well, I'm sitting on sand, aren't I? And that's really the advantage, I think, of this particular choice, is to limit the amount of languages out there. You can always write a DSL. There's no problem doing that. Closure is perfect for those kind of things. But if you're writing the language differently, then the amount of libraries that you can write for closure just goes, well,
00:49:00
Speaker
If you write a new language, essentially, using reader macros, the amount of libraries that you're going to get given to that mini language are just going to drop by a factor of a thousand. So that's a very long way of explaining it. But my point is that if you keep the language sane and common, then you'll get a lot more ecosystem on top of it.
00:49:27
Speaker
Exactly. I think it's something similar to the code that is attributed to Rich Hickey. Or maybe he said that already, is that it's better to have a limited number of data structures having multiple functions operating on it than the other way around. I think that was Alan Pearl, wasn't it? But a long time before Rich Hickey. Yeah. So maybe I'm misquoting. But hey, we like Rich Hickey.
00:49:52
Speaker
it's like on the internet you can just say okay Einstein said this oh god yeah something like that thank you for correcting me though but yeah that's that's um so I'm here for
00:50:06
Speaker
to prevent me from making mistakes. Thank you, Ray. I'm looking forward to that pull request though, man. I really am. I'm really looking forward to it. And I'm sure Rich is as well. Yeah.
Episode Preview and Final Thoughts
00:50:18
Speaker
I think we covered a pretty decent amount of stuff in this episode. Yeah. Perhaps we should explain what we are going to talk about next time.
00:50:26
Speaker
Yeah, I think we'll bring it up a level next time and discuss what we were thinking was, well, we talked about read at this time and that's the beginning of Redevelop print loop. So it would be quite nice to have a little survey or a little kind of discussion around repels. So around the kind of tech in repels, you know, how do they work? What various repels we have around
00:50:52
Speaker
around the community, some standalone ones, some in IDEs, and what kind of workflow the REPL brings for you as well. Some stuff around what happens when you change state, and also some other stuff with reloadable code. So I think we do all that kind of stuff next time around on the REPL. So this is essentially cider versus cursive fight. So people should join in. This is going to be a nice fight.
00:51:22
Speaker
Okay. Well, I look forward to it. Bring your best game. All right. Well, honestly, that's good. I think we've went a little bit long, actually. A little bit longer than last time. But let's try and keep it under an hour. So I think we should stop here. I don't know if we'll be able to cut anything out of this because it was all totally awesome.
00:51:50
Speaker
But we'll see. Yeah, it'll be that comment actually. Okay, so do you want to wrap it up? Yeah, I think that's pretty much it. Okay, excellent. Yep. Right, well, we're going to post the show notes and all of the kind of where we actually post links to the SoundCloud
00:52:13
Speaker
mp3 on defn.audio, dfn.audio. We also host the mp3 on SoundCloud. And if you go there, actually a lot of people like SoundCloud over iTunes, it turns out. Yeah, yeah, that's true. And I like it a lot actually myself. Yeah. I mean, you don't need to have any extra client or anything, just go to the site and start listening. So that's much more easy. I think it's better access, better accessible, I think. Yeah, definitely. Yeah.
00:52:42
Speaker
But we are on iTunes as well, so people want to get it that way. So yeah, all good. Right. Well, cheers, DJ. Yeah. Thank you, Ray. And thank you to you guys and girls who are listening to this podcast and hope to bring more closure joy in your way in the coming episodes. Yeah. Thanks for all the support. And hopefully we can do it all again next time. Yep. Bye bye. Bye.
00:53:40
Speaker
you