Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#23 The Right Honourable Stuart Halloway
48 Plays8 years ago
Wherein we discuss a sweeping array of topics
- the history of Cognitect
- the story of Spec
- past and future of Datomic
- maybe Clojure isn't quite dead yet
Phew!
Thanks very much to Stu for giving his time to us and the community and for his openness to both ours and community questions
Recommended
Transcript
Introduction and Episode Context
00:00:27
Speaker
We're in 23 now, aren't we?
00:00:31
Speaker
Yeah, I know, but I keep on forgetting what the number is. Hello, welcome to episode 23 of the Deaf and Podcast.
Special Guest: Stuart Holloway Introduction
00:00:38
Speaker
We have a mega special superstar guest on the show today. Vijay, introduce yourself. Welcome, Vijay. We have Mr. Stuart Holloway. Hello, Stuart. Hi, Vijay. Hi, Ray. Hi. Hello. Welcome to the show. Thank you for having me.
00:00:59
Speaker
Yeah, it's absolute, absolute pleasure.
Exploring Guilt and Societal Shifts
00:01:01
Speaker
It's great. Stuart, you tweeted in 2016, Stuart, that you left your iPhone stopwatch running, and then you felt guilty. So I did this myself recently, I did a bike ride, and I finished the bike ride and left my stopwatch running for another 10 minutes. And I felt guilty as well. So maybe you have to think about what or talk about or admit what makes us feel guilty now. How about you, Vijay?
00:01:30
Speaker
Actually, maybe I was feeling guilty because I stopped saying fuck, which I just did again, I think. Is that why you stopped feeling guilty? Yeah, I think so. Think about that. Yeah, I need to contemplate it again.
00:01:48
Speaker
But are we sure? OK, we're going to start with what the guilty stuff. Yeah, I'll think about it because there are a lot of things I don't want to reveal it in the public, you know, forum, but we'll see. OK, maybe we'll put that in some outtakes afterwards. Yeah, I think I think I've got some video evidence of you in a Russian hotel and I'm going to hold it against you. You know, coming from the United States, I can tell you that never works.
00:02:18
Speaker
You think that's going to work, but it never does. Actually, it seems like guilt used to be something that we had and we're in a post-Trump era and guilt is just completely finished. You know, just like everything, shame and guilt is, we're just finished with that now. All human guilt and shame is finished. I said it could escalate quickly, Stuart. I'm sorry, but you started it.
Historical Perspectives and Political Parallels
00:02:47
Speaker
That's my fault. I actually don't think that. Before I started my life as a software developer, I was working on a PhD in history. I take a much more long view, cyclical view on these things. Every generation as we age through our lives, you see these
00:03:03
Speaker
things happening, you're like, oh, this is really terrible. And then, you know, your generation finishes up its time carrying the torch and it keeps getting carried. So I'm reasonably optimistic as a person in general, and I'm optimistic about politics in the United States even, even though there's a lot of crazy stuff going on right now.
00:03:21
Speaker
Yeah, actually, since we're getting onto a slightly serious subject, talking about history PhDs, I was listening to this audio book by Timothy Snide, who, if you're a historian you might know about, he wrote a book called On Tyranny, which is all about the kind of markers for tranical regimes. And he was basing on 20th century regimes like Stalin and Hitler. And he was actually saying that,
00:03:50
Speaker
regardless of your optimism, he's saying that there are some quite considerable markers at this moment in the US that may signify a move towards a tyrannical regime. So
Internet Freedom and Technological Responsibilities
00:04:09
Speaker
Yeah, I think that's out there, but I think that we also live in the age of the internet and we have an ability to communicate as a set of global citizens to laugh at nationalist level stupidity where it's happening that did not exist before. I think that a super chilling thing would be successful internet censorship.
00:04:32
Speaker
nation to nation among nations that are thought of having reasonably free speech and free press. But when that's not happening, I think that it is a great counterbalance to local silliness. Yeah, we do have a lot of that coming up now, don't we? With various governments, say the tech companies, they should open up their encryption backdoors and stuff like this to allow this kind of government censorship and activity. So there are some threats out there for sure.
Clojure Community Insights and Developments
00:05:01
Speaker
Um, even to the internet. Yeah, this was meant to be in a light hearted introduction, but okay. Yeah. Great start. Right, VJ, sort us out. Come on.
00:05:17
Speaker
Yes, I'm trying to keep my political thoughts for myself. But anyway, but I think that we have enough discussion about tyranny and other things. So we can just get back to whatever the fun things that are happening in the closure world. And we have, well, can we call you like second in command or I mean, how does it work in closure for you? Like Rich says, call me great leader and you're supreme leader or how does this work?
00:05:42
Speaker
Well, I think Rich is pretty clearly the BDFL for as long as he wants to do that. And I'm hopeful he'll want to do it for a bit longer, at least, as I think he does a very good job. And then there are other people who take on different roles. I mean, the person who probably spends the most hours in the week, probably more hours than there are to be awake, worrying about the stuff in addition to Rich is Alex Miller, of course. You guys have spoken to him before. Yeah.
00:06:12
Speaker
You know my role is I'm a committer on closure and And I read every line of code that goes in so I don't think there's a particular name for that I would like I would like to say that's a careful committer who doesn't ever make mistakes But that also means everything that goes in that's not as good as it could be it's at least partially my fault, but it's been quite an interesting role and I've been doing that for at least a
00:06:38
Speaker
I can't remember when I started doing that, at least probably since 1.2. So through a lot of major releases, reading all of the lines of code that go in.
00:06:46
Speaker
So am I making up the history if I remember? Maybe I remember it from some of the discussions. I don't exactly remember where I heard. Maybe it was my imagination. So you met with Rich when he was presenting at the JVM Languages Summit. And then that's how it all started, right? Because you were doing Ruby at that time. So that is pretty close to right. So Justin Gatlin and I had created Relevance in 2003.
00:07:12
Speaker
And we had taken our company and switched it from primarily enterprise Java to primarily Ruby on Rails. And we had made that switch fairly early in 2005. And we made that switch because Justin wrote a blog post about productivity in Ruby and got slash dotted and we were serving our own blog and our server melted.
00:07:35
Speaker
Fortunately, our server was not self-software we wrote, so we were able to wave that off a little bit. But after having in my career made three or four language switches that were dictated by employment situations, and then one language switch to Ruby that was chosen, but was mostly chosen from a set of outside influences. I got to know Pragmatic Dave Thomas,
00:08:01
Speaker
traveling around speaking at conferences in the United States and James Duncan Davidson and other people who were Ruby enthusiasts to varying degrees. I decided that it was important at the level that I wanted to reach as a software developer to do a more systematic search. And so my interest and closure grew out of a systematic search. It was called at the time the search for Java.next.
00:08:26
Speaker
And so trying to find, you know, the next thing there. And so I discovered closure and it must have been, it probably was about a year before that JVM language summit. So I started writing a book about closure. I decided that this was the thing and I started writing a book, which ended up being the first published book on closure. And then, yeah, that's from a pragmatic programmers, right? That's right. And Alex Miller is currently, um, just started the third edition. So I think the electronic alphas of the third edition of that book are just coming out right now.
00:08:56
Speaker
And so I was working on the book and then because of my interest in closure, Brian said you ought to come to the JVM Language Summit. So I did and met Rich there and I'm like, hey, I'm the guy that's writing the book about your language. And he's like, hey, I'm the guy that wrote the language.
00:09:15
Speaker
We started talking and we hit it off and I said I would love to have any kind of feedback you would provide on the book and he said it's so awesome to have a book that we can release essentially contemporaneously with Going 1.0. So we were able to coordinate that and release the book along with Closure Going 1.0.
00:09:36
Speaker
You know, I think a great beginning. And if you look at the history of other programming languages, you know, they had to wait, you know, often years, uh, you know, before a book to get a book. So, so it was a good start.
00:09:47
Speaker
Yeah. I think there are a couple of most important questions because usually when people listen to the podcast, they want to have the immediate TLDR sort of thing. So first of all, are you a vegetarian?
Personal Reflections and Lifestyle Choices
00:10:01
Speaker
So I spent about three years being a vegetarian when I was in my 20s. Three good years.
00:10:09
Speaker
My primary reasoning for being a vegetarian was environmental, and I continue to care about those issues. But I also think it's important to be an effective leader and persuader. So I persuaded two other people to become vegetarian without pushing them. It wasn't an evangelism thing. I explained why I was doing it. So you mean something like my way or highway?
00:10:34
Speaker
No, no, I just, you know, uh, I just said why I was doing it. And then these two people became vegetarians and they have both remained vegetarians for life. So I feel like I don't have to anymore since they have it coming. Oh man. Two in, one out. They've come in your bets.
00:10:54
Speaker
So there is net effect still one vegetarian, one more vegetarian. Yes, we have. I feel like my life contribution is one net vegetarian. So that's, that's pretty good. And, and my family, you know, we enjoy vegetarian cooking. And so, you know, we do eat a fair amount of vegetarian food, but but no meat-a-terrion if I had to choose.
00:11:16
Speaker
but there was there was a question actually from somebody on uh... on slack uh... show on my own beef jerky your vegetarian beef jerky what what is that that that that's very
00:11:27
Speaker
so i have this example when i'm introducing datomics query language all right that is about it's about it's about a fictional store in montana that sells jerky of various kinds and it just depends on how flip a mood i'm in how much that story gets embellished but having searched back in my memory i'm sure that that story has included strange vegetarian jerky
00:11:50
Speaker
Excursions, but I actually I enjoy beef and buffalo jerky, but I'm not aware of any actual vegetarian jerky Okay
00:12:01
Speaker
Right. Totally fictitious. Right. So, one net vegetarian and some fake news about vegetarian beef jerky. Yes, fake news. Good. Let's get to the most important question of this podcast. So, Emacs are other random shit that nobody cares about.
00:12:22
Speaker
Oh, my. So I actually started my, you know, I feel like I'm obliged to represent for, you know, where I came from. And I started my career in the late 80s modifying Emacs.
00:12:37
Speaker
to support Chinese. That was my first programming job. And I took that job because it was the highest paying job in work study. It was $6 an hour. So I worked on extending Emacs to support Chinese characters for $6 an hour. But I've stuck with Emacs with, you know, I visited other editors, but I've stayed with Emacs ever since. But I'm not particularly religious on the topic.
00:13:05
Speaker
There, in the answer that's guaranteed to please no one, right? Yes, exactly. I smoked, I smoked Emex, but I didn't inhale.
00:13:18
Speaker
Okay, so the most important questions are are now settled so we can go to the I think there were a lot of questions on Twitter when we Started so before we get into that one So you you can you can you give us you said you change the languages multiple times from Java to Ruby to closure and
Cognitect and Datomic: Past, Present, Future
00:13:38
Speaker
So can you can you tell us how did you start relevance maybe a bit bit before in the end from from from programming history yes your programming history. So I was I worked at a very interesting company from nineteen ninety eight until two thousand and three it was a software training company in the united states called development or.
00:13:58
Speaker
And this company prided itself on having probably the best amount of outside Microsoft expertise anywhere. And the technical go-to guy there was Don Box, who is now at Microsoft working on the Xbox. I think no relation.
00:14:20
Speaker
He just named it after himself. No, I think the name was there before he got there. You know, embarrassingly, in terms of contribution to software history, developmentor was one of the signatories on the soap spec. So I'm not sure that's the thing to be incredibly proud of at this point. Well, Richard taught C++, didn't he? So I guess you've got equal sins there. Everybody has skeletons in their closet.
00:14:45
Speaker
Working there, I got to meet a range of people not geographically bound. It was a travel education kind of gig, and so I got to know a lot of incredibly sharp people. What happened back in the day was that Microsoft had this technology called Microsoft Transaction Server.
00:15:05
Speaker
And then Java came out with this technology called Enterprise Java Beans, which was partially a rip-off of Microsoft transaction server. And we saw that as an opportunity to branch the business out from Microsoft to Java. So I made the switch from Microsoft technology to Java technology kind of in a backwards way by knowing a tool chain that had great similarity that ran on top of the language and then working my way back into
00:15:35
Speaker
Learning Java and then in 2001 you had the towers fall down and everybody in the United States stops traveling and Working in a job that relies on your customers to travel to your city for training becomes a very very unpleasant
00:15:52
Speaker
deal for several years. It took a long time to recover. And in fact, it had not fully recovered by 2003 when I decided that I had had enough. I mean, we, you know, bailed water for a couple of years and development or survived. And that was great. But Justin Getland, who worked there with me and I,
00:16:10
Speaker
decided that we wanted to go in on our own and do consulting and actually help people build stuff with the technology we were talking about. And our expertise then was still relevant Microsoft expertise to some degree, but more Java. And so what I would call kind of early agile Java and people that were early adopters of Spring and Hibernate and things like that. So we were doing that and helping people use those things
00:16:37
Speaker
and really started the company with a clear idea of the kind of lifestyle we wanted to lead and a clear idea that we wanted to stay on what we perceived to be the leading edge of business use of technology. And not necessarily a lot of wisdom about how to run a company, but fortunately we've had 14 years to work on that now. So we've survived and gotten better over time.
00:17:04
Speaker
I was about to ask that because 14 years is a long time for a company. When you started the company as founders and now, what changed? What was the culture that you were going for because you said there is certain lifestyle that you wanted to have with the company? Is there still code DNA that's still there at Cognitec because it has a big transition from relevance to Cognitec?
00:17:29
Speaker
So I think that certainly our interest in staying on what we perceive to be the cutting edge of technology has persisted throughout. And that means switching technologies and switching things that you're interested in. After a few years, it became evident that to have the kind of impact that we wanted to have, we were going to have to do more than just be two people who
00:17:52
Speaker
sort of flew in and flew out. And so, you know, deeper engagement and doing entire projects. And so that led to, you know, relevance growing and hiring people and developing a larger presence.
00:18:05
Speaker
And then following on that, the switch to closure led to a new set of people who are interested in working for us at Relevance, a new set of customers and people who are interested in potentially working with us. So that was another big transition.
00:18:28
Speaker
working with Closure, at some point along that trajectory, we talked with Rich about that and said, hey, should this be a company that's primarily about Closure? And if it is, would you want to be a part of that? And we decided, initially at least, instead, Rich said, well, let me show you this thing on a blackboard that I've been kind of interested in. And so he sketched out the ideas behind Datomic on a blackboard
00:18:54
Speaker
over the course of a couple of hours, and I injured my jaw as my jaw just dropped further and further and further. And to be honest, before that, I had never really even liked databases. I had seen databases. For somebody who came up through the languages that I came up through and came up through the cultures that I came up through, there was a lot of the database is the least agile part of your tool chain. It's the thing you have the least opportunity to influence.
00:19:23
Speaker
And so just having the guts to say, you know, we can change that level of the stack and make it something different. And of course, we benefited in a huge way from the NoSQL movement. You know, if that had not happened, it would have been a lot harder to have had the business courage to say, yeah, we'll make a database that works differently and see, you know, if everybody's still perfectly happy with SQL, if they're going to want to do that. So so we did that. And
00:19:50
Speaker
And after having released Datomic, so Datomic just turned five in terms of public release, I guess three months ago. So it's been out for five years. And after that initial release, we got a lot of feedback from users. And as we internalized that feedback, one of the things we decided was that there really is a lot of synergy between Closure and Datomic.
00:20:13
Speaker
And so it made sense to form Cognitect and to have one company that was a bunch of thought leadership enclosure and the team behind Atomic and able to explain
00:20:29
Speaker
how Closure works, why you want to use Closure, how Atomic works, why you want to use Atomic, why in particular you might want to use them together. And that has worked very well for us. So that's been a good thing. And I think it's been a good thing for business users of Closure. And that's certainly something, this is our day job. So we are interested in helping companies that want to use Closure find a way to use Closure, regardless of whether they use Closure
00:20:56
Speaker
regardless of whether it has any financial impact on Cognitec, because as we all know, with the sort of dominance of open source and programming languages, people have a lot of different choices. And so you want your own ecosystem to feel healthy and safe and know that there are going to be other people working in the space that you're working in.
00:21:16
Speaker
So, the following on from that actually is because we had some like topics around this stuff that we thought about and since we talked about cognitive tech and this is kind of the growing up story.
00:21:29
Speaker
I don't know if you can share anything about what your plans are to grow the company. Are you going to do the VC thing? Because obviously, everyone knows on this podcast at least that you've got some pretty awesome technology in there. If you are going to do VC, well, okay, maybe you can't talk about the details. But if you're not, then it would be interesting to know why not or what your feelings are about how to grow a company like
00:21:59
Speaker
like the atomic or cognitive tech or what you're doing, how do you get big? Do you want to get big? I think that if you read between the lines, in some ways, we are answering that question in visible ways by our behavior. But to be explicit about it, if we were taking funding, clearly, we were doing a terrible job at it because we're not growing 5x every year and whatever. That's not the kind of thing
00:22:23
Speaker
I'm actually neutral on that. I'm neutral on rapid growth or not rapid growth, but what I'm interested in is doing the technology right and getting the people bits right. We want to build great tech and we want to be good people and work with good people.
00:22:44
Speaker
You know, if we can grow faster while doing that, that's great. And if it takes time and requires patients, that's fine too. And that's been a mix of that. So, you know, the company has been healthy and has grown some years and has been relatively flat other years.
00:22:59
Speaker
And right now, it seems like there's an uptick in interest on the business side and closure. So if I had to guess right now, I would say that 2017 will be a growth year. Obviously, it's midway through the year, so I can't be 100% certain. But I think that things are trending in a good direction. But it's driven more by who we want to be. And of course, the longer you don't do something like take VC funding, the less realistic it is to do.
00:23:26
Speaker
I mean, VCs want to have something tiny that they can put a small thing into and get a huge return. And we're already not a tiny little turtle egg. We're a small turtle that already successfully got into the ocean and swam around a while. So we're less interesting to them. And we're not that interested at this point and haven't been. So that's OK. OK. Well, let's hope the turtle gets bigger. That's good. Because it's turtles all the way down. We know that much.
00:23:56
Speaker
That's right. So yeah, there was another one was I think about Cognitech was that there was something you built recently about automated simulation testing.
Clojure Spec: Introduction and Potential
00:24:10
Speaker
The kind of tools that you build in-house that you were talking about. There was a question from Jan Oberhagerman on Twitter, I think, about whether or not you would open source this or whether you'd make something more of it.
00:24:24
Speaker
Maybe you've even forgotten about it. I don't know. So we devised a simulation testing engine for Datomic while we were building Datomic. And we open sourced that several years ago as Simulant. And Cognitex has used that in consulting engagements to do simulation testing for customers. And Mike Nygaard and Paul DeGrandis and Craig Andera and other people have worked on
00:24:52
Speaker
simulation testing tools and have open source them. So as far as I know, the things that we have done are open source. But but in particular, I think that that tweet mentioned Paul de grandis. And he is sort of an energizer human. So it's entirely possible that he has built five things in the last two weeks that I haven't heard of yet. So we can double check.
00:25:12
Speaker
But certainly I think that generally, general purpose testing tools, I don't see a purpose in not having them be out there and released. It's not something you're going to. I mean, I think having a full framework or something that's more targeted towards a vertical or something like that, you could make something commercial. But our testing tools, we build and we open source and they include things like test generative
00:25:37
Speaker
and, you know, Cognitech, and we also, you know, support through Closure's infrastructure testing tools that are developed not by us. So, you know, Cognitech people did not write much of the code in test check. And that turns out to be the most important piece underneath spec in terms of the generative side of things. Good transition there, good transition.
00:26:02
Speaker
Yes, well, you know, I got the problem. You're a pro. This is, we didn't rehearse this, but it seems like we should have done. Okay, so suspect, suspect. Let's do it.
00:26:16
Speaker
Spec, yeah. I think probably many people who are listening to this will know about spec, but can we just like, just for the sake of completeness, maybe it's reminders of the basic goals of spec, because it obviously is going to be a huge thing going forward. So just to get us all on the same page, just want to quickly go through what the top tent pole items are.
00:26:41
Speaker
Sure. So the one sentence I would use is that spec is a standard, expressive, powerful, and integrated system for specifying closure programs. And those are official words from Rich. And anyone who's seen Rich give conference talks complete with multiple dictionary definitions on slides knows that we try to take words seriously. So to sort of fill those words,
00:27:10
Speaker
in a little bit obviously spec is integrated right it was added to closure as a thing that's integrated in closure and it touches closure in about 10 different places and that represents careful thinking about all the places you would want to use this and it is one of the value propositions of spec to have tried to think carefully about if you had
00:27:32
Speaker
specification where in your language could you use it and try to make sure it was being used in all those places and that it was suitable for all those places and you know one reaction you can have to spec and this is a completely legitimate reaction is I've seen all this before right there's not in a certain sense there's nothing at all new about spec but I think it's a very tasteful combination of things in terms of how it's integrated so that's the integrated part
00:27:56
Speaker
I think having it be standard in the sense of everybody uses the same one instead of 10 different competing ones, not in the sense of going to a standards body. We're not going to get ISO certified spec or something like that. But I think having it be standard is important. And of course, the integration gives it a huge leg in that direction. It's the presumptive favorite. It's built in. And it's important that it do the things that people who have been using other specification tools and closure wanted to do so that it covers those needs.
00:28:26
Speaker
And then in terms of being expressive and powerful, it's expressive in that it builds on what's already there. So any predicate that already exists in Clojure works in spec. And it doesn't work with an adapter or widget or something you have to add to it. It's just it works. Likewise, all of the collection types in Clojure, of which there are few, and that's itself
00:28:51
Speaker
a value prop of closure, all of those things are fully integrated into spec. So you have out of the box with spec a set of tools for talking about the shape of data that's much richer than say the type system that people are used to coming from Java or C sharp or something like that.
00:29:08
Speaker
And then another way to look at it is, does it cover the different use cases that people have for specifications? So one thread leading into spec is generative testing. Can I describe my programs with specifications and then have the language write tests for me? And I think we have done a good job with that. Another one is
00:29:32
Speaker
better error messages specifically from macros. So in Clojure, because it's a list, you can extend the language with macros. And macros are notorious for being crazy powerful and being able to provide amazing extensions to the language and having completely opaque failure modes. So if you pass in the wrong stuff to the macro, you get a stack trace from the compiler or whatever. And so being able to cover that use case
00:30:01
Speaker
And if you look at what spec does, the regular expression part of spec in particular, because one of the things that macros do that is actually not very common elsewhere in Clojure is a lot of dependency on order.
00:30:15
Speaker
All over closure, we're always saying, let's use maps for this. Let's use maps for that. That's going to be flexible. That's going to be self-describing. But nobody writes their programs with maps. I mean, imagine what your function definition would look like if it was colon, fun name, fun name, colon, arglist, arglist, colon, body, body, or something like that.
00:30:33
Speaker
But the ability of specs, the regular expression component of specs is pointed directly at problems of macros. And if you go back and look at libraries that people have written in Clojure and tools that people have written to help Clojure programmers and talks that have been accepted at Clojure conferences over the past five years, there's this thread of how do we make
00:30:55
Speaker
macro problems more understandable for users. So it covers that use case. It covers making development time safe. It covers providing tools that you would not use in production, but you absolutely would want at development time. So instrumentation, for example, to say, let's tighten down everything in this program and show me if I'm doing something wrong.
00:31:16
Speaker
Then it supports ad hoc data validation. It's completely suitable for you're making a web service endpoint that delivers JSON, doing a spec check at the front door on the way in and on the way out to make sure that you are producing and consuming the data that you expect. It's performant enough to be suitable for that kind of use. I think that gets to the power of spec is covering this broad set of use cases. Yeah. Recently, it's been moved out of the core.
00:31:47
Speaker
Yeah, I was just going to say, an interesting point about 1.9 there is that you've, I take your point about the integration, but at the moment you've made a split in the alpha. So could you explain that a bit more, given your assertions about the need for integration here?
00:32:10
Speaker
Right, so I'd like to, before I talk about the spec split, talk about what we mean when we say alpha on closure releases and in closure documentation. Because I think that people have three or four different things that alpha might mean to them. One thing that alpha might mean is brand new.
00:32:30
Speaker
Another thing that alpha might mean is crappy. Another thing that that another thing that alpha might mean is not tested very much, which might be crappy or might not. And another thing that alpha might mean is subject to change. Right. This is something that we don't consider fully baked. And so if you come to depend on it, you may be broken later. Right. In terms of things that are released in closure, the things that are marked alpha are only ever that last thing.
00:32:56
Speaker
When you see something in Clojure or a contrib library, now I can't speak to every person who writes Clojure code, but when you see something in a Clojure library or a contrib library that says alpha, that means not brand new. I mean, it might or might not be brand new. We think this might be crappy. It does not mean anything except subject to change. Having said that, spec is subject to change.
00:33:20
Speaker
We want to change spec. Spec's going to evolve. People are going to figure out ways to do things better. And so the namespace that spec now lives in as of the alpha 16 release of Clojure has the word alpha not just in the Maven artifact, but in the actual namespace name.
00:33:35
Speaker
And that's a warning to users that you really can't avoid seeing, right? It's actually when you do your namespace reference in your project, you have to type the word alpha or let your IDE auto complete the word alpha for you and say, hey, I'm using an alpha thing. And what you should take from that when it's coming from Clojure itself is not that this is crappy or filled with bugs or whatever, but that it is possible that this is going to get changed out from under you in the future.
00:33:58
Speaker
Now, having said that, I don't think we'll change a lot. I hope we don't change anything. We aspire not to change anything. But there's definitely more likelihood of something in spec changing than something in closure changing. It's also the case that spec is likely to want to evolve more quickly than closures release cycle, which is anywhere from six months to 18 months.
00:34:18
Speaker
between major closure releases historically. And so the reason for this split is to allow closure one nine to be released and allow spec to continue its own evolution without having to be, without spec having to be held up for waiting for closure releases. Okay. So just so I'm clear about that, does that mean that 1.9 itself will not respect or are you still, you're still using it internally?
00:34:47
Speaker
Absolutely. Closure of one nine will use spec internally and does already. And, you know, I hope that by the time we get to one point nine done that all the macros, right, those are the things that cause people the most beginner pain and closure. All the macros will have specs would be would be great. Several of them have specs right now. And so there's actually three libraries. There's closure itself. Then there is spec alpha and then there's closure specs alpha.
00:35:11
Speaker
which is specs for closure and those can evolve independently as well. And so closure will be spec'd and closure specs can get better. Spec itself can get better and closure specs can get better without those things having to be depend on closure's release cycle. And that's the objective. Okay. Well, I think that's pretty clear. Yeah.
00:35:31
Speaker
There was another question actually, which was like from Arno Boss actually on Twitter. And that was, do you have any, I mean, I guess it's still early days, but you have any plans for building tools on top of spec? There is a part from error messaging, which you were kind of alluded to.
00:35:48
Speaker
So do I, Stuart Halloway, personally plan to build tools on spec? I'm thinking about whether it'll be tools released in the, OK, forget it. No, I want to talk about this because I think it's incredibly important. I think it's important for people to, it's much harder to visualize the potential of a thing without a glamorous user interface. And so I would love to see people building glamorous user interfaces for spec.
00:36:17
Speaker
And those are going to be context dependent. So there's going to be an IntelliJ ID glamorous interface for spec. There'll be a separate one for Eclipse. There'll be a separate one for Lighttable. There'll be a separate one for them, Emacs, whatever. There's a million things to be built there. And there are all these people who are enthusiastic about making closure easier for people who are getting started. So here's a nice concrete idea. Closure spec.
00:36:45
Speaker
is a tool that lets you draw red squiggly underlines. Smarter than static typing would let you draw red squiggly underlines. That stuff's not done yet, and it's IDE specific, right? Somebody needs to go and draw red squiggly underlines. This part of your data is broken.
00:37:01
Speaker
And it's much more general purpose than a type system, right? It doesn't have to be your program. It can be the JSON input to your web service is nonconformant. Here's a red squiggly underline where it's nonconformant. And so I think that that spec is a rich enabler of GUI tools.
00:37:17
Speaker
And those GUI tools are not reliant on any changes to Closure. There's no need for Closure to do anything to enable those tools tomorrow. And we're starting to see some of that. I know that I think some of the Closure on iOS stuff has cool spec-enabled highlighting already. And I think that stuff is fantastic, and I would love to see more of it.
00:37:41
Speaker
In terms of spec itself, you mentioned that it's alpha or it's subject to change. Is it about refining what you've got? Or is it about you think that there are some ideas that are still not in the spec world yet that you want to bring into that spec world? Or is that, again, just something that is you just want to leave it open to evolution? I guess the question is, is it kind of done, needs to be polished? Or is it kind of still open? Is the door still open to new ways of doing things in spec?
00:38:11
Speaker
So I think that there's definitely a need for polish. There's no doubt. There are tickets right now for things that need to be done. There are things that are chapping me when I use it that need to be enhanced and fixed. And I'm easily satisfied. So we should get more input. And I would say it would be great.
00:38:32
Speaker
The biggest thing that would help with spec would be for people to use it and provide experience reports. That's incredibly valuable, those experience reports. And so part of the answer is the Closure community is way, way smarter than any one or 10 of us. And so I would love to know what those stories are so we can all digest them and think about that. Having said that, I have a couple of immediate thoughts
00:38:58
Speaker
of things that are unlikely to have further refinement in the 1.9 timeframe, but I think are clearly areas where things could substantially change, hopefully, additively. One of them is test performance. And this gets into having a more closure-ish way to think about dependencies. I mean, right now we have this very
00:39:22
Speaker
monolithic way of thinking about dependencies where the granularity of dependencies is an entire jar file and you know as soon as the happy world that I want to see come into being of people writing generative tests with specs becomes a little bit more established it's going to be immediately followed by a sad world of oh my god these specs take a long time to run
00:39:42
Speaker
I used to write 10 unit tests that did 10 things. Now I write 10 specs that each do a million things each because they're generating a million cases. It takes a long time to run. We can be a lot smarter about dependencies and be able to say, you know what, I know, and you've seen some research on this. We had the codec project with the atomic a few years ago. We can keep track of dependencies in a much more granular way and say, you know what, I know in this namespace, these 75 functions didn't change and this one did.
00:40:10
Speaker
And so if the spec doesn't depend on code that goes through a path that hits that one function, I don't need to run it again. And you can even take that information and share it, right? You could shaw that information up and put it in your GitHub repo and say, you know what? I already tested this. Here's a shaw certifying that I did. You don't ever have to run that again.
00:40:28
Speaker
So I think there's a lot of sophistication about what tests we run that ought to happen. Another thing I think that people will encounter is the programmability of specs themselves. So given a spec, you can call form and get back the form that was used to make the spec. But if you use specs for a while, you find yourself saying, you know what? I want to be able to write programs that pick and choose and combine bits and pieces of other specs. And that's possible, but I think that could be easier and better.
00:40:57
Speaker
And so I think that's another area where spec might change and improve in the future. And I'm sure there are others. But those are, like I said, there are 10,000, 15,000, 20,000 people, whatever, in the closure ecosystem. There's ideas for one person. But I think we're going to see a lot of other ideas as people use it more.
00:41:17
Speaker
I mean for me I was thinking that I just thought about this but one of the things that we've got are these support for strings and for ints and that kind of stuff and we have that kind of thing in the reader as well and the reader is extensible but it seems like
00:41:37
Speaker
There's very little sharing of those extensions, except between clients and servers in particular projects. But actually, there are quite generic things that could be useful outside of it. And I guess spec should benefit from that as well eventually. There should be some kind of stuff built around or on top of that. Maybe it's not by you, but by the community to share.
00:42:05
Speaker
reuse commonly defined specs for domain specific things, maybe? Yeah, I think it would be fantastic. I mean, how cool would it be? Like money in dates and stuff like that. Right. And well, not just money in dates, but like in terms of like, you know, the getting started started experience is so important. What if all of the cool little sample apps enclosure ended with having a spec that was, you know, you would reuse if you were actually doing real work in that domain?
00:42:33
Speaker
Right, so if you actually if you actually were building the next big to-do list or whatever right you know or the next big That you would already have the the appropriate specs for that domain But it would demonstrate the value problem and the real point is that yeah We could have we could have spec sharing and I think that that will happen and it's just a matter of you know right now people are still just integrating the ideas and
00:42:57
Speaker
Just one, I mean, this will sound a bit sucky to you, maybe, but I think that what you guys have done with a spec guide is really good. You've got a new yourself, I've done a series of videos about how to use this thing. The community has really stepped up and I think made a lot of articles and tools and stuff like that. This, I mean, honestly, for me, this feels like maybe the first, not the first time, I don't know, feels like for me, maybe he's living through it.
00:43:24
Speaker
it's a really great way to introduce new technology. It almost feels like it's been planned or it's that you guys have kind of like done a really great job of putting it out there and and gendering kind of enthusiasm and I don't know was there a plan around this or is it just kind of like we'll do it this way and then we'll adapt or did you have a did you have a kind of strategy for for releasing this to the world in a certain way and because it all seems to have worked I just wondered if it was kind of forethought of it's just being
00:43:54
Speaker
serendipity. So well, first, thanks. I mean, I'm pleased. I'm pleased that that people were able to get into spec. And I think that the work that Alex Miller did on the guide is an important piece of that. And
00:44:09
Speaker
And it's easy to prove out your statement, right? We can go look at the number of things that are interesting and powerful and complicated in Clojure, and then we can go and look at the number of things that have guides for them. And there's very few guides. And so, you know, we have, I guess, Chaz Emmerich originally, and then more recently, Cognitech have run the state of Clojure survey.
00:44:35
Speaker
It is a feedback item every year, more and better documentation. I agree, more and better documentation is a really good thing. First off, I want to say that I think that the closure community as a value nails it on emphasizing rationales.
00:44:57
Speaker
Having a rationale and explaining why you're doing what you're doing and situating it in a context of other things is incredibly valuable. And I think that there's a good job being done there. I think there's a lot of work that could be done at the guide level, which I think is the next thing you want. And so yeah, there was definitely, thinking about how to build this, it was like, okay, rationale, check, we know we do that, that's how we roll. And then it was like, okay, what is the next highest value thing that could possibly be made?
00:45:27
Speaker
And I think the, to me, the answer was guide. And so that's why we did it. And I think people liked it, but you know, that would be great to get feedback on what other things people would like. And then I think the videos that I did and other things are really kind of third order, right? That's nice. And, and you know, there are people who want to, you know, get things from videos and that's great, but I would, you know, if I could only do one thing, I would do the rationale. And if I could only do two things, I would do the rationale and then the guide.
00:45:51
Speaker
So yeah, I'm pretty happy with the order that we did things. And it would be great to have more guides. It would be great to have people looking for the best writing in Closure blogs and reaching out to them and saying, hey, you ought to make that a guide.
Datomic Development and Community Engagement
00:46:06
Speaker
You ought to go contributing to the Closure website has never been easier. It's by pull request. So it's a workflow that people are familiar with. And Alex Miller helps people shepherd things that they're writing through in there. So it would be great to have more guides.
00:46:22
Speaker
Yeah. I'd like to know what your typical day is like. Most of the time do you spend on Datamic or do you spend on customer projects or how does it work with you at Cognitec? It varies from time to time, but in terms of the things that I am working on right now, I spend the bulk of my time thinking about making the Datamic experience on the cloud better.
00:46:48
Speaker
Yeah and then i spend a substantial minority of my time thinking about development at the rebel and how to really evangelize the rebel particularly closure developers and i think that we. I think we under sell the dynamic development aspect of closure if you go to the website.
00:47:07
Speaker
If you go and look, I was surprised when I was doing the research, I'm giving a couple of talks on REPL development in Chicago in a few weeks, and I was surprised when I went back to the website, you know, closure lists dynamic development before functional programming. It lists it before, it is the first thing.
00:47:24
Speaker
on the list as this is really important. It's not just about being dynamically typed. It's about having this tangible runtime that you have a streaming interaction with that you can evolve and change while you're programming. And so I think trying to...
00:47:40
Speaker
Right now, for me, it's like trying to take what is sort of unconscious competence, you know, things that I do that I don't even know that I'm doing and turn them into pros or blog posts or conference talks and thinking about how to, you know, share that with others and also encourage other people to share their stories because every time I talk to anybody about how they use Closure at the Repel, I learn something.
00:48:00
Speaker
My experience is one experience, but there are other interesting experiences as well. I happen to have been thinking about that all weekend, so I'm kind of gushing probably. Nice. So as I said, bulk of the time, you're spending on Datamix. So Datamix has been like, of course, as I said, it's almost five years.
00:48:22
Speaker
since Datomic has been released. And there has been a lot of chatter around when it was released. People were asking, OK, why it is not open source. And we also got a couple of questions from Twitter, which are essentially centered around, I think, one from Nola Stowe. I'm sorry, I don't know how to pronounce it, but it's a closure geek on Twitter. He says, will Datomic ever be open source at a lower price? And there is another one, spiral ganglion.
00:48:50
Speaker
And it's also fairly similar to that one. The question says they have a very small startup and they want to use Datamic. They're very much in love with Datamic. But the pricing seems to be only open source or enterprise or VC kind of thing. So these two questions, I mean, this has been there for five years now, right? Will it ever be open source? Will it ever be, I don't know, so cheap that anybody can buy it.
00:49:20
Speaker
So, just to tell people who maybe have not followed the history, so we have had, first let me just say, I feel incredibly lucky that we launched a product that a lot of our interest comes from the closure ecosystem and that we have people who are incredibly enthusiastic. And so, you know, our community of
00:49:42
Speaker
people who are trying Datomic and our customer base have been incredibly nice and supportive and so it has been a really fun process of evolving the product and we have
00:49:55
Speaker
We have evolved the licensing model and the product and the pricing two or three times over the five years and all of that was in close consultation both with customers and with people who are about to be customers and people who are a little bit further out and said, you know, well, I can't be a customer because of X, Y, and Z.
00:50:19
Speaker
And so that dialogue has been really interesting and challenging and the changes that we have made have been very well received. And that's because we came out with our initial story and then we entered the market and we found out what we thought that was correct and what we thought that wasn't correct and made changes.
00:50:39
Speaker
So that's been good having said that talking specifically about the pricing part of my interest in making the atomic experience better on the cloud. Comes from our history we launched atomic as a cloud only a very future thinking database it was it only ran in the cloud and the very first thing that happened was.
00:51:01
Speaker
Some people said, great, and they started using it on the cloud. But a lot of people said, we want to run that in our data centers. So we got dragged from future think right back into present think. And over the last five years have done a lot of work on the datomic experience in data centers. And that's work that's not super visible in the surface area of the product.
00:51:23
Speaker
the API, it doesn't make a difference there, right? That's been stable. But in terms of interactions with customers and helping them run on their storage or work around strange things about their data center, which nobody else would care about. And those are people who are willing to pay for software. And so that's been an important part of the story. At this point, we have the time and the resources and the flexibility to be really refocused on cloud again.
00:51:52
Speaker
Right we have we have good answers for people who are running in data centers and of course our customer base is five years further in the future than they were five years ago and so they're more ready. To be in the cloud and and so on and so we're able to look at things to say okay you know atomic is already.
00:52:09
Speaker
split apart compared to a traditional monolithic database. A monolithic database has these high value, high ceremony sacred processes that you have to sort of baby along and Datomic really breaks that model and it says we presume cheap ephemeral read resources that are distinct from cheap ephemeral write resources that are distinct from cheap ephemeral storage resources, any of which might fail.
00:52:35
Speaker
And the architecture is designed like that. But there's more that we could do there. We could make it simpler still. We could break apart parts that are currently delivered together. We could also do...
00:52:48
Speaker
more automation, so more of, you know, you focus on the semantics of your application, and Datomic does more operationally for you, so you don't have to worry about that. And when you simplify further, and when you automate those pieces, so simplifying, as people who come to Clojure sometimes learn, simplifying, you know, the short run is not always fun, right? You had one thing that was really complicated that did what you want, and now you have 10 things that are simpler,
00:53:16
Speaker
that you have to put back together to make do what you want, but you're going to be able to make them do other things in the future. But once you have that simple thing, once you've got simple sorted out, you can really focus on easy. And you can say, how can we assemble those 10 pieces and make it really easy for this use case without compromising on the simplicity? And one of the things that comes out of that then is the ability to deliver a more granular
00:53:41
Speaker
Datomic and a more elastic Datomic. And this is just facts of the software and facts of its operation. It has nothing to do, it seems, at the front with the business model. But once it's more granular and more elastic, then that lets you come back and revisit the pricing model. Now you can say you have super tiny granular Datomic that's suitable for a hobby project. So that's definitely an interest and definitely an area where I'm spending my time right now.
00:54:08
Speaker
Okay. So I'm assuming that the open source thing is kind of a no-go for this one because you want to keep it like a closed source one with the core team being Cognitec. Sorry, the Datamic Inc. Yes, that's right. Datamic is not and will not become open source. So that's an easily answered question.
00:54:29
Speaker
Yeah, yeah. And is there a plan or something that you're going to build your own storage engine for it? Or you're always going to piggyback on, I don't know, Amazon, Redis, or Postgres? So you keep using the same storage modules.
00:54:46
Speaker
So that's an interesting question. We already built a storage engine for it back in 2011, right? Five weeks before we were gonna ship, Amazon announced DynamoDB. And we took a look at that and we looked at, and we were happy with our storage. It was Rich Hickey certified simple, but it was a new thing. It was a new thing that people had to run on their own behalf.
00:55:12
Speaker
There's not a need for that. People are not looking to us. People are much more interested in one of two things. Either they want to run Datomic in their own data center, in which case they would be happier if it ran on storage that they already know how to manage. And that has been a really big value prop. But then the other side of it is they just want it to do everything and they don't care how it does it. And in that world, if we were going to release a more
00:55:39
Speaker
sort of shrink wrapped, you know, you press a button on the internet and 30 seconds from now, you have the atomic instead of your running processes on your own or whatever. Then, you know, how that worked, A, how that works under the hood doesn't really matter to you, but B, I would be much more likely to use something that was well established that you could point out and say, hey, rock solid storage provided by DynamoDB or whatever.
00:56:02
Speaker
Yeah. Actually, there is another little question that people get there is that if you want to move, because Amazon is the big, you know, the kind of gorilla in the room, the elephant in the room, whatever you want to call it, not the elephant, the gorilla. It's a big thing.
00:56:15
Speaker
I'm great with words. I've got the best words. So you've got these, the big Amazon guys, but some of my friends, they run an atomic, a company that runs on the atomic and they'd like to move to the Google cloud. And suddenly they're kind of like, oh God, nightmare or joyant or some of that. So they have to run Cassandra themselves or they have to run something else.
00:56:39
Speaker
Do you think about maybe this is not on the list of questions. What the hell do you think about potentially looking at other dynamo DB like storage service on like on Google or joint or Microsoft or other cloud providers. So we keep an eye on all of those things to to varying degrees.
00:57:01
Speaker
Those degrees are really determined by feedback that we get from users, which I should take the time to mention that just in the last six months, we rolled out a system where people can do feature voting.
00:57:17
Speaker
I think there was a question about that. You're killing a question as well. Now we can take care of a question as well. The question I think was, how's that working out? The answer is pretty well. I think that this is something that the closure ecosystem could actually benefit from as well, which is features and feature voting really ought to be totally separate from bug tracking.
00:57:41
Speaker
Right now, sometimes feature requests in the Closure ecosystem come in through Jira, which is just not where I want to manage feature requests, ideally. With Datomic, there's a separate system for doing feature requests and people can vote.
00:57:56
Speaker
And it has the important property that you have a certain amount of power, and you can choose where you spend your power. So you can't go in there and vote, say, I think all 100 things that people have asked for are equally important. Because that doesn't really add information. You have to choose where you're going to spend your capital. That's been helpful. We've probably knocked off a half dozen features that people requested through the system. We could definitely would appreciate more people going in there, especially things like I'm running on this
00:58:24
Speaker
this particular cloud or this particular storage system, and I could use the atomic if it was running there. We don't have enough people using it yet that you could draw strong quantitative conclusions. So it's going well, but we'd love to have more feedback.
00:58:43
Speaker
So on the dynamic itself, I remember maybe it was a long time ago, even on the mailing list, you said you're working on a book. So how is it coming along? Or is it on the horizon?
00:58:55
Speaker
So one of the things you'll notice consistently across the closure and the atomic ecosystems is we are very circumspect about providing calendars for the future or roadmaps because we've been writing software all our lives and we know how unrealistic those things are. As an experiment, just to prove to myself that roadmaps are incredibly terrible and painful, I decided that I would make a prediction on the calendar about writing an atomic book
00:59:23
Speaker
And I can tell you that from having made that prediction, which I think came in past like a year ago, that I have again been reminded of my lesson and will no longer make calendar predictions. I know what it is that I teach to it. I know what it is. Because you kind of gave us the clue where you're on in this conversation, which is that when you get a book, you'll go one zero with the atomic.
00:59:43
Speaker
And until then, you're not going to do it. That's right. That's right. It's all it's all it's all tied together. It is it is definitely I write about the atomic all the time. And it is it is my intention to write a cool book about the atomic that I will be really proud of and that hopefully will be very useful for people. And I'm going to make no more calendar statements because I am appropriately I'm appropriately chastened by the colossal failure of the last statement that I made.
01:00:11
Speaker
There was some other questions which I think we just stick on the topic for a little bit longer because we do want to talk about some other bits and pieces as well. But we're going a little bit longer today, but I think we can take that.
01:00:27
Speaker
because it's such a fascinating conversation. The other thing is, and I've been interested in this for a long time, is whether Datomic will actually ever support another language of peers, especially where peers run without the JVM. Will you ever run that on a JavaScript environment, for instance? Well, certainly language support for Datomic is an area that we want to be very user-driven.
01:00:51
Speaker
And so that's one of the things that people vote for on the site. I also think it's important, as much as I love the peer model, and if you've used Atomic, you may have also come to love the peer model. Since November of last year, we've had a more traditional client model.
01:01:06
Speaker
And I can imagine that in the fullness of time that the majority of Datomic users will be on the client model. Certainly in the client model, once the client wire protocol is out of alpha, that will be fully specified. And we will both work to implement clients and to support other people with technical resources implementing clients. So I would say that in the short run, if you're running not on Java, not on Closure, it's a lot more likely to have an awesome client
01:01:35
Speaker
than it is to have an awesome peer. Really delivering the peer experience is deliver the closure experience in another platform so you can deliver the peer experience. But we've kind of got that with ClosureScript and the CLR so I guess that's a yes then.
01:01:52
Speaker
Well, it's doable. It's not that easy. There's a lot of platform sensitive code in the peer. And there's not a lot of platform sensitive code in clients. I mean, it's an intrinsic trade off. And the peer provides a fantastic locality. And along with that locality, you're sort of wedded to the platform.
01:02:17
Speaker
So, um, I, you know, will there in the fullness of time be other peers? I don't know. Will there in the fullness of time be a lot more language clients? Almost certainly. Sorry. Go on VJ. Yeah.
01:02:30
Speaker
Yeah, so maybe we can talk about the recent ongoings as the, I think, as the last topic.
Clojure Community Concerns and Future Directions
01:02:36
Speaker
So there has been some kerfuffle on the internet saying collage is dying, blah, blah, blah, something and that kind of stuff. And then there is a lot of discussion. Of course, there are, as Ray was pointing out before we started recording, that there are some valid points or valid criticisms that collage can do better or collage community can do better.
01:02:53
Speaker
And we had Eric Normand on the show as well, the functional TV. And he also wrote a blog post about what Cognitec can do better to mobilize the community. So what are your thoughts on this one, this whole thing that is going on right now, the Closure Community?
01:03:13
Speaker
Well, there's a lot to say because this touches a lot of different things. First off, I would say that Eric's blog post was quite thoughtful and so people should read that. I think it was very considered on all sides. I think that it's difficult
01:03:30
Speaker
So first off, there are always problems and we should strive to learn from each other and from other language communities. And I think that one of the great things about the Closure ecosystem is that enthusiasts for Closure tend to come from very disparate technology backgrounds.
01:03:50
Speaker
And so throughout most of my career, every time I hopped from one language to another, I was going with a large cohort of other people going in the same direction as me. And when I came to closure, I realized that I was hitting this melting pot of really different cohorts of people coming from different interests and different backgrounds. And so because of that, I think there's, there's a potential to sort of, you know, you think you understand the whole elephant, but you really just have your hand on the elephant's trunk or on the elephant's tail. And so people don't necessarily see.
01:04:20
Speaker
you know, what the experiences of other people are. Just to be quantitative about the closure experience, we find that the vast number of people coming to Closure Conch
01:04:33
Speaker
every year are new, right? It's like almost 50% people who are new to Closure, new to the conference that year. So I think that that's healthy evidence that there is a large funnel of people coming to Closure. And I would suspect, if we could do the data analysis, that if you looked at the other Closure conferences, the regional conferences and other events that people are running, that those things tend to pick up more advanced users.
01:04:58
Speaker
and people who have been in closure longer that they know the overall ecosystem better. They have a better idea of what they want and what they're interested in. They're hunting by topic for what conference they're going to go to instead of, hey, I'm new here. I should go to the big one. And that's all great. And it's fantastic to see, you know, especially in Europe, all the little closure conferences that have sprung up, you know, in the last couple of years, but not just in Europe and other places as well.
01:05:23
Speaker
One of the anecdotes that was selected to make the argument that the closure is stagnant was that the mailing list traffic seems to have leveled off. And what has actually happened there, I think,
01:05:39
Speaker
is you know, it's kind of ironic because in one week you have this, the mailing list is leveled off is closure dying, followed by the number of closure users on slack are going to overwhelm slack's free tier and slack is going to fall over. And so what I think happened is slack. Yeah, I think I think what happened was, you know, slack got very trendy for a certain subset of the closure community. And they went there preferentially to
01:06:05
Speaker
the mailing list. I think it's great that the community is big and diverse enough, and maybe people have different ways that they want to communicate that people are in different places. So I think that from a numbers perspective, there are a lot of different proxies you could look at. You could look at downloads from Maven Central of closure, which just are on a sort of ongoing uptrend over time. So it's very difficult to interpret any of this information, obviously.
01:06:35
Speaker
including the Maven downloads, right? It could be one big company just started downloading in a bunch, you know, in a given month, who knows. But, you know, I see a thriving ecosystem in terms of the people that I interact with, and I see a lot of interest in,
01:06:53
Speaker
the growth of the tech, and in particular, the additive growth. I mean, I think the thing that's just stunning about Clojure is that it's had a series of things that felt revolutionary in terms of your experience. You had ClojureScript, you had CoreAsync, you had Transducers, now we have Spec, all these things that feel like, wow, this is a revolution in how I think about writing my code. But if you look at what they actually did to your Clojure programs, they're additive.
01:07:19
Speaker
These are all additive changes. None of those things I named broke existing Closure programs. And that's just amazing that over 10 releases that we keep doing that. And I think that as long as we continue to focus in terms of the tech on making that kind of additive change, that you're gonna see people who are building businesses around Closure appreciate that and say, you know what, I wrote code five years ago and it still works.
01:07:48
Speaker
And I think that the talk that Rich gave at the last conge about compatibility
01:07:55
Speaker
Speculation was the title of the talk. It's very well worth people in the Closure community understanding. I think we have an opportunity to show leadership there in how we manage naming and releasing artifacts. And I think that we could, pervasively throughout the entire ecosystem, build software that's more stable in the face of change. And I think that that will really facilitate people not just coming to Closure, but staying with Closure once they're there.
01:08:21
Speaker
So if you have to pick up something like based on different kinds of feedback from, as you said, when people are coming from different backgrounds, they have different priorities, they look at different things in the community. From being one of the core team member, and as the guy who is looking into every line of code that is getting into closure, what would be your first preference and what is the first thing that you would like to do better?
01:08:48
Speaker
So I think there's a couple of things there. One of them is that there's a difference between what happens to Clojure and what happens to the ecosystem. And so it's important to keep those two things separate in your mind. One of the rules for Clojure, and this should be true for any programming language, is don't make any change here that you don't have to make here.
01:09:07
Speaker
If you look at a problem and say, I could solve this enclosure, or I could solve this in a library, and there's not a compelling reason to solve an enclosure, you ought to make that solution in a library. And that goes further. If you look at a problem and say, you know what? We could solve this enclosure, or we could solve it in a library, or we could solve it
01:09:28
Speaker
with documentation or a convention about how we work as people in the ecosystem without writing code at all, we probably should solve it there. So we want to find ways to make the changes where they should be made in the right layer of the stack. The flip side of that is you want to make changes in the core that are going to be most impactful to the broadest set of uses. And you want to think generally and not in terms of one-off
01:09:55
Speaker
solutions to problems and band aids like that. So that's kind of some of the technical prioritization. But then the other thing is to realize, you know, it may be the case that the biggest problems the closure ecosystem is facing don't require language to changes to closure at all, but they're still worth working on. And so, you know, one of the reasons that I'm focusing on the closure experience at the REPL right now is that
01:10:20
Speaker
My experience doesn't gel with some of the concerns other people have. And I want to explore that and say, look, when I encounter the problem you're describing, here's how I would solve it at the REPL. And I want to have that conversation before I would make a change to closure.
01:10:35
Speaker
And that conversation might lead to me realizing, you know what, I have experienced blindness here. And so I don't see the thing that you're seeing. And once I can see it through your eyes, then we can do something different. But I would, certainly, there's a recurring theme of documentation could be better.
01:10:57
Speaker
What is the most powerful thing one person could do to make documentation better? I could write documentation. That's the thing that I could do. Or the core team could say documentation is now managed via pull request on GitHub, which it was not until a year ago when that change was made. So that is a much higher leverage change than any specific piece of documentation I could write.
01:11:21
Speaker
And it opens the door for a much larger group of users to come in and make needed improvements to documentation. And so that's the kind of change, the kind of higher leverage change. And it doesn't require a change to the language. Or pointing out to someone, you've encountered this problem and you have developed a patch to closure that would solve this problem. But here's a solution that could be done in a library.
01:11:48
Speaker
that doesn't require a patch to closure. That's a much lower bar. It also provides a lower cost way to vet the solution and try it out in production for a year before we make a change to closure in the same area. So those are some of the kinds of things that I would look at in terms of priority setting.
01:12:04
Speaker
The other thing I was going to ask you also Stuart was because you were in a company as well, you're kind of eating your own dog food when you go out and do consultancy gigs. I think what some people are looking towards, and we see this in some of the questions actually, is
01:12:22
Speaker
what kind of like tools or templates or libraries or those kind of like power packs if you like do you guys put together on consulting gigs that might help other people have success is that something i mean it doesn't have that's definitely not a change to closure and it's not even a change to closure.org but is it something which you know your your kind of experience as a consulting company could help to to kind of
01:12:50
Speaker
Give people a better leg up or a bigger lift and leverage in terms of you know what you're doing real world in terms of your consulting assignments. Yeah, that's a good question and let me take it in a one-off way and then in a.
01:13:07
Speaker
maybe sort of a step back kind of way. So one thing that Cognitec test produced that is a power pack that's grown out of the consultancy building things is Pedestal. Pedestal is not the dominant way necessarily people build web applications and closure. It is the dominant way probably that people at Cognitec
01:13:27
Speaker
currently build web applications and closure but that's not a mandate right this is this is we're up in toolkit land and so different projects are going to have different needs and different mandates but this is a concrete example of something that's being battle tested in consulting engagements and then
01:13:43
Speaker
As a second thought, given away as open source and documented and reaching out to people and trying to sort of help people adopt it where it would be relevant. It's not necessarily going to be a one size fit all solution. On the other hand, it was devised and has a rationale for specific problems that at the time it was created, other Closure Web Toolkits didn't cover.
01:14:05
Speaker
And so it's a concrete example of that and just sort of, you know, does Cognitect, how could Cognitect share, you know, a power pack of, you know, things that we have, you know, built to solve problems? Okay. I think there's a general, I mean, I don't know. I think there's a general feeling that the pedestal hasn't got the kind of adoption in the community that maybe was foreseen. There could be reasons behind that. You know, I've, but we
01:14:38
Speaker
I guess the question is, if that's what you're mostly doing, then that's what you're going to push and that's fine. I guess there should be other consultancies like Juxt and people like that. They're also pushing their toolkits and that's all fine also. I think there's a higher level of value that we can add though, which is that we are in conversations probably with more big
01:15:02
Speaker
stodgy enterprise companies using closure than almost anybody else. I mean, we try to talk to everybody out there. And so I can relay some feedback from there. And I've been doing this over the weekend. If you've been following me, tweet over the weekend. One of the challenges that we have is that we sometimes make things sort of too much for ourselves. And the way that this shows up, this is the symptom and not the cause,
01:15:28
Speaker
is the lining and plug-inization of valuable tools. And so people say, well, you know, when I make a tool for the Closure ecosystem, the first thing on my documentation on my readme should be, here's the line plugin for using this. And they justify that by saying, you know, look, 90% of Closure projects per the Closure survey use lining in. Now, these things are all true. But, you know, we're going out and evangelizing to the non-converted. And let me tell you, the non-converted do not use lining in.
01:15:57
Speaker
And the non-converted who have money to spend writing software, we all know what they use. If they're in the JVM ecosystem, they use Maven. And so I have been reaching out to individual closure projects and saying, hey, look, can you please document your library story for a Maven user?
01:16:15
Speaker
And lead with that, I mean, they may not lead with that, and there are reasons not to, I understand that there's a, and this is not about lining in or even maven. It's really about, we are producing tools that are usually kind of functional tools, and so I want a functional entry point. And having a lining and plug-in, lining and plug-in, or a maven, a maven plug-in would also be wrong, by the way.
01:16:37
Speaker
A Maven plugin would be wrong for the same reasons that a Gradle plugin would be wrong. The right way to do it is to say, here's the jar file, and here's its Maven coordinate in Maven format, and here's its Maven coordinate in Liningen format. And we definitely have projects where we are prohibited from using Liningen.
01:16:58
Speaker
And we have projects where we're prohibited from using closures. So another thing I would say to people is that if you're trying to facilitate enterprise use of closure, I understand that it's more of a pain in the neck to put things in Maven Central.
01:17:12
Speaker
There's a higher bar there. But if you're out there saying, I want to do open source and do a favor to other people, then if you put things in Maven Central, they are going to be able to reach a much broader set of users than just the Closure ecosystem as it exists today. So those are kind of higher order feedback on the topic of how do we make a better power pack? We can make the stuff we have into a better power pack by making it more consumable by people who don't necessarily share all of our presumptions about how we build stuff.
01:17:43
Speaker
Okay, then one final question then. I think we can, I think we'd have almost one and a half hour. Wow, it's like one of those longest, but of course very informative and it's a pleasure when we can continue talking for hours and then I know, you know, it's a never-ending things in the enclosure world. But we'll just make it an annual thing, Stu, okay, from now on. Exactly. Standing invite.
01:18:07
Speaker
Sounds good to me. Rich Hickey is in some sort of a witness protection program or something. He seems to be like the guy for everything and then people are kind of concerned that what happens, what is the future of the closure when it has a bus factor of one. So the political party that is worried about
01:18:28
Speaker
Closure, if Riticki's gone, must be like the opposite faction from the political party who's worried about that closure is dying or that they don't like how things are going right now. So I suggest that we let those two political parties fight it out and the rest of us get back to writing software. 100%. 100%. Awesome.
01:18:47
Speaker
Okay. Well, well, no, it's just because it's been a great conversation. I think I want to let Stu make it on with his day though. And I think so. I think we I think we've taken up a hell of a lot of time and it's been great, a great time.
01:19:03
Speaker
Yeah. But we would love to have you back on the show, by the way. I mean, we still need to pick your brain on a lot of other things as well. I know it's a one and a half hour is not enough. And one thing that I noticed is that you're extremely good with names. I think probably that's because of your historian or something. I don't know when you're very specific about people's names. So usually when you're in a conversation, people start with, oh, I'm not good with the names, that guy there and that girl there. Well, I appreciate that. I mean, I try to I try to be precise
01:19:32
Speaker
I think it's, you know, I mean, we talk about closure, you know, valuing simplicity and certainly that's a distinctive thing. And, you know, Rich gave a great talk about it, but I think that, you know, valuing kindness and respect is another thing.
01:19:49
Speaker
that is very important in the closure community and that we try to lead by example in the forums that we started, things like the mailing list. And I think that's been a very important part of closure success is people being able to expect that they're going to be treated with respect and kindness in the interactions. So I think that's another important thing. And I would close by mentioning, because we should have mentioned this at some point,
01:20:17
Speaker
this is going to be the 10th conge this year. So this is going to be an event of extreme splendiferousness. And I'm super excited about it. I think, you know, it's going to be, you know, the announcement is going to come out sometime in the next couple of weeks. I'm not sure how long between this recording and when this publishes, but maybe it'll even be out by the time the podcast is out. I think I'm probably going to be doing a
01:20:45
Speaker
day-long workshop on making datomic also in the cloud so we didn't spend a lot of time talking about that today but something that has been dominating my thoughts over the past year and I'm eager to get out and talk about it with other people so hopefully we'll see some people there certainly hope I'll see the both of you at conge if it's possible
01:21:05
Speaker
And if not... We could be media sponsors, okay? We'll have to find a way about that. That's right. We'll have to work that out. Yeah, yeah. As long as we get best passes when it comes to show up. Yeah, yeah.
01:21:17
Speaker
We'll talk about the off-ass too, okay? Great. But I think it's... On that... Bombshell? No, I think we can, yeah. Well, I expected a bombshell, but it was much more like a proper way of... As you said, you know, closure community, the community thing is one thing that I really, really like about closure.
01:21:42
Speaker
the way people being kind to each other and then being good and then doing good. So thanks a lot for joining the podcast. It's almost one year and it's been an absolute pleasure for us to talk to different people at Cognitec and different people in Closure World. And we've been learning a lot talking to all these people. So this is something that we really appreciate and it's been almost one and a half hour and we'd like to wrap it up now.
01:22:07
Speaker
And thank you Stuart Holloway. I don't know if I can call you Stu. Stu is fine. Stuart is fine. Whatever. Perfect. Thanks again. And we'd like to wrap up by reading some credits. So the music is by Pizzeri as the intro music and the outro music. We might actually change the music this time, you know. This is a bit post hoc, but we might change it for a special. Still made by him. Made by him, but a different track maybe. So signifying. Exactly.
01:22:36
Speaker
the regal presence of Mr. Holloway. Oh, you're really embarrassing me now.
01:22:44
Speaker
to honor this special episode with a special special number starting the episode and our artwork is done by Lubov Sultan and you find the links in the technical notes and of course we'll add a lot of links that Stuart will send us in the show notes and that's it from us for episode number 23 and this is Vijay signing off. Yeah likewise Ray. Thanks again Stuart it's been an absolute pleasure. Cheers.
01:23:13
Speaker
Thanks very much guys, have a great afternoon.