Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#15 EuroClojure 2016 Interviews (Part 2 of 2)
42 Plays9 years ago
At EuroClojure 2016, we had the pleasure of talking with these fine folks:
Malcolm "Yada" Sparks
Dragan "The GPU Tamer" Djuric
David "The CLojureScript Guy" Nolen
Recommended
Transcript
Episode Introduction
00:00:17
Speaker
Welcome to Deaf and episode 14 part 2 or 15... I don't know which number it is? 14A. 14A? Well, 14A is supposed to be the previous one. 14.1. This is 14.2 then. Oh, we started 0, so 14.0 and this is 14.1.
00:00:36
Speaker
We'll have to see. When it goes to 20, we'll stop this. I think once we get to 20, we'll just say the next episode of DeafN. Because we're lazily loading these episodes, so we don't care about the indexes anymore. Because it's an infinite stream of things. Exactly, we're just mapping over it. It's a seek. Lazy seek.
00:01:00
Speaker
Okay, so welcome to DefN, whatever the number of episodes this is. But this is a special episode because we don't have any guests or this is not even a alleged it's episode but this is a special episode continuing on the previous edition of this part of this episode.
Euroclosure Interviews Overview
00:01:22
Speaker
go on don't interrupt me i'm trying to find words it's a trouble yeah this podcast shit okay go for it go on carry on yeah so uh this in this episode we have uh three new interviews that we did uh well not new technically because we did that at euroclosure
00:01:40
Speaker
And we thought we couple them together and then release it as an episode and in this particular one we have three interviews. The first one with Malcolm from Jext, who is the author of- Malcolm Sparks, I think we should give him his full name. Yeah, yeah, Malcolm Sparks. Yeah. Okay.
Interview with Malcolm Sparks on Yada Framework
00:01:58
Speaker
Malcolm Sparks, and he is the author of the Yada framework for web framework, web application development. And we got to talk to him at Euroclosure. And then we spoke with Dragon, Juric, I think.
00:02:20
Speaker
He's the author of Neanderthal. He gave a talk about how to bring, sorry, the closure is not afraid of the GPU. That was the talk name. And he was explaining how Neanderthal works and all the magical GPU stuff that you can do with that closure. We had a long, interesting talk with Dragon.
00:02:40
Speaker
You know what was funny about that was that it actually broke into two, didn't it? And I think he found out halfway through that we were recording. Well, didn't he find out that he had, he got some confirmation or something about his professorship. So he got tenure. So that was like a momentous thing. So during our interview, I'm sure, I'm sure he'll remember it for that if nothing else.
00:03:10
Speaker
Yeah. And then hopefully we'll get him. Well, he already promised that he'll be doing a full episode with us, so we'll get
Future Plans for Interviewees
00:03:16
Speaker
to. Yeah. Maybe we should say that actually, that hopefully all of the people that have appeared so far, we were hoping to have back for a prolonged interview because these were only like 15, 20 minutes of interviews. So we'll hopefully get them all back at some point in the future for a longer, lengthier conversation. Yeah. Yeah, of
Interview with David Nolan on ClosureScript
00:03:36
Speaker
course.
00:03:36
Speaker
And finally, we had David Nolan as our last guest for EuroClosure, or at least last guest interview that we're going to make up, that we're going to release. And we got to talk with David and all sorts of things about ClosureScript and his work at Cognitect and ClosureScript Community. So we were very grateful for all these speakers who
00:04:04
Speaker
spent their time with us at Euroclosure and especially a special thanks to Lynn at Cognitec. She gave us support to set up our own small studio and everything and record all these things.
Expanding to Video Content
00:04:17
Speaker
Yeah, thanks Lin, that was really great. Yeah, and we hopefully get the videos processed pretty soon because we not only just audio and Defund is now moving to kind of talk show level I would say for at least a brief period. Maybe it's not the talk show though. Maybe it's a slightly better production values than we have.
00:04:42
Speaker
We have to start somewhere. But we hopefully will release the videos soon. We'll publish them onto our own YouTube channel and we will announce them on Twitter, on the tubes and everywhere else. All these interviews have videos as well, so you could actually see who is speaking.
Closing Remarks and Sound Quality
00:05:02
Speaker
We did have a nice idea that we would swap our
00:05:05
Speaker
voices on on the video but I don't think we still might happen it still might happen yeah we don't know just to confuse people but anyway um so enjoy the episode and um we will get back to the regular schedule from our schedule I don't know american versus british thing um let's go for schedule yeah schedule is the british one right well you're repeating here yeah yeah of course
00:05:37
Speaker
The other side of the pond is pretty cussed up right now. Let's not go there. Let's just not go there. Stay in the happy closure place. Parenthesis, parenthesis, parenthesis. Yeah, let's live in the parenthesis. Yeah, I just want to add that there is a little bit of a sound quality issue on the recordings because we only had one microphone and we're passing a microphone around from person to person
00:05:54
Speaker
We are on this side of the pond where things aren't right.
00:06:04
Speaker
So sometimes you hear little clicks on the microphone or maybe sometimes you hear people not speaking quite so loud as you'd like them. So apologies for that up front. We still think that the vast majority of the quality of the recordings is pretty excellent.
00:06:21
Speaker
Anyway, enjoy the episode, enjoy the interviews, and we'll get back to the regular schedule from episode number 15, I think. We'll make it up as we go along. Exactly. Just like the rest of this stuff. Yeah. Okay. Cheers, guys. Cheers.
Malcolm Sparks on Closure and Juxt
00:06:42
Speaker
So, Malcolm, welcome to Deaf and Talk Show, Euroclosure Live Edition. So this is the first time we're trying to record video as well, so it's going to be tricky. We hope it's going to turn out okay, but we never know. Anyway, so we spoke to John yesterday. We had an interesting discussion with him about his experience with closure and he told us that
00:07:06
Speaker
You were the one of the guys who made him start Jexxed and pull him into Closure. So can you give us some idea about how did you start with Closure and Jexxed and what are you working on these days? Some ideas about your projects.
00:07:21
Speaker
The two of us were in separate banks and we didn't know of each other's existence but we had very, very similarly named blog articles and so we sort of bumped into one another and it was a time when we later on and when we both left the banks we decided
00:07:41
Speaker
or you didn't want to leave Closure. We had Closure teams, what do we do? And we kind of discovered Closure was a really secret and a very effective weapon to solve lots of the problems they had in banks. So we thought that might be generally applicable elsewhere. And that's why we started Juxt.
00:07:58
Speaker
So now you're kind of doing jokes, and I think that's a great thing of getting more, putting closure people in touch with the needs out there. But the project that you've been working on kind of mostly recently has been the YADA project. So it would be really good if you could give us just like a background of it, what motivated you to do it. What's the name? Yeah, yeah, what's the name? Tell us everything.
00:08:26
Speaker
And you got five minutes, go.
00:08:30
Speaker
Well, I mean, a lot of our projects in Juxt are of similar architectures. We love APIs. We love the integration of using the web stack and how that's driving a lot of the economy now and the making integration between companies so much more viable. And so because it's such a common thing that we do, we want to be able to industrialize it and give people the ability to create APIs super quickly. And we think closures are
00:08:57
Speaker
a great platform to do that on. I'm a bit of an HTTP geek, I've been fairly disappointed by... Well, I like to implement standards properly and I feel HTTP is such a big standard and it's almost impossible to ask
00:09:13
Speaker
developers and organisations to implement all of HTTP. In fact, they should be thinking about their application domain and not be protocol network engineers. And so Yada and similar libraries like Liberator are trying to bring web developers kicking and screaming halfway to HTTP purity and also to exploit all of the advantages of HTTP that many people don't know.
00:09:40
Speaker
So that's the point of YADA, to make it quick and easy to implement HTTP, but in a way that doesn't sacrifice performance and scalability. So that's been why YADA's been taking me a long time to develop. I've been working on it almost two years, and before that, thinking about it a lot. But it has been a bit of a labour of love for me, whilst doing other project work.
00:10:05
Speaker
But it's true to say that a lot of our projects are in the trenches and that's really the best place to, you know, necessity is the mother of invention. So when we find that we need something and we build it and we think, well, we need something like this in a more general place, we kind of go off and write it. And so that's really where the libraries come from.
Yada Framework and HTTP2 Standards
00:10:28
Speaker
Maybe you probably don't want to caught too much controversy, but what the hell? What was wrong with what was already there? And we can think about other languages or other stacks, but what's wrong with the kind of stuff that's already there in the closure system that you wanted to improve with Yara? What was the kind of
00:10:49
Speaker
the motivation for just doing something new. I think a lot of libraries really let you start you at this CGI interface where you get the request and then your job is to create the response.
00:11:07
Speaker
So there's not a lot of support for doing HTTP fully in content negotiation, conditional requests, caching, ranges, partials. There's whole loads of things in the HTTP specification. It's so hard for people to implement. So that was the real motivation to try and
00:11:23
Speaker
Rather than create it in a modular way where we pick and choose which bits of the specification we want, I really try to wrap the whole of the specification in one library so that you just use that library and you get HTTP asynchronously.
00:11:51
Speaker
Before creating Yala, I assume you're using it already in your projects right now. Before creating it, which stack were you using? The projects that you're using, can you give us an example of a stack? What is the database and what kind of solutions that you propose to your clients?
00:12:08
Speaker
Well, before YADO, it's not as if we're using YADO in all of our projects and we're maintaining other projects that are before. We used a whole mixture of things. We've used computer API from Tommy Riemann. We used a lot of pedestal actually and the pedestal interceptors. We've used plain ring and other things too.
00:12:29
Speaker
and Liberator. And it's partly that that was kind of the drive of actually using all these different libraries that did different things. It was kind of, well, why can't we just have one library? And so I just built another one. So now we have even more. But that was really the coming off a project where we had so many different ways of doing things. I thought we just need to consolidate because otherwise we're just going to have too many of these things.
00:12:57
Speaker
I mean, HGP itself, especially from version 1 to version 1.1, has been a static protocol, a static standard for a long time. But you'll know that it was recently agreed that there will be a new standard called HGP2.
00:13:17
Speaker
Google have done a lot of work on this, it's being adopted by the browser manufacturers as well. Is this something, I'm kind of springing this on you but what the hell, is this something which you're kind of thinking about or is this the kind of thing that you're going to wait a little while until other stuff settles down or infrastructure gets good or what's your feeling about history too?
00:13:40
Speaker
There are aspects of HTTP2 which are more at the network level and I know the support for HTTP and Neti which is the server that YADA is built on through Aleph.
00:13:55
Speaker
So that is the stack, and in things like HTTP2 requires, or pretty much requires HTTP S and transport level security. But things like server push as well, that's at the network end. So there aren't too many things that have changed in the semantics and content of HTTP that are that different in HTTP2.
00:14:21
Speaker
So I don't think it's going to be a lot of work to adopt it from what I've read of it. I noticed from Google's side, for instance, they have this GRPC and that seems in terms of implementing
00:14:38
Speaker
web socket style interactions because obviously the problem with web sockets was that it was nice, it was a bite, you suddenly went to TCPIP, you got a multiplex and you got very nice speeds up and downstream but the problem was that you kind of lost a lot of protocol, you lost a lot of visibility and everything kind of went away.
00:14:58
Speaker
So it's a valid thing to do because of firewall friendly things, web sockets. But it seems like HTTP2 will standardize that a little bit. Is that something? Because that definitely seems to be changing the way that you program it at that point. Or maybe these web sockets aren't interesting at the moment. I don't know.
00:15:16
Speaker
I mean WebSockets is where you leave the HTTP protocol and so you upgrade and you switch and much of the impetus is to have very fast low latency for message passing for games and so on.
00:15:32
Speaker
Yeah, so I've tended to ignore WebSockets and seeing that as a network upgrade, but I want to concentrate on things that are in the HTTP and HTML space, which is things like server-sent events.
00:15:47
Speaker
and server push in HTTP2 makes that the pushing of content makes for a very fast and performant latency experience on the web. So it's possible that you can build very interactive experiences without requiring web sockets and with the advantages of all the security or authentication authorization that you have in already built in HTTP and years and years of
00:16:16
Speaker
known security and deployment issues that have been fixed. So I think there's a lot of value in HTTP and allowing more people to stay within the fold is valuable. So when people are writing their HTTP APIs, they often want to do documentation around them and people using stuff like Swagger and stuff like that. Is that something which you integrate with as well?
00:16:43
Speaker
Yada does swagger out of the box. You can just say, OK, I want to take this API and I want a swagger document. And because Yada is very much declared in data, the swagger transformation to the swagger description is just really a function transformation.
00:17:02
Speaker
Yada makes use of Ring Swagger, which is Mettison's library for doing transformation between prismatic schema and the JSON schema format that's used by Swagger. Yada has always had support for Swagger. In fact, it was one of the first features I implemented when it was experimental.
00:17:25
Speaker
I have mixed feelings about Swagger, it's pragmatically useful but HTTP is really about dynamic discovery and the point is to allow a user agent to discover
00:17:42
Speaker
and new things about if the server does change, that through a discoverability and a solvability, the client can say, aha, I know there's a link called foo, but now it's pointing at a different URL, I'll follow that. So it's really putting up signposts for the client to navigate what's new about a server. And that's not a new idea. Humans do that on Facebook. I mean, when Facebook do a new release, I think they do a lot of new releases every week. They don't have to notify everybody that there's going to be a new release of Facebook.
Challenges in REST and HTTP Discoverability
00:18:10
Speaker
You just cope.
00:18:11
Speaker
with it and so those coping mechanisms are built into HTTP by delivering links, by negotiating media types and so in that world it should be unnecessary to have out of band documentation through swagger.
00:18:27
Speaker
However, there's an interesting thing that one of our jugs associates, Oliver Hine, is doing, which is using the Swagger documentation as a media type in its own right, actually downloading it in clients and being able to utilize it to allow clients to call links and discover it. So it's certainly not a kind of clear black and white where Swagger is out of band. It can be used very effectively in band, which is quite a new and novel idea.
00:18:57
Speaker
do you think probably the whole swagger thing and all rest API thing is somehow developers want to have a let's say an easy thing rather than a simple thing or you know they definitely want something where they can grapple with and the most popular APIs have been fairly static and even from Google and Facebook and Twitter so
00:19:22
Speaker
No more philosophical memories is a final question because I really like you know, we really want to extend this conversation, you know During the podcast proper if you'll be happy to do that But it would be nice to have a little philosophical question about you know, why is that such a problem? Do we think you know why is HTTP itself actually is kind of like a
00:19:43
Speaker
Failure is too strong a word, but why has it failed to sort of rest and the whole concept failed to kind of grasp this Why have the developer community failed to grasp this whole notion of this discoverability because I agree with you But I also see it that in reality it just hasn't worked
00:20:00
Speaker
I think it's really the answer to that question is more cultural. We want things now, we want our systems to be delivered in three weeks or yesterday and the fast food generation and we don't want to wait and so you get that instant gratification if you can get a document and hard code a URL for a list of users or something, you hard code that and then of course as soon as you hard code URLs then if
00:20:28
Speaker
the server then changes, you've broken. So you're actually, by adding discoverability into your API, you're paying it forward. Where you're getting the payback is in maybe one or two or five years down the line. But I think it's because there's so much emphasis on doing things now, we tend to pay for it with compound interest much later on. So is this a perfect case for mortgage-driven development then?
Juxt Hiring for Closure Projects
00:21:02
Speaker
Just a quick final comment, though. Obviously, you're the sponsors for the conference. Thanks a lot for that. And I'm guessing you guys are hiring, probably. So, yeah, we'll try to put the word out there. And you're primarily doing just closure, right? Yeah. So we'd love to have you on a podcast, if you're interested, because I know that combined Just Associate experience and closure is way beyond the roof.
00:21:12
Speaker
Yeah, very much so
00:21:28
Speaker
So as we were talking to John yesterday, he built this stuff a long time ago, three, four years in the closure world. It's pretty long, long time ago because I used to follow his blog on Closure at a Bank and Closure at a Publishing House and all that stuff. So we'd really like to have you on the show and thanks a lot for doing this by the way. Yeah, we'd love to come along and we'll do a nice long podcast next time.
Dragon Juric on Neanderthal and GPU Integration
00:22:02
Speaker
Okay. Welcome to Deaf and talk show edition, live from your closure. So Dragon, right? Dragon. Like Dragon? Okay.
00:22:21
Speaker
Oh, sweet. It's a very, very, very beautiful name, by the way. Yeah. So you just gave a talk yesterday about the closure is not afraid of the GPU. And yeah, that was an amazing talk, by the way. So can you give us some background on why did you start this library? And you know, and what is your experience with closure when you start the closure?
00:22:49
Speaker
So I've been using enclosure for a long time, since 2009. And since I work at university, it was mostly some of my research ideas and you need lots of flexibility, lots of
00:23:05
Speaker
freedom to experiment, to not commit to a specific architecture or specific typeset or whatever. So I've been doing some exotic things in Java and I needed aspect J or some specialized libraries in, let's say, the beginning of 2000s, the middle of 2000s or so.
00:23:28
Speaker
But it was always like you wrestled with it for months and you're trying to create something that is you don't have a place to look at literature. You have to find your way, but the language, the platform is great, but the language is quite rigid for some such stuff. But it worked kind of or so, but I always tried to implement some dynamical stuff and was
00:23:58
Speaker
lots of time with it. So when Clojure was released, so it was, I think, the beginning of 2009 or so, I was regularly reading the most popular hacker news or so. So I mentioned that there is an increased interest in Lisp.
00:24:23
Speaker
And before that and when closure appeared, I tried it immediately and it clicked. Yes, that was what I was trying to do, but in a much better than I would do this. So I converted almost instantly and started using it exclusively.
00:24:43
Speaker
So for years, I was just using it casually. I had some ideas, mostly in machine learning and some Bayesian stuff. But in the Java days, it was really difficult to see how to implement this in a way that you can explore, experiment with this, because the area itself is a bit difficult to get into to experiment to see what works, what doesn't. And I think closure is a great match for that.
00:25:13
Speaker
So then when I needed to implement this enclosure, it was like, okay, closure is flexible. I have a good idea how I would implement it. I created some prototypes or so.
00:25:26
Speaker
But I realized, okay, Java is fast, but now when I really have big demands for really large computations or so, now it is too slow. So basically when you put on paper what is the number of
00:25:45
Speaker
iterations that you need to do, what are the numbers or so you realize your algorithm is good, flexible, and nice, but everything, but the toy problems would run in minutes, hours, weeks, months, years, or whatever. So then I started investigating about, okay, but there are people, C++ and everybody heard about GPU,
00:26:15
Speaker
So there are people getting some speed increase with this stuff, but how it's done and getting be done in closure. There were some experimental libraries, but not working. They didn't work that well. So I had to...
00:26:32
Speaker
Now you go down the rabbit hole, you fix one problem and then you discover 10 other problems. So I realized that the libraries that were available at the JVM at the time, even some excellent featureful libraries,
00:26:50
Speaker
Are actually not what i need because they always try to satisfy the corporate java world to be java center to hide these these high performance features from the user like to protect him but the in protecting the user they just.
00:27:08
Speaker
take away lots of performance and lots of flexibility and add many complex layers. So if you need to debug something or to fit it to your own needs, then it's really difficult. So I realized I have to provide some
00:27:26
Speaker
So you're working on this Neanderthal library and that is part of your whole uncomplicate umbrella. So can you tell us something about the library? What are your plans for the library and why did you start this
Neanderthal's Goals and Community Reception
00:27:41
Speaker
thing? Is there a commercial project or something or is there something that is a use case that you're going after?
00:27:50
Speaker
Well, so basically I have some ideas of how I would use Bayesian machine learning or data analysis commercially, or some of it would be open, some of it would be part of some service, some long-term plans.
00:28:14
Speaker
Basically Neanderthal is free, open source, works really well. I have some feedback from real users who are excited. It works really well for them. And I'm on this long time, so I have such luck to have freedom.
00:28:35
Speaker
to not be a startup that needs money and have some short run a runway or Have to provide something in two months, so they need to rush it to just work somehow But don't be really well polished or so so I have time to dedicate to this so I
00:28:57
Speaker
Sometimes I have something working well, but I realize that there are some points that could be improved and I spent two or three months just
00:29:12
Speaker
to get it really right and That's why I think it could be really useful for a lot of people because I will certainly develop this for years so basically and I also this is not the only stuff I work on so basically I Completed the stuff that is the most important that I needed and
00:29:35
Speaker
There's stuff that could be implemented relatively quickly. It's ready. The infrastructure is there. The native libraries are there. I know how to connect to it. I know how I would implement that. I didn't want to rush it because I didn't have the concrete use case.
00:29:54
Speaker
So it depends on the interest of people. So if I see that people want, okay, I really need this functionality. I need for my application, especially if they can help or provide some code or contribute some funding so we can develop something that is, for example, more polished or more documented or whatever.
00:30:21
Speaker
So there are many opportunities on that. So basically, if you use this, you don't have to be afraid that I will forget about it in a month. Just on that subject, you're a professor, so are you thinking about
00:30:42
Speaker
any of your PhD students or anything like that interested in fields where they can use this kind of technology and maybe also contribute and add to it or is that the sort of thing that you're looking for?
00:30:57
Speaker
Well, there are some PhD students experimenting with this and some master students experimenting with this. But I think the point is that I don't think at this moment they can help much because all of these people have their day jobs. So they cannot dedicate too much time to this.
00:31:24
Speaker
Yeah, of course, but the point is I don't run away from coding. I actually like coding and I'm not some professor who have 10 PhD students and then just have some bright ideas and then just put it to the workers. I do everything myself and I think that's why Nyan.com is quite
00:31:51
Speaker
easy to use because I use it myself. So I create this for myself, so that's why it should be easy for developers. So that's it. There was one question I was asking before, which was about the tool chain for all of these things, because whenever you get into this really high performance computing stuff, when you're going bare metal and all this kind of thing,
00:32:18
Speaker
Well, actually, maybe there's two parts to that, actually. It's like, I guess we're really talking bare metal here, and virtualization is not a great idea. Or are you kind of on the fence about that? Obviously, Amazon runs virtualized. But then if you get that core CPU, whether it's virtualized or not, people have to often include native libraries and all these kind of things, which makes the tool chain a bit more tricky for standard closure developers.
High-Performance Computing Challenges
00:32:45
Speaker
How do you think that can be addressed?
00:32:49
Speaker
I cannot give specific answer because I don't use it myself on containers. But I suppose because let's say if we counting developer machines, let's forget about virtualization at start. The thing is it's one of the main values that Closure adds here.
00:33:13
Speaker
I think it's pretty much easier than using this stuff without closure. So for the native part on macOS, for example, you just need to have your development machine and Neanderthal will work out of the box. So you don't have to do anything special if you have development environment set up for Macintosh programming or so.
00:33:42
Speaker
So it works out of the box as any other closure or Java library.
00:33:47
Speaker
On Linux, you have two options. You either want it to be optimized for your machine. In this case, you have to compile a Atlas native library. It's automatic, it's not so difficult. It's a make and standard C stuff. But it could be a problem for people who don't really, didn't really do that. So there is documentation, it's clear,
00:34:17
Speaker
Some people might be afraid of it at first. But luckily, I think all major distributions provide Atlas as a package. So maybe it will run at 70% of speed or 90% of speed or 40% of speed depends on how your machine is close to their machine where they compiled it.
00:34:38
Speaker
But it will still be much faster than Java stuff. And the point is, there are lots of algorithms in native libraries. You don't even have them implemented properly in Java.
00:34:53
Speaker
So that's one door where we can get amazing functionality like decades of development and optimization basically for free with no overhead, with some pleasant API, with lots of functions that feel like closure. So you don't lose much.
00:35:17
Speaker
And there is a third part which is actually the most awkward, it's Windows because I don't use it for a long time. Actually I don't even use desktop, I use Window Manager.
00:35:34
Speaker
And I don't have a mouse. But the point is, on Windows it's a bit more tricky but now it's not difficult because if you send me an email I will send you my libatlas.dll. So it's not a problem. And you can also build it yourself. This is the native part.
00:35:54
Speaker
So if you want to really put it on a server, I guess virtualization or so, I guess you will see how native libraries themselves work on that. And I suppose because all software that uses those
00:36:13
Speaker
also works on virtual machines. So I don't see that there could be any issue there. Maybe you need to configure something in VM or do some stuff, but there is documentation people already do this. As for the Neanderthal stuff, if JVM works, it will probably work. The GPU stuff is a bit different because
00:36:40
Speaker
To use GPU, obviously you have to have drivers for that GPU. So it depends on the vendor if you use Intel, if you use Nvidia or AMD. It depends on which operating system you have, how stable the drivers are. But pretty much it's useful today.
00:37:01
Speaker
So depends on your specific case, there could be more or less some need to experiment. But basically, it works. It works well. It's not as stable as standard server software for CPU, obviously, because it's exotics hardware. But if Google uses it and don't have any problem, I suppose that it's mature enough that you can
00:37:29
Speaker
create a new novel algorithm or sell for it. I'm pretty sure you're right. I'm just thinking about talking about Google there.
Dragon's Views on Machine Learning
00:37:39
Speaker
Some of the stuff they're doing in terms of, and this whole, you tell me whether this is in your kind of space, is the TensorFlow and the OpenAI stuff, is that the kind of thing which you're hoping to target on some of these activities?
00:37:58
Speaker
Well, it seems that you're really well prepared because you're asking the right questions that I would love to answer. So, have you prepared for this or you're just improvising as well as I do?
00:38:23
Speaker
No, the questions are really the right ones. I couldn't think of better questions. The thing is now, as I see, I don't have any connections to Google. I don't know anyone who has some important position in Google, so I don't know what do they actually plan or think or whatever.
00:38:46
Speaker
But what I can see and probably what you can see is that TensorFlow and deep learning is now like the most popular machine learning stuff that 99% of people who are looking into this stuff, they are like looking okay, deep learning, I can create some terminators with it or some kind of
00:39:09
Speaker
some kind of artificial intelligence that will work while I'm sitting on the beach or lying on a beach and just taking sunbed, but I don't think we are close, even close to that. The thing is, yeah, the thing is, I think Google and Facebook and Nvidia and those powerful companies, they,
00:39:38
Speaker
Now they have several use cases that work really well, like image recognition or so. It worked really well, but mostly to be successful with that you have to have lots of data.
00:39:57
Speaker
And that's great. I think many more use cases fit into it, but not all use cases. So me personally, I'm not very interested in deep learning because
00:40:10
Speaker
I think there are lots of people working on that. There will be lots of good results. There are lots of good results. It's working fine. Let's see what the hype will produce. And it's not only hype. Lots of useful, real stuff is there. But there are also cases, especially with smaller companies with exotic use cases or so, when you don't have big data.
00:40:37
Speaker
You have small data, like you don't have 100 billion photos, but you have 100 data points about something. And Bayesian methods are actually good for that kind of problems. And Bayesian is also quite exotic. Now lots of people are kind of interested, but not much technical output is there. So I think with Bayesian stuff,
00:41:05
Speaker
I understand this problem more, I understand the area more, maybe it's just my personal preference, and I also see how I can use it for some specific stuff. And I hope, and also the competition is much weaker there. So I hope while all other people are working on deep learning problems and big data,
00:41:29
Speaker
I'm working on Bayesian stuff and small data, so when this kind of application comes on the spotlight, I will have lots of solutions that already work, and I will be able to provide some real value. So that's some kind of general plan for these libraries, Bayadera especially.
00:42:00
Speaker
No, I think we're pretty much done. That's OK, yeah. It's awesome, yeah. It's really good. So of course, we're going to do a full episode with you and then dig deeper into all the libraries that you're building. Probably, probably. There is a lot of interesting stuff going on, obviously.
Dragon's Euroclosure Experience
00:42:18
Speaker
So this is your first time at Euroclosure? Or were you before? I was there. Oh, OK. So how are you enjoying Euroclosure as a final question?
00:42:31
Speaker
Well, this is my first, not only your closure, but it's my first developers conference. Because previously, all the conferences that I went to were like academic conferences, lots of people, everyone was presenting their work, ideas or so.
00:42:57
Speaker
But I actually like this way more because there are not 1,000 talks. There are only selected 10s or 15s. So you can concentrate, like hear what people do. Talk with other people. The atmosphere is much more focused and also relaxed. So you can find people who are already interested in the stuff that you work
00:43:24
Speaker
So you can share the experience. And actually what is interesting for these exotic libraries that I presented, like there were like 15 or even more people who approached me and says, oh, that was interesting. I think I can use it in my startup or whatever work I do.
00:43:47
Speaker
I was trying to do this, but I couldn't succeed before I see how this could benefit me or my friends. And most of those people didn't really know that there are other people in closure conference. For example, there was this guy who has some company
00:44:08
Speaker
in germany and russia or so they they they actually don't do closure or just started doing something but they do gpu with ten video and so in c++ so he was like i was chilling and uh he didn't he didn't look at the the talk list so he he scheduled so he didn't even know that there would be gpu stuff
00:44:28
Speaker
She was like, oh, awesome. This is the stuff that I need to understand or so. So there are lots of people who are interested in this who are not aware that there are lots of other people in Closure Community that actually want to try this stuff. So it's an important point. You're not alone.
00:44:49
Speaker
Right. Well, thank you very much, Dragon. Great answers. And like Vijay said, it would be really good to have an extended conversation over the podcast. So that would be really good to have you back on there. But I think we should stop now. Right.
David Nolan on ClosureScript Development
00:45:08
Speaker
Well, here we are. Deaf and podcast talk show, podcast talk show type thing. David Nolan in the house. Thank you very much, David, for coming along.
00:45:18
Speaker
Awesome, honor to have you here today. You've given your talk at your enclosure now, it's all over so you can relax, have a few beers or whatever. And we just thought we'd have you on here just to talk about a few things to do with growing the community, that kind of thing. So yeah, so open question really about how do you think we can, you made a few comments in your talk about this. What do you think are the big hitting factors that we can
00:45:46
Speaker
help to grow the community from the JavaScript people, that kind of wider community of people that are kind of curious about functional programming but maybe aren't quite there yet.
00:45:59
Speaker
I mean, so that's a great question. I think, you know, when it comes to programming languages, when you're sort of starting out, you have to convince people to use it when you have no users, right? You have no users. And often, programming languages, they cater towards, like, the sort of language geeks. Like, you have a cool feature, immutable data structures, functional programming. You know, all these, that's what gets the people to try it out. You're offering something that other languages don't have.
00:46:26
Speaker
But I think in order to sustain the growth of a programming language, eventually you're going to have to be able to reach out to developers that maybe aren't really interested in the fanciest features. They really want to solve typical problems. They want to know how your technology helps them solve typical problems.
00:46:43
Speaker
And I think there's a huge hurdle, a certain point in the life of a programming language. People want to know, show me something that you've done with this. Show me that you've done something with this. Show me products or applications. Show me that you've been successful with that technology. And now that we're this far along with Closure, you go to these Closure conferences. And it used to be I would talk to somebody like to use Closure.
00:47:07
Speaker
you know, late at night when I'm hacking. And now, you know, people are like, yeah, I use it on my job. I've been using it for two years and we've built a really great project with it and we've got new projects in the pipeline. And at that point in a programming language, this is life, you really want to take a moment and say,
00:47:28
Speaker
OK, we know it's succeeding. We know it's good. How do we grow our user base? How do we reach out to, again, this wider group of developers? Because again, you can't just focus on people that are looking for just going for cutting edge features. That's not enough for, I think, the bigger audience. Yeah. Did you?
00:47:54
Speaker
So one thing I wanted to ask is that, so you're now spending a lot of time on building Closure Script. And so do you also do consulting along with writing Closure Script to different customer projects or you're primarily full-time working on Closure Script?
00:48:10
Speaker
So at Cognitec, Cognitec mostly does consultancy work, but they're also a product company. And the product that they have is Datomic. I'm actually on the Datomic team. So I do occasionally spend some time to help out on the consulting side with consulting projects. But most of my time is spent working with the product team. I do, of course, spend a lot of time continuing to steward
00:48:39
Speaker
and guide the ClosureScript project. It used to be that I would do a lot of work there, not just actually write tons and tons of code. But these days, it's been great. There's more contributions now than there've ever been. And a lot of my time is helping other people succeed at contributing to ClosureScript. And I think that's also
00:49:03
Speaker
A huge factor in its future success that people beyond you know, just me at Cognitec or a couple of the people contact Understand how closure script works fundamentally unable to make significant contributions and push the project forward contributions as well to guys back on track and
00:49:28
Speaker
because it's the kind of thing we should do. We're talking about contributions and we saw this talk from Maria last year when she was talking about the Google Summer of Code where she added the support for modules, I think it was, into the Google's, the CoasterScript compiler.
00:49:44
Speaker
That seemed like a huge move forward in terms of interoperability between ClosureScript and JavaScript. We see that with other languages that that seems to be the big thing which actually is bringing the JavaScript people in to the non, the compiler JavaScript ecosystems. So it seems to be a barrier for Elm for some reason because they want to have ports and stuff like that. Do you think we can go any further?
00:50:12
Speaker
than we have already in the closest, not necessarily you, but other people in the community contributing, what kind of things do you think we could do there to make it a bit easier to consume libraries or to reach out to existing JavaScript libraries?
00:50:25
Speaker
So that's a good question. There's actually a lot that we can do. And I've spoken about this a little bit. But we also have to be very realistic about the limitations of what's possible. I mean, one fundamental limitation is that ClosureScript uses the Google Closure Compiler. And when you sit down and you write a ClosureScript library, it absolutely is ready to be consumed by Google Closure. Google Closure can optimize it. It can dead code eliminate it.
00:50:55
Speaker
the perfect code for it to consume. But people that write typical JavaScript libraries, they're not used to this idea of advanced compilation and getting sort of highly optimized code splitting. They've never had it, and they don't know the value of it, so they don't feel compelled to write code in a way that's easy to statically analyze. So at a certain point, there will be certain libraries that just won't work that great. There's nothing we can do about that.
00:51:25
Speaker
That said, it's surprising how many libraries that people write that actually don't involve a lot of metaprogramming that are written mostly in a static style. It's really surprising. People tend to write
00:51:40
Speaker
pretty boring code, which is actually something that Google Closure can understand. Maria's project is a huge step forward in saying, in the universe of JavaScript libraries that are out there, I bet 30% of them can pass through Google Closure without a problem.
00:51:58
Speaker
make that stuff far, far, far easier to consume. I hope it's clear that we take Interop extremely seriously. And that initiative is basically an alpha status, the JS module processing. And just recently we landed a patch in the current release that updates our support for that. And people have tested it out and it works pretty well. And we want to hear more feedback and we want to make it better. So yeah, we want the JS Interop story
00:52:27
Speaker
to be really great, but people do also understand there is a limitation to what we can do based on the fact that we can't convince the entire JavaScript world to change the way that they write code.
00:52:39
Speaker
One more point, and I guess it's about benefits, isn't it, at the end for that kind of stuff? Because dead code elimination, people I think often think on the server side it doesn't really matter. Whether you ship 400 megabytes or 300 megabytes, it's 100 megabytes, it doesn't really matter. But when you're shipping JavaScript libraries, you can bring it down to 2k or 10k rather than 2meg.
00:53:04
Speaker
That's going to work a huge amount of difference, isn't it? And that's the kind of thing you can really sell if you're giving people that kind of win. It's not necessarily a performance win in the sense of raw numbers on the GPU, but it's a performance win from a usability perspective and a time to function kind of perspective. Do you think that's the kind of thing which could potentially, like the JavaScript guys could leverage a little bit with ClosureScript, that they could bring that story out to bear a little bit?
00:53:33
Speaker
That's another great question. We do have to remember that while I think ClosureScript has somewhat of a different philosophy about what the best way to go about programming for clients is, the truth is that most JavaScript developers are experts with respect to page load time latency.
00:53:55
Speaker
They are concerned about this problem and people have done all these studies. If your webpage takes longer than one second to load, chances are people don't actually wait. They just go off. So performance is critical for engagement.
00:54:12
Speaker
So they are very aware of this. But I think what's happened is as the JavaScript community wants to build more ambitious applications, they want to bring in React, immutable date JS, low dash, moment JS, high charts, and then suddenly their JavaScript application is two megabytes, and even with their webpack and their compression methods are like,
00:54:38
Speaker
Our JavaScript is still a megabyte. It's a gzip. It's a megabyte download. And it's because, again, there isn't this wider discipline in the ecosystem to support things like dead code elimination and code splitting so that I can manage that. And I think that's why we've come out of necessity. It's because if you look at the ClojureScript standard library, it's 10,000 lines of ClojureScript.
00:55:04
Speaker
ClosureScript would have been more or less a non-option. It would have been dead in the water if we did not have an answer to the fact that we have a very large standard library. So out of necessity, we decided to go with Google Closure. And I hope that as the wider JavaScript community understands it, that we could be, that could actually draw people. People who aren't necessarily, their interest is not in JavaScript or ClosureScript. They're like, oh.
00:55:34
Speaker
I really want to solve the UI problem as best as I can. And actually, ClosureScript lets me leverage JavaScript libraries, and then there's a really nice language. And I think there's kind of a large audience thereof, I would say, UI programmers who don't necessarily aren't obsessed with JavaScript or not JavaScript, that just are looking for the best tools to solve the problem. And I think if we focus on that group, I think we can make a lot of inroads.
Challenges in Porting Closure Features
00:56:00
Speaker
So, you're leading the ClosureScript project and Rich Hickey is leading the Closure project essentially. So, is there any situation where when you guys are thinking together that, okay, there's going to be a new feature that we want to add into Closure or wait a minute, we need to think about whether this can be ported into ClosureScript. Are there any hurdles like that or did you ever find yourself in that situation?
00:56:27
Speaker
So that's a good question. I mean, Rich Hickey wrote ClojureScript, so he's pretty familiar with the limitations, although sometimes he doesn't actively work on it, of course. So he'll implement something. But fortunately, I'm there at Cognitech. When Rich Hickey comes up with an idea, I'm there. Other ClojureScript developers are there. And so we'll often chime in and say,
00:56:54
Speaker
Well, last question is, what are the repercussions for something like CloudScript? Yeah, so Coreasync is a great example though. He already knew that he wanted that to work in CloudScript, so he'd already planned that one pretty well.
00:57:19
Speaker
So transducers, again, also very clean, very clean to port. Actually, ClosureSpec is actually a case that was pretty challenging. Because there's a bit of metaprogramming in ClosureSpec. And we can't faithfully port specifically some of the testing functionality. But we can provide something that's good enough. So some things that are functions in Closure, they have to be macros. But of course, these come up in discussion.
00:57:48
Speaker
They may have picked a certain design, and I'll be, well, that's great. Is it OK that ClosureScript is going to be a little bit different from Closure here in that we'll have the same API, but we'll have to have documentation that's clear that these things aren't functions in ClosureScript. They're functions, the macros in ClosureScript.
00:58:05
Speaker
We're having some fun with the screen. Sorry about that, by the way. Well, it's a blue screen, so we can change it to anything. You know, that's much better. Yeah, so I just want to ask you a quick question about, so you're in a band, right? You go around the tour and so...
00:58:25
Speaker
Can you give us some idea about what you play and that kind of stuff about
David Nolan's Musical Journey
00:58:30
Speaker
your band? Yes. So I don't actually have a band currently. So I did have a band like about a year ago that was sort of active and we did some stuff. But in the band, I played guitar and I did a little bit of singing and I've been playing music for a very long time with some very good friends. All right now, I'm in between bands, so to speak, but I do have people that I collaborate with. We work on music together.
00:58:54
Speaker
So that's still happening. It's still active. But there's nothing for me to speak about yet. It's still like in the gestation. That's right. Yeah, that's much it. Well, yeah. That's really good. Thank you very much, David. Yeah, this is our first venture into this stuff. Fantastic to have such a great guest as yourself. Really honored to do it. Thank you very much. And you get a free mug on the back of this. OK, which is really great. Yeah.
00:59:41
Speaker
you