Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#33 - Pierre-Yves Ritschard aka @pyr
47 Plays8 years ago
We talk with Pierre-Yves about how he uses Clojure in the Cloud and why he and his co-founders are betting the business on it.
Vijay also wants to submit to bit-masking. Yeah, so be sure to listen to the end.
More information and links are posted on the website
http://defn.audio
Recommended
Transcript
Introduction of Pierre Yves Richard
00:00:14
Speaker
Okay, welcome to Deaf and episode number 33 and today we have a guest from Switzerland. CTO of Exoscale, Mr... Okay, I'm gonna pronounce your name and then only once in the show. Pierre Yves Richard. Yeah, that's quite close.
00:00:36
Speaker
But it's a bit tough usually for English speakers to say. I think in French they say, Richard. It comes down to two syllables in French almost. So how do you say your name? My last name is Richard.
00:00:58
Speaker
Oh, OK. Anyway, so I think that's the last time I'm going to try this thing. Welcome to the show. So first of all, maybe a quick introduction.
Pierre's Background and Role
00:01:08
Speaker
I think you guys know each other already. Maybe it's good to introduce yourself, Pierre. Yeah. Well, I'm J.F.A. John. And I'm currently CTO and co-founder of a small cloud hosting company in Switzerland. In the past, I've worked with Clojure on
00:01:28
Speaker
in different companies and been mostly doing things related to development and operations of large systems for quite a while. And that's about it.
00:01:43
Speaker
Yeah. And we worked together on my first, uh, like proper closure job. And, uh, you gave me the, you were foolish enough to give me the opportunity to ruin your code base, you know, uh, which is, this is your performance review. Yeah. We, we, we needed.
00:02:03
Speaker
In every company, it's nice to have someone to blame all your problems on. Anything breaks you, you can always say. You can always say, oh, and that's Ray.
00:02:21
Speaker
well that's good that's good so every time you open some code with anybody else you just say oh this is race code this is shitty that guy i know it's got my name on it but he told me to do it yeah
00:02:34
Speaker
Well, he knows enough git to rewrite the history. Yeah, but that was really good. I mean, it was really good to work on with Exoscale. I think it was really good fun.
Business Focus of Exoscale
00:02:48
Speaker
And I think you guys are doing fantastic stuff with Closure, which obviously we're going to talk a lot about during the course of this conversation. But maybe you can tell us a little bit more about what Exoscale is, Pierre-Yves.
00:03:05
Speaker
Exoscale is a company that primarily focuses on infrastructure right now, so not exactly the type of company you'd expect to find a closure at, especially since a lot of hosting companies come much more from the data center business, and we focus quite a bit on software.
00:03:28
Speaker
What we try to do is to build an infrastructure as a service that's really business centric and that gives our customers, especially customers building applications, building software as a service businesses, the kind of tools they need to be efficient. So that's our nation. The hill we're ready to die on is helping
00:03:55
Speaker
people are building software as a service correctly.
Exoscale Services Overview
00:04:00
Speaker
Right, excellent, okay. But to be a bit more concrete, I suppose that's a bit vague to be more concrete, but what we offer people are virtual machines, object storage, which is especially compatible, and network, and on top of that, additional services that make the lives of people building SaaS easier.
00:04:24
Speaker
What kind of things are there? I mean, are you talking about things like billing and stuff like that? Or what other things or security? What kind of things do the SAS people really want? Yeah. What we're currently rolling out is a number of things with regard to billing, with regard to handling transactional emails, watching over a sales funnel,
00:04:51
Speaker
things that you're going to need to build as a company that steps into the software as a service spectrum. I think in terms of, I mean, for most infrastructure as a service providers, things with regards to security are pretty much a given.
00:05:12
Speaker
That's the ball that you have to pass to be in the game and then how you specialize is what's left.
Technical Stack and Interfaces
00:05:24
Speaker
So where is the where is closure coming into this picture? Because usually, you know, system level stuff or what I mean, what kind of stack do you build? Which part is is built by exoscale? There are things from, you know, procuring the hardware all the way up to providing the front end everything. So all these parts, basically, we we build hardware.
00:05:46
Speaker
We have custom design chassis that are built for us and that we deploy to our data centers. We have people that operate our BTPS to ensure that our IP space is correctly announced.
00:06:06
Speaker
And we go all the way down to building the JavaScript and soon ClosureScript interface that people interact with from the web. So it's a bit wide ranging. But the one thing we don't do is build actual data centers, score meters, power, and cooling.
00:06:37
Speaker
So why Closure then? I mean, how did you come to Closure? This seems like an odd choice to do, you know, because especially systems level stuff and everything is done by Go or some other fancy
Why Use Clojure at Exoscale?
00:06:48
Speaker
things. Well, we started in 2012 building all this, so back then Go wasn't
00:06:54
Speaker
a huge option in that space. But what I mean, this isn't the first company I use Clojure. So definitely I might have had something to do with it. But I did use Clojure for similar things in past companies. And the way I got to it was
00:07:19
Speaker
realizing somewhere in 2006, 2007, that there was not going to be a way around the JVM ecosystem. Back then, especially in the ops world, the
00:07:38
Speaker
The space was mostly Java resistance, and it started shifting. For me, the first times where I saw compelling JVM projects was around the time Elasticsearch really became a thing. I guess 2008, something like this.
00:07:59
Speaker
And at this point, we needed to interact with the JVM in the company I was at, and we tried out Scala, we tried out...
00:08:10
Speaker
Erlang, which also was a bit up and coming, especially coming from teams who mostly dealt with Ruby and Python back at the time. And we tried out Clojure. And so Erlang didn't work out because of the bad compatibility with the JVM ecosystem. Scalab ended up quite frightening for the team, especially in terms of outputs and in terms of
00:08:37
Speaker
would I say inconsistency of code between two people. And at that time, it was a mostly Ruby centric team, I was running and it really clicked with plenty of people and
00:08:53
Speaker
ended up being quite a natural transition. And it sort of stuck. Then I used it quite a bit in the next company. And so it was a natural fit for some parts of Exoscale. We don't build all of Exoscale in Closure. But it does fit in its niche. And it ends up being the most widely used language. But we don't use it at every level of the stack.
00:09:25
Speaker
Do you do? Do you do? Is it just closure or is it also closure scripts as well now? Yeah. So right now we use closure script mostly for internal tuning.
00:09:34
Speaker
like our internal interfaces, not the single page application that we present to users, but we're switching to that, you know, but at our own slow pace, because it's a big project that was built before Clouchescript was even an option, really.
00:09:57
Speaker
But the ecosystem has matured quite a bit. And to manage the complexity we now have in that application, we are starting that project. What is it currently written in?
00:10:12
Speaker
JavaScript, it was initially Angular, then some React snuck in, and now it's a huge Frankenstein of an application that's... Maybe we don't talk about it too much. It's hard to move around.
Market Positioning and Growth
00:10:30
Speaker
I mean, the IDs that were used in the current app were
00:10:37
Speaker
inspired what we saw in the ecological world, but just a full-on rewrite was impossible at that moment. Sure, sure. Well, you have to justify it as well. What is the, because I know you said you're kind of S3 compliant, I know that from
00:10:58
Speaker
from our history, but what is the story in terms of cloud hosting in general? Are you able to, how do you compare yourself with Amazon or are you comparing yourself with other ISPs? Where's the kind of market niche for Exoscale? Well, definitely in terms of
00:11:23
Speaker
I think the easiest comparison that can be made as far as Wilkinson is digital ocean. We go a few steps further in some areas, but that would be the most obvious comparison because digital ocean focuses a lot on the
00:11:45
Speaker
web developers, people want to publish web pages. And they recognize that web hosting today is a bit more complicated than it used to be. You can't just set up some FTP and for people to use a mutualized PHP MySQL server, like the one and once and the OVH of this world would.
00:12:12
Speaker
But they really still focus on that, you know, most, most of the time single developer crowd, you know, even though, of course, eventually, they'll, they'll try to cater to a wider crowd. Yeah.
00:12:30
Speaker
I mean, ExoScale came out of my history and Antoine, my co-founder's history. I was stepping out of three companies where we always had to make a choice between, on one hand, AWS, which was super expensive and provided very, very slow IO at the time.
00:12:51
Speaker
And on the other side of the spectrum, dedicated server hosting, which sure enough were much less expensive, but also brought a lot of problems with them. On this really cheap dedicated host, you have to set up a lot of things by hand. Your neighbor is most likely someone using a botnet.
00:13:19
Speaker
And the quality of service is what it is. And most people building software as a service also don't need as many services as AWS offers. You're fine with having local storage because your application is itself distributed and most likely horizontally scalable. It's probably...
00:13:49
Speaker
fault tolerant at an application level that allows the provider to make a few compromises to make the whole thing more efficient.
00:13:59
Speaker
You know, it also probably stems from the fact that I'm very modest and I thought, you know, this is going to be an easy thing. And here we are five years into it. But you've got some pretty good names on your customer list now, haven't you? Yeah. I mean, a lot of whom we can't talk about because
00:14:26
Speaker
OK, I'll be quiet. But CERN is definitely one of the ones we're most proud about, especially when you consider the size of the infrastructure, the fact that it would reach out to us was a nice moment. Yeah, very good. I hope we put more of those names on the list in the near future.
00:14:53
Speaker
But what is the size of your technical team? We're 15 people today. We're trying to get to 25 as soon as possible, which includes hiring a lot of closure people.
00:15:14
Speaker
We have quite a few spots in that domain as well. But yes, 15 people mostly focused on the end. I would say almost a half an half split between people who are focusing a little bit more on operational maintenance and people who focus more on pushing the product forward. That's about it.
00:15:39
Speaker
Yeah. The other thing I was just going to say a bit more light hearted really was when I visited you guys in Switzerland, I noticed that half of the company seemed to skateboard to work because the station is on a hill and then you're further down. So you could just you could all kind of skate down. Yeah. And that seems like, you know, I think it's super cool for sure. I think on a normal day,
00:16:07
Speaker
I don't think on a normal day anyone writes a car to work. When the tough winter kicks in, sometimes we diverge a little bit from our good habits. Doesn't someone have one of these skateboards which can go up the hill as well?
00:16:31
Speaker
The Antoine has an electric scooter, which can ride up your... Because there are steep hills in Switzerland. Especially in Los Angeles, it's a coastal city, in a way. It just drops down into the lake.
00:16:54
Speaker
We never hired for this, but there's a good mix of people who like having something to do on the weekend that's not necessarily computer related on the team. That's great.
00:17:13
Speaker
That's quite nice. To be able to do something that we like, but also not leave the Silicon Valley dream of doing nothing but work. So you're not typing Emacs while skateboarding then? No, that's that. OK. Because we're coming on to Vijay's perfect question soon, I guess. Of course. So Emacs or something else?
00:17:43
Speaker
I never really considered there was anything but one editor. Of course. I think this seems like the best company to work for. That's pretty much everything. A short period of time.
00:18:06
Speaker
All of our network equipment config management was mostly done for me, Max. It's been used in the... Yes. And actually the first iteration of the company's internal documentation was all org mode with a clever Jenkins script that would... Anyway, it was a bit too Max-centric at the beginning. All right.
00:18:36
Speaker
OK, this sounds like a perfect company to work for. In Switzerland, everybody goes by skateboards, and everything is in Emacs, so org mode or Emacs. This is the best. Wow. It's much more diverse now in terms of editors.
Serverless and Platform Services
00:18:51
Speaker
We have some Emacsers, a few people in VI, and I think still someone who's joining the IntelliJ resistance.
00:19:12
Speaker
This is what you're talking about, about the weekend thing, right? I mean, the people who don't use Emacs in the weekend, you train them to use Emacs and then from Monday, they're back on Emacs again.
00:19:24
Speaker
that there must be. There is no other editor. So I was wondering, if you see all the cloud stuff these days, people are going, the serverless is the latest buzzword.
00:19:41
Speaker
So what are your thoughts on this one? Because more and more, you know, application development is becoming like making smaller services and Amazon, like people providing all the smaller components, the infrastructure, everything is just provided by them. So what is your opinion on some of that stuff? I view serverless as another iteration of platform as a service. We went down that route with Lexus Gear, we integrated a platform as a service that was
00:20:08
Speaker
Heroku compliance back in 2014, and 2014 through 2016. And we, I mean, at that time, we knew people at .cloud, and by proxy, some people at Heroku, which we talked with as well.
00:20:32
Speaker
And which is why we decided to partner to bring pass on Exoscale, because they were saying that something which turned out to be true for us as well, that it was becoming commoditized, that there was something that would be a point where something that would be a bit like pass would
00:20:55
Speaker
be available for any platform, which turned out to be true. But also that most people who use these types of projects tend to either use them to play around, like for, you know, really, really as a test lead. And when they, when they become big enough, they transfer back to infrastructure.
00:21:19
Speaker
I see serverless a little bit in the same spectrum. I think there are niches where that won't be true for serverless. I think typically in the IoT sensor handling stuff, this sort of co-procedure that run on inbound events make a lot of sense. There has to be an answer in that space.
00:21:43
Speaker
And that won't necessarily be commoditized. The only thing I'm worried of, I mean, as an application developer and not as a provider, when looking at developing with serverless, is that I remember the days of people being a little bit too enthusiastic about MySQL stored procedures, for instance. And you run that risk of completely being
00:22:14
Speaker
I mean, you lose a lot of visibility when running code in that fashion. And the amount of tooling that you need to get back into the same amount of visibility that you have with standard code is substantial. So I don't see it as a fit for every purpose right now. And I also think that some stacks, especially on top of Kubernetes, will end up being somewhat commoditized as well.
00:22:42
Speaker
available for many providers. I agree. That's what I'm seeing in AWS as well. There are these Lambda functions that they advertise as serverless for connecting smaller pieces or ETL if you don't need a lot of processing power or if you don't need long-running things, then you can connect them with step function, these kind of things that they project for.
00:23:11
Speaker
but they're also coming up with the EKS like the elastic Kubernetes service and running the Docker stuff. I think for some of the use cases in my mind, it makes sense to use the step functions when there is this very low need for high memory usage or something.
00:23:30
Speaker
But as you said, maybe traditional applications, probably they don't need this kind of stuff. What are your plans for the cloud? And I think that as soon as you start pushing too much, the amount of looking that you end up in is quite substantial. But definitely, as you say,
00:23:51
Speaker
reacting on events that could happen on a bucket, for instance, like on an object storage bucket or when receiving an IoT sensor, having a way to provision small workloads would definitely be valid. In the infrastructure space, every company were also moving towards having
00:24:15
Speaker
as soon as possible, a hosted Kubernetes option. That's just something that's in the works for us and we'll see the light of day for the first part of this year. But that's clearly something that I think will help relieve this fear of lock-in for many customers. We see a lot of customers wanting to go towards
00:24:43
Speaker
higher order abstractions for infrastructure, but be very afraid of the potential locking that they would find themselves into.
00:24:54
Speaker
Because AWS and Azure and Google tend to throw 100K at you when you start the company, but when the free credit runs out and you bet everything on them that then things get a bit more complicated. Yeah, and there's a lot of nonsense talking about moving from Google to Azure and from Azure to AWS and from AWS to other places. And people are a bit like, I think people are a bit like,
00:25:20
Speaker
When they choose a bank or a database, they don't move. It's like you're in there for life. It's very rare that people migrate from one of these things to another thing. Yes. They know that. They know that they can give away 10K, 100K, and the lifetime of that company is they're going to get it back in spades. Okay, can we move on to the closure now though?
00:25:49
Speaker
I don't know how far we're into this, you know, probably like half an hour in.
Clojure's Role in Operations
00:25:53
Speaker
This is great stuff, but I think, yeah, we're half an hour in, so just about time to, we'll cut all this from bit out. We'll just cut all the first bit out, and we'll just talk about closing it.
00:26:04
Speaker
I know that you're using closure for some of the API driven stuff that you're doing, but can you give us a bigger picture of where you're using closure at Exoscale and why you chose closure for those specific pieces?
00:26:24
Speaker
Well, Alex Oskar we started out not, I mean, I stepped out of a company where I used closure for everything. I previously worked for paper and add paper yellow, he was written in closure, basically.
00:26:45
Speaker
Exoscale is a bit different in the sense that, you know, at the beginning, we didn't use Clojure that much, because it didn't make sense, really. And the first thing that made sense was the billing. And billing initially at Exoscale was quite simple. It was a simple batch process that would put in usage records and it was still, you know, this really
00:27:11
Speaker
canonical data manipulation thing where you had this first space where you would pull, then you had nicely formed map data, and you had to transform it to produce an output. And it was also the first thing, canonical use case, but also the no serious enough that we wanted that thing
00:27:36
Speaker
produce bad results since that's what paid our salaries. So we had to make sure it was root code.
00:27:46
Speaker
And then from then on, I mean, fortunately, early enough, we ended up having an issue with it being a batch process, and it needed to move to a more of a stream processing approach. And so a collection of little stream processors. Yeah, we started a collection of small stream processors with the Kafka that still work closer
00:28:15
Speaker
Here, especially back at the time, Kafka being very tightly bound to the JVM closure was the obvious choice, at least for us. And then it moved on. 2014, I think we started working on Pithos, our first implementation of an S3-compatible object store that we built internally.
00:28:42
Speaker
and again was built completely in Clojure and a lot of internal tooling. Basically now our approach is to say that anything that happens at the orchestration level, so anything if you look at our standard workload is having an orchestration piece,
00:29:05
Speaker
that talks to agents that maybe run on hypervisors, that maybe run on storage nodes or whatever, and to have the orchestration they are built in Clojure, and to try and factorize as much as we can from project to project. So that's the niche. So we end up building quite a few demos. I mean, the thing we do most often in Clojure is build demos.
00:29:35
Speaker
Looking at the labor ecosystem out there, I think our key difference is that we deal with
00:29:44
Speaker
non-json data quite a bit. We write RPC clients for protocols that need to have streaming data packed in binary. And so we had to integrate with Netty quite a lot and to work with bind buffers quite a lot, which isn't something that you find a lot of support for in the culture ecosystem.
00:30:14
Speaker
No, no. It's a total pleasure to work with those bite muffins, yeah.
00:30:25
Speaker
But from the look of the problem, isn't Erlang more suited for this kind of stuff? Or what kind of challenges did you face in using Closure compared to language like Erlang? Because they have a really nice process-to-process stuff and binary support in Erlang, as far as I know on OT. Yeah, I think that it would have been a great fit. I mean, that part could have been a great fit.
00:30:51
Speaker
But we also tied with a lot of very good libraries that run on the gdm and that's it. You know, if you look, we use Cassandra exclusively. If you're going to target Cassandra, it's hard to interact with it. I mean, it's much easier now, but when we started there was basically no choice.
00:31:10
Speaker
Kafka was the same. And these things, we would have had much more trouble sourcing in the early world. So yes, it was a give and take situation. I'm also the whole agent thing. I'm not a huge fan of.
00:31:31
Speaker
the unbounded queues on mailboxes can produce situations which are a bit hard to cope with in the early world. And I talked about a situation that's way back in 2007 or something. RabbitMQ was a bit hard to
00:31:58
Speaker
to keep in check in production for this specific reason back a while ago. So there's a whole gymnastics that you have to go through when programming in Ireland that we didn't have the skill for.
00:32:17
Speaker
We assembled around Clodru with its advantages, but its drawbacks as well, especially for the process-to-process stuff. For sure, life's a bit more difficult when you want to implement, I would say, non-blocking code. It's a bit tougher than it would be in the end of the world, for sure.
00:32:39
Speaker
And how does the, because one of the things that people complain about closure is the performance. So did you find any roadblocks in terms of performance of closure? What kind of techniques did you use?
Performance Considerations in Clojure
00:32:53
Speaker
No, not at all. Anything on that one? That being said, we don't do a lot of huge CPU bounding in closure. As I was saying, there's a lot of shuffling data around.
00:33:07
Speaker
The storage components we use are mostly, yes, mostly not written in Clojure. So it's not something we run into that much. I think we have this attitude of relying on as few libraries as possible and avoiding them, except for core, I think.
00:33:31
Speaker
avoiding libraries that do code traversal and rewrite what you do. So no, we're really definitely nothing. Performance is not where we're taking a hit when we're taking a hit with Clojure.
00:33:52
Speaker
I'm guessing what you're saying there, Pierre, as well, is that according to you with the asynchronous workloads and the process stuff, the core async is at the core of what you're doing. Is that true? Are you relying very heavily on the core async abstractions? Yes. It's still a constant worry of mine if we took the
00:34:17
Speaker
the right road, but we sort of bet the farm on Corrising in a few places indeed. And right now, it's quite evident that Corrising has much more following in the Clojure script world than in the Clojure world. And we ran into quite a few oddities in the internals of Corrising.
00:34:47
Speaker
You know, the work has now paid off. We're now confident about being able to twist, or I think, in doing what we wanted to do. But it's been a long road, I'd say.
00:35:05
Speaker
So obviously having that library has been good, but what specifically have been some of the negatives or some of the things that you've bumped into that have been a bit rough edges in general?
00:35:18
Speaker
Well, there are a few things about the semantics of the API that aren't super well articulated in documentation, and that you're likely to, especially with regards to the behavior of John. When you've only got five lines per method, you can't really articulate everything, Pierre, it's going to be fair. Sorry, that's a bitch.
00:35:47
Speaker
So yeah, there are things we found out the hard way about the actual semantics of the API. And when you want either to change the semantics or to figure out what's going on, it takes a while to dive into Core Async's code and to understand what exactly is going on.
00:36:11
Speaker
It's not the project that gets the most attention from its maintainer. And so sometimes, you sort of feel like that person with the
00:36:30
Speaker
The person is psychoverflow that where there's just a question and no answer. Want somebody help me out here? And that's sort of what the closures Jaira feels to me. Yeah, we'll get there.
00:36:51
Speaker
I think that's a nice segue into that one now. Because I think before the show, you were mentioning that you had a pull request open or a patch open on, was it the reducers transducers thing? This was open somewhere in, I don't know, three, four years ago. I actually had a ticket on core.cache that was closed, I think last week.
00:37:14
Speaker
that I opened back in 2013. Oh, no, no. In 2011, I mean, something completely... Yeah. The cash was finally invalidated. At that point, I wasn't, you know... That's why I made a fix there. It was literally least recently used, yeah.
00:37:44
Speaker
Yeah, but I think what's interesting about you is you've tried to make contributions to Closure, which is really impressive, I think. How have you found that process? I mean, here it was quite simple. I mean, to be fair, especially with Core, the Core of Closure, for me, works out fine.
00:38:06
Speaker
I would be wary of starting substantial work and submitting it because having seen the exercises in the past from other contributors,
00:38:21
Speaker
The chances of your work being dismissed and reappearing a bit later in a different form is a bit too high, in my opinion. I mean, that being said, the patch I proposed, the path to it was we're using transducers quite a bit, which for us are a very key component to allow us to do
00:38:51
Speaker
quite a few elegant things in terms of testing. And one of the functions in Core didn't have an RIT that allowed you to create a transducer. And the patch I submitted was to make a transducer out of the lower RIT. It went back and forth a bit. I think it's now in the
00:39:12
Speaker
in shape to be pulled in, but it hasn't for 1.9 because it wasn't on the priority list, which is something that's, I mean, I completely understand. For now it's in our code and it's actually in one of them, but whenever they put it in, they put it in, they put it in.
00:39:36
Speaker
But you will have a different perspective towards Closure, right? Because especially you're betting a lot on Closure, running your entire infrastructure, your company. It's much more because people say, hey, you know, Closure is a beautiful language. All the developers want to use it. So that's a different kind of complaint. As a developer, I can complain, hey, you know, the process is different or the documentation is lacking.
00:39:59
Speaker
But as a company, the risk is higher if you see any of these problems with closure. So what do you think about that one?
Future of Clojure
00:40:09
Speaker
Yes, I mean that's a topic I think though that I don't think closure itself is at risk. Let's say that
00:40:21
Speaker
the, the language, I mean, let's say that cognitive, cognitive goes bust, or, you know, he finds a new hobby. Because if that if that happens, the closure led by cognitive will die. Because there doesn't seem to be strong enough characters next to hitch to, you know, pick up the work
00:40:48
Speaker
to carry the plate afterwards. So I think what would happen if that were the case, I think a few companies would step up to the plate. We'd certainly try to be a part of the effort. I'm sure Jax would. Because we were seeing much more adoption in Europe, I mean, from where I'm looking at anyway. So I'm sure people would step up to the work and try to
00:41:17
Speaker
So surely, if that were to happen, we'd probably find ourselves in a, you know,
00:41:26
Speaker
risk averse, but it's... Medging the patch. A world host cooperative. Yeah, it would probably be something like Debian where everything gets argued to death and nothing ever moves, but at least the language would be maintained. And for sure, we wouldn't get a new spec or a new transducers every year like we do today. So I hope it doesn't happen.
00:41:53
Speaker
We'll keep him going, yeah. Yes. I hope it doesn't happen, but I think there would be a contingency plan. Yeah. Yeah. I think there was recently one thread on Reddit closure or subreddit. How old is Rich Hickey? There was discussion around it. Everybody was like, oh, he must be around 4,000 years old with his wisdom. He's a time lord, yeah. That's right.
00:42:22
Speaker
But this is an interesting discussion because if you think about it, there are movements like Closure is together, trying to support different libraries and those things.
00:42:36
Speaker
I think it would be nice more and more members contribute to that one. Because that's all said and done. If one person is leading the effort, there should be some touch-bearer, so to speak. At some point, he wants to retire, I would assume. He wants to stop. Yeah, or he gets bored because sometimes you...
00:43:01
Speaker
Yeah, fair enough. He gets bored and starts creating a database. Yeah, exactly. Don't you think that like, I mean, if you remember, like Node.js, you know, the original author of Node.js burnt out and then handed it over to some other guy. Now there's been like, you know, three or four leaders of that project.
00:43:24
Speaker
So, you know, in theory, there could be, you know, someone from the current cognitive tech crew or someone from the community could step forward and, you know, pick up the gauntlet, but it would be, it would be definitely, I mean, obviously these are big shoes to fill, you know.
00:43:40
Speaker
So yeah, I think it's possible. But I think there's enough adoption in the outside world now to make that a reality. Exactly. No one's going to just think, oh yeah, fine, forget it. There's enough people out in the community who know the light. And it's open source. So like you say, it'll probably be maintained if nothing else. Yeah. So I mean, one of the things that's
00:44:04
Speaker
one of the things that irritates every everyone that uses closure at one point, but is actually an opportunity is the lack of blessing from, you know, an extended standard library. And I think I mean, I just stumbled on to closure is together. And I think such efforts are, yeah,
00:44:28
Speaker
we really need to be made and more likely to actually support Kujos together quite soon. Because sure enough, Cognitec doesn't want to own that space. Cognitec doesn't want to tell you. If you want to do an HTTP request, that's what you should use. But there should still be an answer to what that process should look like. Because otherwise, and bringing in
00:44:55
Speaker
because since we're hiring a lot of torture people, we mostly bring people who haven't used torture before. And it's tough for them to just step into a language where, I mean, if you compare that to Python or even Golang where there's a single point of documentation where you're gonna figure out how to do basic tasks.
00:45:23
Speaker
And, and that's how did closure, because you're presented with a variety of, you know, sometimes, yeah, efforts that that weren't that didn't completely go through. Some of them I'm responsible for. So I'm taking my part of the blame here. So yeah,
00:45:53
Speaker
I do agree that we have passed the critical mass already. Yes.
00:45:59
Speaker
What closures together are doing is really amazing. Anyway, so of course, it's not all doom and gloom.
Hiring and Development Roles at Exoscale
00:46:08
Speaker
So you're planning to hire a lot of closure guys, or at least trying to think of training closure people. So how do you see the market like? And then what do you think of what kind of skills or how much time people will take to acquire closure skills?
00:46:29
Speaker
Well, first of all, in terms of the market, I see a lot more of a reaction from the cloud transcript community, which surprised me. That wasn't my... But clearly, there's a lot of steam in that area, and we're going to have less trouble hiring people.
00:46:51
Speaker
I think the fact that we write demands that sometimes deal with system level stuff makes our hiring journey a little bit harder. We have these small exercises that we send out to applicants, one of which is a bit of a... Yes.
00:47:17
Speaker
It's free. I would love to do that. Yes. One of which entails writing a parser for an obscure binary-level protocol in Closure. Oh, cool. Totally cool. I think Ray enjoyed it. It's quite a bit...
00:47:46
Speaker
Actually, I mean, it's proved to be quite useful. It's proved to be quite useful in the end. Yeah, because because we deal a lot with like mixing these the the usable first approach of Closure with the completely new to my first approach of Java quite a bit and sometimes figuring out how to efficiently write algorithms which switch from
00:48:11
Speaker
when we'll to the next isn't that obvious. That's definitely something that people end up doing, Alex, in some areas of the stack. So yeah, so that's this. Cool.
00:48:25
Speaker
So it's much more closer to the lower level programming. For some of the projects, yes. Especially everything that deals with the communications channels, building the non-blocking network servers, that's definitely something that we work a lot with.
00:48:44
Speaker
And in terms of training, what we try to do is have for new newcomers have a project ready, like something that's gonna get them in a tunnel where they can incrementally learn because we tend to have a wide ranging area of technology. So it also, you know,
00:49:07
Speaker
gives a bit of comfort to newcomers without trying to drown them into too much information at the beginning. But we also have now figured out a little bit what our applications look like, how we structure our applications and demands. And so there's a whole training period around that as well, ensuring that
00:49:38
Speaker
How do we build Tenerines basically at Exoscale as part of that training? I'm pretty sure we have a significant amount of closure code now and one of the complaints or one of the things that people talk about the dynamic languages is the maintainability and then
00:49:54
Speaker
I remember when Zach was here, Zach Oakes was on the podcast and then he said, you know, the hell is other people's code. So it's very, as long as you don't need to read other people's code, it's really nice. So how do you, how do you tackle this one? Because with, with any, any tools that you use. I don't tend to read that when, when we did that exercise way back, like 10 years ago, between Scala and Clojure and Ireland, that's really what, I mean,
00:50:23
Speaker
We read the output of four people's Scala code and we were like, I mean, you know, this is Pearl and C++ all over again. No one's going to be able to read any.
00:50:34
Speaker
Yeah, and it could produce that by them. And foreclosure, it was pretty much the same. That's still true today. That being said, spec has been a game changer for us. The first iteration of Arm object store is open source. It's called Pithos. And the first iteration was pre-spec and pre-component.
00:51:00
Speaker
and it's a mess. We learn a lot in terms of large scale production of project while building it. I'm a bit ashamed that it's open source because it's mostly me.
00:51:20
Speaker
But it helped understand what we needed that we weren't getting from Plodger. And that specification at the border of your functional boundaries is something that we now rely extensively on. And that gets us much more comfort in what we're building. And in terms of long-term maintainability,
00:51:51
Speaker
The other thing I like about Closure, by the way, is that Spec has been in Alpha for about a year and you've been using it for more than that. And it's still in Alpha, and yet it's kind of like everyone's relying on it in production systems. I think this is a really strange thing for many languages, I think.
00:52:09
Speaker
Yeah. This aspect of the ability to trust the stuff that's in the core library far more than you would trust something from other languages, I think, especially JavaScript. But that's an easy shot, I think. But even Java, I think some of the Java stuff at the beginning of some new things with generics and streams
00:52:31
Speaker
They just seem a little bit janky sometimes, especially generics. But with Clojure, it always seems like it can be depended on pretty quickly. That's true. That's one thing we like about the language is that you don't need to think too much about trusting what comes from the team. And Speck is one of these occurrences of when I saw the announcement for Speck, I was like,
00:53:01
Speaker
Why would they write that this prismatic schema already? Why are you wasting your time on this? Six months into it, we're like, yeah. It does bring something to the table that we needed. OK, I'll show that next time. So yeah.
00:53:26
Speaker
Are there any libraries that you want to talk about that you built? I know you have a async Kafka library that you released, and there are a couple of other libraries that you released as part of your company. I think there's one library that people should use more, which is Uniload.
00:53:44
Speaker
It's like the simplest one I've written and coincidentally it's one of the most documented. But basically it wraps a lockback and gives you tools to configure a lockback from Closure.
00:54:06
Speaker
And when I started with Clojure, the hardest thing for me was figuring out how to log something.
00:54:17
Speaker
not coming from a Java background. I mean, there was this whole thing about the lock4j properties and I had no clue what went in there. And any other language that you use lets you set up logging simply with a few bits and pieces. And Clojure is really suited for this because you expect to give a map of configuration and then
00:54:40
Speaker
things work well. That's what Unilock gives you without breaking, you know, without isolating itself in the closure world, but interacting, you know, if you do have log4j, if you do have logback, or if there are any other parts of the libraries that you use that rely on.
00:54:56
Speaker
They are going to fall into the same thing, so I think Unilog is quite nice. Kinski is a Kafka library, which I think now is one of the last libraries to support the more recent Kafka analysis.
00:55:17
Speaker
We're slowly working towards Kafka Stream support for people interacting with the Kafka Streams ecosystem. We tend to be bad actors of the library ecosystem because we work on these, I've raised a lot internally and at some point we don't code, but that's the whole, I mean, that's the whole, you know,
00:55:45
Speaker
There's a whole lot of chaos underneath where we're like, we have to get this thing out to the market. And so we write a few libraries, a few bits and pieces. And then when things settle down, then we pretty things up. One thing that's moved out of this is that our smaller
00:56:07
Speaker
shim interface to Netty, which is now mostly published in the open, even as we go. Right now, what it gives you is really a good toolbox to work with asynchronous networking, a bit lower than Aleph would
00:56:32
Speaker
And it gives you no choice in terms of what the semantics for working with Async are going to be. It's called Async. And that's it. Whereas, Zaktam and Biot's manifold, which is
00:56:53
Speaker
like this awesome abstraction of what you want to work with in the async world. It gives you this option. And we wanted something that allowed us to be maybe a bit closer to Netty, to interact a little bit more with Netty, and to better on co-icing. What we gain out of it is a bit of simplicity, but it was helped right now.
00:57:21
Speaker
Looking back, I don't know if you would make that same choice, but I mean, it must be pretty stupid on the outside what we did gain from from the exercises, but now understand that he quite a bit. Yes, which we didn't necessarily at the beginning, you know, because you you might go into it thinking, you know, it's just async networking, it's basically a
00:57:49
Speaker
a small wrapper around the OS is select. It turns out it's a bit more complex. It's a hell of a project, isn't it? It really is, yeah. Well, it's not that much code, which is quite frustrating. It's always the same with the Closure project. You're like, I wrote this awesome thing. And it's been like one year and a half. I'm so proud. And it's like 2K lines of code.
00:58:18
Speaker
Yeah. But there is always a fine line, right? Because I was also dealing with some Google AdWords API these days. That's like a SOAP client generated by Google access stuff, which is very interesting. I mean, Google provides SOAP API. That's weird.
00:58:34
Speaker
And then I was interacting with it. And then I thought, oh, you know what? I'm going to make an amazing Closure wrapper around it. And then I started with like, you know what? Fuck it. I'm just going to use the Java Interop and then get the thing done. And if you see the amount of thinking I need to do to make it Closure-like versus just using the Java Interop, there is a significant gap. So I'm like, I'll just use the Interop, which is quicker and clearer. And I don't need to worry about one more layer of Closure on top of Java.
00:59:02
Speaker
So this is like a dilemma sort of situation. I mean, for a team, the trade-off is a little bit different. Because for a team, when you write that, how would I say that, small wrapper around the job, I mean, we have an internal thing that wraps the Apache Curator Zookeeper library, for instance.
00:59:28
Speaker
And sure enough, it's a small wrapper. But having done that, you've also shown to your colleagues, this is the things that we know how to do with the keeper. And we give you a closure way of dealing with it. You don't have to deal with all the, how would I say, the reflection warnings and everything.
00:59:56
Speaker
It's contained in there. Yeah. Well, I mean, the API itself is not a problem, but the data structures that you need to communicate to the closure side, that's where all the weird shit happens because you get this, if you're interacting with SOAP API, you're getting all these Java objects from it.
01:00:20
Speaker
And then now suddenly you need to convert everything into a map or in a dev type, def record, and then give that kind of shit back. That makes it like super, you know, icky for me. I'm not sure, you know, I haven't written like thousands of lines of code yet, but for me, that is where the, not the confusion, but the design dilemma comes from. Because Closure, you're dealing with maps, you're dealing with these kind of things. And true enough, the default approach of the
01:00:50
Speaker
the initial default approach of the closure community was, you know, well, I mean, just use interrupt. But yeah, but again, I think for teams going through the effort of providing something that gives you a sense of that's that looks like closure is is always is already a good plus. Yeah.
01:01:14
Speaker
So I think we talked about lots of things. By the way, I would like to thank you for supporting Dutch Closure Day and Second Time. Thanks a lot for your support. As you know, it's a free event and we are really dependent on the sponsors.
01:01:33
Speaker
It's been growing and this time we have a lot of sponsors and that gives an indication that Closure is picking up in Europe a lot, which is a really nice thing actually. And quite comforting. Yeah, exactly. Because we needed a life for a little bit. And you are hiring. So are you going to be there as well? Yes, I intend to come. All right, okay. So I'll see you then because I will be out there also. Perfect. Excellent.
01:02:03
Speaker
It'll be amazing. Of course, I know you submitted a talk as well. We are now sending all the talks to the selection committee and then hopefully you'll be able to speak about your exascale experience there as well. And I know you're hiring, so are these positions in Luzon or are they remote? And I don't know, people who are listening to the podcast might be interested in applying for the jobs. We're really thinking about
01:02:31
Speaker
restarting the remote hiring efforts right now. So in terms of closure, we're hiring for both positions that focus more on the orchestration layer, so mostly in
01:02:46
Speaker
mostly behind the scenes, distributed systems work, I would say. There's a lot of work in the storage subsystem, but there was also a lot of work on the
01:03:02
Speaker
cloud controlling, so basically interacting with a type of virtual machine hypervisor agents. There are a lot of things we want to do there and we now built a nice data layer and API gateway layer also, which give us a lot more flexibility, but it's also more work.
01:03:28
Speaker
So we need help here, and I think I'm hiring as many as four people for these positions. But we're also trying to, as I was saying, we use ClojureScript for quite a few internal projects.
01:03:45
Speaker
and to support the effort that I was describing to help SaaS businesses get to business faster. Our frontend is going to have to evolve and it's now the good time to invest in the console. So here there's a really nice
01:04:06
Speaker
gain field project to tackle. And I'd like to start with two additional people specifically working in Closure Script to build that effort. But we obviously already have front-end developers in-house.
01:04:27
Speaker
Do you have like a preferred stack for the front end these days, Periv, or are you still kind of working it out?
Frontend Development with Reframe
01:04:35
Speaker
No, we did a few experiments and Reframe came out on top, mostly because I
01:04:50
Speaker
This whole reduction of the reframes, it seems like they took all the wisdom from home and framed it in something that's where there's a little bit more attention to detail. There's more documentation, there's more support from the community. So yeah, seeing
01:05:14
Speaker
It seems like a good bet. It was rather easy to build mock projects with it. And with the agent, I love the DevGuard approach, which we need to, because we have this nascent library of components that encode how we want the UX to be to ensure it's consistent across
01:05:43
Speaker
this rather large and complex application. So yes, it's an ecosystem that gives me, you know, comfort and really seems like it can support a long time effort. So I think we'll make that better. Okay, that's a good idea.
01:06:03
Speaker
So for the people who are listening, it's about Closure Script Reframe and a lot of orchestration, Closure work. And you're hiring almost five? Yes, I think all together. You should count front-end and back-end. That comes around to seven.
01:06:25
Speaker
Very nice. So I think, you know, people who are listening, I think they should just go and check out Exoscale jobs page. And then you guys should apply because, you know, Pierre seems to be a very nice guy and Ray worked with him. And so that means that doesn't mean anything. We're still talking.
01:06:47
Speaker
So that doesn't mean anything, but you know, but he seems like a very nice guy and then doing fantastic work in closure. So I think we should we should we should we should record an extra special like little sponsor thing at the front of this episode. You know, yeah, I'll do that offline. Yes. Or I'll have a disclaimer about your interesting. Oh, yeah, that's right.
01:07:11
Speaker
The first fully corrupt show that we've had. We should call this the Trump Show or something. Well, there is no harm in telling people that there are opportunities available, so that's a nice thing. Of course, a couple of other things that the DCD thing coming up and what else we want to plug in this week, pretty much that's it, I think.
01:07:41
Speaker
Oh, of course. Ray is doing another show because he's becoming slowly the Rupert Murdoch of closure media. So he's doing a proper cast. I think some random crap that people shouldn't worry about. Anyway, I don't want to raise that show on this show. So I give it a little. What is a proper way? Destroy it.
01:08:04
Speaker
Exactly. Sure, go ahead. So the idea is that this shows, I think I really enjoy this show, doing this show because we get to speak to lots of different people. And I think it's really good for the community to hear the people's journey to closure and all the voices and the great people in the closure ecosystem.
01:08:24
Speaker
But sometimes people just want to talk about code, you know, sometimes people just want to see some code and talk about code. So what we're doing with apropos is actually just talking about a little bit of like bits and pieces that are in the zeitgeist of closure. And there's four people, it's the same four people every week, every two weeks or whatever. And we speak for about half an hour about the zeitgeisty type things, like it's closure script main at the moment, which is, you know,
01:08:50
Speaker
really fantastic innovation in the ClosureScript community. And one of the core ClosureScript contributors is actually on the panel, Mike Fikes, he's a really great guy. And what we then do at the end, after the half an hour discussion, we then spend half an hour like live coding in a REPL and we share the REPL and it's all broadcast live. And that gives rise, we just do simple exercises or simple code examples
01:09:18
Speaker
But it gives rise to all kinds of discussions about the various ways in which you could implement this function or how can you count those numbers? Is it a functional style? Are you doing things stateful? How do you evolve these little functions? We've only done two so far, but they've been great fun and I think they've had a lot of good feedback.
01:09:38
Speaker
Yeah. That was more than a minute, I think. I mean, I would I would I would love to endorse the show, but I haven't been paid. So fucking so that sounds like a decent show, I think. I didn't spend two hours yet on that one, only one hour so far. So it's really fun. It's really I know it's it's really nice to see the code, you know, rebel on the screen and then people trying different things. And it's very, very educational.
01:10:03
Speaker
Anyway, so, of course, you know, I think if there are no other topics that we want to discuss, maybe we can roll the credits. Pierre, anything else to discuss? Not on the top of my head. No worries. That's really good. It's been a fantastic conversation. Then you need to hold your silence until the next show. It's been really good.
01:10:30
Speaker
The kind of stuff that you're doing in Exoscale, like VGS has bet in your business on closure is quite amazing, I think, for people listening out there. And it gives people confidence that they can do it also. There are people that really believe in a language this strongly. And I think the community sometimes needs to hear those things that, you know,
01:10:57
Speaker
that there was a big set of beliefs out there. And you can do very big projects, not just kind of one-liners at the repl. Yeah, that's for sure. You know, things on the side. I hear a lot of people saying, you know, in their microservice architecture, they have this one thing. That's the closure, but it's more of an oddity, really, because, yes. Or, I mean, yeah, the one thing we
01:11:26
Speaker
Or maybe just if there's one thing I'd like to mention is that another good example of this in the open source world is Riemann, which I also have maintained a little bit, even though I'm a bit more, I'm a bit less active on that front lately, but it's found its way inside huge companies. I worked with, I mean, I worked with huge tacos, or huge
01:11:55
Speaker
providers of technologies for telcos, a Swedish one, not to name it, who relies extensively on women. And the places where you would see that pop up, you'd never think it would end up there. But still, I mean,
01:12:17
Speaker
It has its words. It's a tool for monitoring that made the choice to have Lisp as its configuration language. So let's say that most of the time in these large companies,
01:12:34
Speaker
the guy that or the person that introduced Riemann isn't the most loved person in the company. Sometimes she doesn't have any friends anymore. This is not a good sell, by the way.
01:12:53
Speaker
But if it's still... You've got to turn it around somehow, you know? You've set yourself a high degree of difficulty here. You're going into the diving board. How are you going to turn it around? But still, it's still there. It's because it's fulfilling a function that's really hard to fulfill in these companies. And I think still, I mean, now they have various tools that are catching up in that system. But, you know,
Riemann as a Monitoring Tool
01:13:20
Speaker
And Riemann started in, I think, 2012 or 2011. There have been two Brazilian cultures that really made an impact. And so, yes, people should shy away from that.
01:13:35
Speaker
the language even for things that don't seem to naturally fit in its current niche. Okay. So, of course, it's been a fantastic discussion. Thanks a lot for, Pierre, taking the time and spending the time with us.
01:13:50
Speaker
explaining all the things and again one more time a big thank you for supporting Dutch closure day as an organizer i mean it's it's it's a bit of a tough thing to organize a free conference but you know it's amazing to get your support thank you for putting on the show because not not in the tech world but in another world island i've put on the shows and
01:14:10
Speaker
It's not a fun job. Well, it is mostly fun. For the speakers or the attendees, it's nice because you get to see everyone, to have a beer or two, and to enjoy time with people who share similar interests. And it makes a huge impact on everyone. So really happy to participate. Thanks.
01:14:37
Speaker
And of course, I mean, I hope people will apply at XO scale and then your team will grow and then you'll make more and more closure in Switzerland. And by the way, I would love to do your assignment. So maybe not this week, probably I'll get in touch with you because I'm really curious now. I'm not particularly looking for a job. So, you know, I can be very defensive about it. I can just fail it spectacularly.
01:15:03
Speaker
Then I can wing it off like, no, no, I'm actually not looking for a job. Things got busy. But yeah, exactly. But I still love to try that one. It sounds really exciting, the kind of closure that you're writing. It's a bit off the beaten path, I would say. So I think that's it. That's it from us. A big thank you one more time. And of course, a big shout out to all the people who are supporting us on Patreon.
01:15:31
Speaker
We were lucky to meet some of them at ClosureD. We had a lot of fun there. I gave a talk that most of the people didn't walk out and very entertained people with singing and all that stuff. So we had a super fun at ClosureD. And again, thank you for supporting us on Patreon. And we are hopefully bringing some good entertainment and good infotainment to your ears.
01:15:59
Speaker
and check out UpropoCast as well because you know yeah Ray is trying to plug that show in this show so please check that out and have fun there and then go and dislike it and then like this show so I didn't mention it okay yeah I'm just doing my part you know like bad marketing for your show all right thanks very much
01:16:23
Speaker
Anyway, that's it from us. Episode number 33 with Pierre-Eve Richard. Did a good job, man. From Exascale. Hopefully, we'll see you. Thank you. Bye.
01:17:11
Speaker
So before we start, I'm going to tell you my joke, okay? Because that's... Yes, please. You have to be tortured first. Okay, now then.
01:17:22
Speaker
ruin the show oh christ okay um there's a rumor going around you know that um all of the bots that are harshing on the Haskell type system are written in closure but there's no proof all right