Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#93 Malcolm Sparks Returns!
112 Plays1 year ago
Malcolm Sparks, CTO of JUXT (the team behind XTDB, Yada and more) returns! We talk state of software, OSS, and most importantly Malcolms Atomic Architecture (https://www.juxt.pro/blog/atomic-architecture/)
Recommended
Transcript
Episode Banter and Introductions
00:00:15
Speaker
Hello everyone, welcome to Defend episode number... 93 says that. Yes, I think so. I don't know, I just keep adding some number by looking at the previous one. I'm not sure if it is incremental or monotonically increasing. I have no idea anymore. It might as well be like a 200 episode or 100 thing. But yeah, mystery.
00:00:38
Speaker
What am I going to say? Yeah. Welcome to DeafN. And episode number of something. This is Vijay from Holland. And Ray, as usual. And then we have the second. Is it the second one? Or is it the third time you're coming, Malcolm? I don't know. I have to say, after 93 episodes, the intro is getting really polished and professional, Vijay. That's true, right? Yeah, yeah, yeah. I apologize for that. I mean, this is not something that you expect on this show.
00:01:08
Speaker
I get it, I get it. But hey, welcome, welcome to the podcast again. So we were talking about, what, emacs and, oh, what happened to Ray's video? Okay. Yeah, I've just turned it off for a second. Don't worry. Oh, okay, okay. Yeah. It's great audio here. We can't see him and luckily you can't either. So we can continue. It's much more fun now.
00:01:37
Speaker
Yeah. Polished as ever, like you say, Malcolm. What a star this blog is. Right, come on. Who is this blog? Anyway, Malcolm, this Malcolm blog that we've heard all about. Do you want me to introduce myself? Yes, sir. I thought Vijay was going to do it there. It's like in a super polished way. I mean, he just said it's super polished and I'm insulted now, so I'm not going to do that anymore.
00:02:01
Speaker
We're going to remove the police now. I'm Malcolm Sparks, a closure programmer, and I am the CEO of a company called Juxt in the UK, and Juxt are a kind of closure projects, software engineering firm, and doing projects with clients and a lot of stuff around.
Clojure and Simplifying Complexity
00:02:20
Speaker
Closure, but now lots of other languages as well, so kind of branched out from closure, but sort of my heart is still very much in the closure language and a lot of the things I do and think about have been inspired by that journey.
00:02:37
Speaker
Yeah. So also it's the X2DB, right? The product that you guys are working on as well. Yeah. I mean, there's a lot of things that just over the years have sort of, you know, through the nature of consulting is you're right there by, you know, the trenches and you discover things, you reflect on things. So X2DB is one of those. And so it's, I think, you know, X2DB was really us trying to
00:02:58
Speaker
figure out how to pull out a really complicated thread inside of most projects, which is the thread of time. So, you know, we've got this kind of philosophy around the, you know, simplifying what the complexity is really that idea from Rich Hickey, simple made easy about complexity comes from these
00:03:18
Speaker
threads that are woven and tangled up which really should remain unwoven and untangled and so we conceptually kind of identify there's four constantly four threads that are always tangled up in systems and we've laid with them
00:03:34
Speaker
time, data, form and code. But, you know, it's kind of a fun way, a kind of set of principles, really, that if you try and keep things untangled, it leads to simpler systems. And that's the bet that we're trying to research and see, is that true? And, yeah, we were just talking before the program about, you know, the deep complexities of life in ops.
Are Developers the 'Bad Guys'?
00:03:57
Speaker
And I think, I mean, all these kind of things, all kind of, they all come from the same sort of
00:04:04
Speaker
font, you know, they all, they all come from the same well, which is the, you know, like just very, very, very early little complexities. It starts off very early and then kind of gets amplified and amplified. And then you get into such a mess. You can't really go back and you're, you're stuck. Welcome. One thing, I mean, I don't know, it's a bit of a sort of sideways question or sort of weird question, maybe, but I'll ask it anyway. But
00:04:33
Speaker
One of the things I've noticed in my career in computing is that you've got a lot of these people that are talking about how it's all gone wrong. And it's all gone wrong because these commercial people keep on asking us to do things that are compromising things, they're doing things in a hurry. And yet, it feels to me like that's a lot of horseshit. Actually, we seem to be doing most of these things to ourselves.
00:04:59
Speaker
Yeah. So, you know, I think it's about time like we in the industry started and, you know, as developers started to own that a bit as well. Yeah. That we can't like just blame it on business and people like in product or whatever, you know, that we have to actually take responsibility for some of the shit that we're dealing with here. I think so. I think we have to ask the question, are we the bad guys?
00:05:25
Speaker
Are we the baddies? It kind of feels a bit like we are, though, you know, a little bit like we've kind of got a bit drunk in our own supply, a bit high in our own supply, you know, we think we can make all these things work. And then we then we hit some sort of barriers of complexity that we should have known what we would reach. We could have known if we read the papers from 20 or 30 or 50 years ago.
00:05:49
Speaker
Yeah. It's not that black and white, right? No, I know. That's the point about the conversation, but I do feel like that we kind of like rush headlong into certain things and feel like we're doing great work, and then we hit boom.
Complexity in Modern Development Practices
00:06:04
Speaker
Yeah. Complexity. Come up with some examples because I was reading a good example of that. We were talking about DevOps before where we liked it. Yeah. Have the devs made ops worse by trying to do ops themselves.
00:06:17
Speaker
I've never made it much more complicated. Well, an example, I mean, I'll give you a recent example of something that work we were we were designing a new project that just this last couple of weeks. And one of the things that came up was that we were we were looking at sort of like billing systems, essentially. And, you know, every billing system has some every every company has billing, you know, but an attempt to happen the end of the month or or every month or every quarter or whatever, whenever they happen.
00:06:48
Speaker
The question is, how do you actually manage that? How do you manage that regular cadence of time? Because there clearly is time there. Then how do you make sure that when something goes wrong, you don't miss a bill or you don't miss some parts of that bill?
00:07:09
Speaker
Yeah, it was quite interesting to think about how, although it's completely reliant upon time, how we can get time out of that system. So we ended up doing was essentially like, you know, this functional core imperative shell type thing where the, the schedule isn't the time, the little clocks that tick over that are at the very edge and eventually just produce some data and these bits of data have actual time in them.
00:07:35
Speaker
They're the ones that have the time in them, and then all you're doing in your systems downstream is just relying on data that happens to have some time. That's right. When you test systems that are
00:07:48
Speaker
very time-based and you create a simulation, you know, you simulate tests and you control the clock and you use your test cases and I kind of feel that in fact that's how all systems should be built. Actually they should all be simulations, they should all control time and there shouldn't be any physics in computer systems because physics is, you know, so race conditions and timing and, you know, like latencies, all of those things, building on that mess of
00:08:19
Speaker
physics is going to lead to horrendous complexity. What we're really looking for in computer science is abstractions that actually say, hey, yeah, physics and the universe is really, really complex. What we're going to do is massively simplify it for your business. You don't have to worry about the complexity of physics, and you can actually worry about this really pure abstract mathematical
00:08:39
Speaker
thing, which is never going to go wrong. We're never going to run out
Critique of Web Practices and Simplicity
00:08:43
Speaker
of numbers. It's going to always be. I think that's the trick. And we failed to do that. And especially in ops, we've let physics back in. And so everything then is, you know, things moving around pipes and kind of timing, latencies are all kind of a source of great complexity. And that's kind of what, you know, I don't want to go there now, but it's kind of
00:09:08
Speaker
one of the sort of feelings I had about, we've got to start again, we're moving state all back into the same place and sort of changing it consistently one frame after the other. We've got to make the whole thing like a computer game again and not like a kind of, yeah, not like a real world, sorry. Distributed state. Distribute, yeah. Even though the world distributes state, we shouldn't because if we do, we're then,
00:09:38
Speaker
making things much more real world, but the real world is nasty and complicated. We wouldn't have needed computers otherwise, we would just do it. The computers isn't just making things faster and automating and speeding things up. The computers were about being able to bring some mathematical abstractions to solve problems that we wouldn't have been able to solve.
00:10:05
Speaker
Well, I mean, if you think about the very early times of logarithmic tables and stuff like that, those were essentially a kind of cache of pre-computed data. And that was really handy for navigation and stuff like that. And the same way, you want to try and build those kind of systems which are just, like you say, flawless.
00:10:30
Speaker
like the listex tables, you know, working out the notion of being able to work out how far a particular cannibal will, you know, to be able to work that out with maths is much deeper than just doing thousands and thousands of experiments with live cannibals because...
00:10:46
Speaker
You win more wars that way, that's for sure. But maybe we shouldn't talk about launching the nukes and things like this because I feel like the world is full of this kind of stuff at the moment. I was struck by, well just as in spirit, finding other examples where devs have been bad baddies. I was watching today, just looking at how can you use this
00:11:08
Speaker
I think this afternoon. And there's an article about HTML first, and it's essentially about the front end and about how we started off as on a really nice HTML, you could view source it and, you know, for somehow devs brought all of the baggage from the back end to this actually quite beautiful thing, early browsers.
00:11:25
Speaker
had HTML. Everyone thinks, oh, that's such a simple age. The age before it wasn't like we just had computers invented the day before. It was actually a great application. It's a wonderful thing. So people would think about the web, the early HTML, the browsers, and all that stuff. Oh, it's so simple and too simplistic. So we brought all this tooling and we've ended up with stuff like React where everything has to be
00:11:53
Speaker
like you have to build chain and tool chain. It all has to kind of spit out this gargantuan JavaScript file, which is totally impenetrable. And actually, this article was saying, no, we want to go back to HTML, where you can right click and view source and you can teach people and then read it. And they were talking about
00:12:14
Speaker
Tailwind and CSS and really just about trying to make everything open and accessible and tangible again. And I think that's another thing. I've actually really grown disliked for build chains and anything that actually takes
00:12:33
Speaker
any source code and turns it into something else. It's a non-reversal operation and you've just made it harder for the person receiving the compiled derived artifact to figure out what went wrong. Since we're talking about closure, by the way, I think let's bring it right back home. Why the fuck do we not just ship code?
00:12:57
Speaker
Like I, I, I've been saying this for like, I don't know, God knows how many years, but why are people compiling code in closure? Why? I mean, okay. For closure script, you have to do it. Forget that for the moment.
00:13:08
Speaker
Well, although let's talk about skittle in a moment, but, you know, and what bulk dude is doing to make that really a possibility. If you've got closure, you put the code that you run it. There's no change required. Now people will bang on about performance and stuff like that. And I get it, but that should be an edge case, you know, prove to me that your performance is really terrible and it needs to be compiled. Then you can add that step in if you really need to, but that should be, that should be something which is like not normal.
00:13:38
Speaker
You know, it should be something which is
00:13:41
Speaker
normally you just put the code on the server and run it like PHP or something. We've got a list. The design of it that because we are running on JVM and I think as Malcolm, you're pointing out that it turns into something else before it runs, right? So it's also going to turn into JVM. We don't notice that. That's the whole point about closure is we don't care about that. Yeah. Yeah. The closure code does get shipped and that's kind of one of the
00:14:08
Speaker
advantages of closure over Java because in Java, make the point a lot that we had class files. So you've opened up a Java file, you get class files, and then you decompile it. You really benefit from just seeing the closure files as they are, and that they are, of course, compiled.
00:14:25
Speaker
but dynamically during pass loading. And those optimizations are done, but they're done in such a way to make them invisible to most people. You're only seeing that. And then, of course, the JVM does that with its hotspot compiler. Yeah, exactly. And so I think definitely
00:14:48
Speaker
Do build steps, but do them just in time and make them completely invisible. But the problem is that the build step that we bring, you know, the whole React tool chain is not invisible at all. You know, to the people who receive that code in the browser, just see this kind of completely opaque.
00:15:03
Speaker
But isn't this because of the cargo culting or maybe i'm gonna use my newly learned humanities knowledge like the grand narrative buying into some sort of a grand narrative right because you you start with oh you know everybody is you're doing javascript we did plain html cgi stuff pearl cgi and all that shit.
00:15:23
Speaker
When I started, that was OK. You have some templates here and there. And then there is a bit of reusability. So when I was writing it using Perl and CGI and Apache, that was the beginning of it. And then we started having this. When Gmail came onto the scene, that is the explosion of this spa or single page applications, JavaScript being doing all this magical stuff in the browser. And people were blown away by it.
00:15:47
Speaker
Because until then, we were used to this rich user interfaces on the desktop, and everybody wanted to have those things in the browser. And removing the issue of software delivery, because you don't need to build and send it, that was the buy-in. And then everyone started jumping onto this bandwagon, regardless of whether there is a use case or not. And now, if you're talking about React, for example,
00:16:13
Speaker
every application has to be react for some reason. Every application needs to start with this 200,000 NPM package JSON yarn install. And it seems like
JavaScript's Dominance and Challenges
00:16:25
Speaker
going back to, are we the baddies? And the reason being thinking that there is a grand narrative when it is not, everything is like a local ones, which are much more optimized. But on the other hand, what I see is when you buy into the local narratives, like closure, for example, then you're limited by tooling, then you're limited by
00:16:43
Speaker
you know the libraries then you're limited by the community. I think it's kind of a weird situation. I don't know if if are we the bad you know are we the baddies yes when you buy into the grand narratives but we are still the baddies when the smaller communities don't have the same level of support. Does it does it make sense or is it am I ranting now?
00:17:10
Speaker
I don't know what the right answer is. I mean, I do, I almost think we're in a fortunate position to have JavaScript running natively in browsers. That's a really good thing. It's not, you know, perhaps in the future, it'd be Wasm or something. We'll have to write everything Rust and compile it. And, you know, it would be a great shame when browsers turn off support for JavaScript because the ability to just go into the console and, you know, it's really a kind of,
00:17:42
Speaker
That doesn't seem to be like a simpler idea of why everyone is writing JavaScript because it's kind of a self-fulfilling prophecy now because there is more tooling available and more training material available.
00:17:57
Speaker
debugging tools available, which is a bad thing. That means you're constantly debugging shit, but there are tools there. But isn't it really just like the, what should we say? It's like the unexpected consequences of success because really JavaScript itself, I mean, you know, if you're saying who's the bad guys, I mean, you would argue that, I'm not going to say Brendan Eich was a bad guy because he wasn't, you know, but he kind of was in the sense that
00:18:26
Speaker
uh, or maybe the people that again, it's like coming back to the point I was making originally, which is that. To some extent, the people worth saying, Oh, you know, you've got to make this language and it's got to look, it's got to be looking a little bit like Java or it's got to have the curly braces or whatever, even though you don't want that or.
00:18:44
Speaker
It's got to have certain things that we want. And then whatever, he took a couple of weeks to make it and now we've got this JavaScript. And then for whatever reason, the tech debt wasn't paid back. I don't know if he's like, oh, actually, I'm quite happy with this equals thing. This all works out for me. I don't give a shit.
00:19:09
Speaker
or whether he feels like not a number is fine or whatever is undefined. It means these are the things. But that's the sort of technical debt of that language that
00:19:21
Speaker
In the end, it was part of the browser, which was a massive success. And so now we're just all contending with that platform. And all of these things you're talking about in terms of code delivery and tree shaking and all these kind of things are optimizations of delivery. I mean, you know, the wonderful thing about the web is delivery of software on demand, but then you want to optimize for it. So, you know, you get back into these kind of, uh,
00:19:45
Speaker
these bits of work that people do to optimize for that delivery because they don't want to, you know, they want to say, well, why, why should every person, you know, pay, especially if they're paying on their phone bills or whatever, or their data plans, why should they be downloading a hundred meg when they can just down 10 meg because we can tree shake it out. So, you know, I think we're living in a kind of, that's an optimizing world, isn't it? You know, that's a kind of a world of pain on the backend to make it a bit nicer on the front end.
00:20:15
Speaker
There's reasons for it. There's always reasons for these things. No one's doing something without good cause, but the problem is this narrative thing. No, but I think the narrative thing is that you're hitting the nail on the head there, Vijay, I think, because the narrative thing is the problem, is that why does everything have to be like this?
00:20:35
Speaker
Not everything has to be like that. We find the boundaries between what has to be like this and what can be like something else. That's the constant struggle of computing, I feel. I think it's a search for the right abstractions and if everybody is
00:20:55
Speaker
doing free baking and knows about free shaking if that even is a word that has common and it's in the common developer lexicon then we sort of failed because like it should be there's a huge amount of work that's gone into jvm garbage collection over the years right and then there's people who understand.
00:21:13
Speaker
garbage collectors in incredible detail and they would know but most people just use them and they don't even have to know they existed most of the time it's just something that they've built into so that that is an abstraction that has really succeeded and it kind of felt that it really
00:21:29
Speaker
should have got to the point where web development had that. I've always looked at this as an indication we've got the wrong model of HTTP. We've talked about CGI. I think CGI was a good placeholder model, but we should have run away from it eventually. And in fact, we didn't. We're still in the CGI. So CGI stands for Common Gateway Interface. And it's this idea that the web server
00:21:56
Speaker
When it can't when it isn't serving a file for us and wants to do something dynamic, it has to call a program and the way it does that, it says, well, I'm going to call a program and here's everything I know about the request and just tell me everything. Just give hand me on a plate the response and I will send that to the client. And then that's the interface. It's so low level.
00:22:17
Speaker
that the programmers of that, the back end of that CGI have to know everything about HTTP request and HTTP responses such that everybody in the world who does web development has to be a garbage collection expert or has to work at that.
00:22:33
Speaker
So that's where I think the CGI as an abstraction is the wrong one. You know the wrong one because people are working, too many people are working at too low a level and a low level is bad. You need to have experts who understand the lower levels, know how plumbing and sewage and steam engines and stuff work. But then the rest of us should be standing on the shoulders of those giants and being able to reach yet higher
00:23:03
Speaker
higher things.
HTTP and the Need for Simplicity
00:23:04
Speaker
But when we don't, we're trading on the toes of dwarves or something else. We're all standing in mud. It's just layers and layers of mud. In terms of the CGI thing then, Malcolm, what kind of thing are you thinking? Because we're always talking about
00:23:30
Speaker
Goddamn HTTP codes and stuff like this. That's a bike shedding dream, isn't it? Oh, let's spend a week deciding whether it's two or four or two or three or two or one or whatever the shit. Is that what you're talking about? Should we be having a better set of abstractions over that? Yeah, totally. In the same ways we don't.
00:23:53
Speaker
Really anyone who writes an FTP client or anytime you use an FTP library or use a TCP library to send a packet to some you don't have to do i have to.
00:24:05
Speaker
Is this IP version 4 or version 6? Or where do I put the destination IP address? You don't. You just create a TCP socket in whatever language you just send something. You don't have to worry about the details of the TCP IP approach. You certainly don't talk about return codes and accent and all that. I can't remember that stuff. And I can't remember it because I don't use it every day, thank goodness. If you don't know about Nagel's algorithm, then I don't want to speak to you.
00:24:34
Speaker
I don't, I know, is it John Nagel that came up with it and it was his algorithm? I don't know, it was something to do with like...
00:24:42
Speaker
The thing is, you know, that's the thing. That's how, like, how does a human progress work? Because other people know all the hard stuff, so you don't need to. Yeah, exactly. How do we make nuclear reactors? Not by all of the tooling laws. It's crazy, really. I mean, it's the same everywhere, right? I mean, making food, making, you know, whatever that you want, you know, if you want to make it make a table, you're not going to OK, let me find some land and then plant a tree and then
00:25:11
Speaker
get the wood later. I've done, you know, build my own tools. Yeah, exactly. I mean, it's not gonna happen. You know, the key spark of any progress is if you got everybody making their own food and making their own cakes and you know, then yeah, it's fine. They can get you in a bit. You just everything's a cottage industry until somebody figures out how to like, what are the right organization of labor to actually start then the trend to kind of
00:25:36
Speaker
make it ever cheaper to create cakes. And then you start manufacturing and you start being... But there is the question of the line, right? Because you can say, okay, everybody is just writing their own abstractions and then repeating the same grunt work. On the other hand,
00:25:55
Speaker
everybody's using J2EE and then writing super complex EJBs and shit. That seems to be the struggle, right? I mean, compared to Clojure and compared to other languages. And even today after, I think it's been 12 years, 15 years in the community, looking at these communities, every now and then someone comes in and then says, is there something like Rails here? Is there something like, you know, that there is always seems to be like people who are, oh, it's a,
00:26:22
Speaker
you know, libraries that I'm going to plug together. And that's where I have my freedom. And then on the other hand, the cargo-culting level, like, oh, give me ABC generate and then continue. But come into that though, Vijay, isn't it? Because like, let's say we, as an industry, we basically, again, I think there's a lot of, um, there's a lot of arrogance in the industry.
00:26:51
Speaker
And there's a lot of, um, there's a lot of thinking that we're, they're much, they're much further along than we really are. Um, you know, when I, when I started, I was using like, you know, and C and, um, then eventually, um, Java, but, but using C felt terrible. I mean, you know, there's lots of 4GL languages, you know, there's lots of languages that were meant to be better.
00:27:16
Speaker
But, you know, we're 30 years on now and sea and rust and things like that are still, I mean, they're very, very ridiculously low level languages. Um, and even closure is quite a low level language. Um, you know, so we're, we're still stuck there. We're still stuck at that kind of, we can't escape that level of language. I mean, and I don't know what you feel about that, Malc. I mean, whether, cause it's like this notion that we're,
00:27:43
Speaker
We're still stuck at the very beginning. Yeah, we are. And I think, just let me, with this kind of search for the right abstraction, the problem is the right abstraction is 0.001% of all abstractions that you can find. There's less wrong avenues. And actually, the benefit of languages like Clojure is that they just let you go through and
00:28:10
Speaker
because they let you turn over and search, they're search tools really. You're searching and you can get through a load of bad obstructions quickly. You've got tight feedback loops and you can play around with things and that was actually one of the point we were talking to on the Jugscast, not to plug another podcast on your show, but with Ken Beck. Well, there are other podcasts in Closure. This is the only one.
00:28:38
Speaker
Dropping the cat back as well. I mean, Jesus Christ, you know. Well, that's all we have in this show. That's it for the episode, guys. And then we're going to, you know, remove the redact the mentions of competing products on this show. Anyway, but seriously, let us do a podcast with great big names, come on.
00:29:07
Speaker
His argument for test-driven development was that it was just a tool to be able to try out lots and lots of solutions really quickly and get rid of all the rubbish funds. It's about the speed at which you can... But the attitude I think is to know that most of your solutions, especially the early ones,
00:29:28
Speaker
are not the right ones, and you should actually be very careful about going for the first choice. And with Richard, his talk at Closure Conge earlier this year, it was about, you know, stop and create a decision matrix or a spreadsheet, just start enumerating your solutions. Don't just go for the first one. And unfortunately, I think test-driven development, that spirit of research and exploration was lost, and it became just a quick way of
00:29:56
Speaker
getting everybody to get to the keyboard and building working software. It doesn't matter. You'll be able to change it. But go for the first solution. And sometimes that first solution, unless you've got that spirit of, well, the first solution is likely to be wrong. So many of these development teams, you're under such pressure to implement the next feature. You've got half an hour. By the time you've done your first solution, you've run out of time. That's it. Go on to the next thing. And so we just build pile on, pile on all of these
00:30:26
Speaker
poor layers and we're into the layer up layer layer and we think all those layers are right but they're not they're just they could be there's so many different alternatives that we could have found and I think some of them if you find the right ones and you know we the CGI think it's just that nobody has really challenged that because it's got so it's become so entrenched that it's almost very difficult to show people that there are probably 100 other ways of doing the same problem
00:30:53
Speaker
Well, the classic way that I've addressed that is WebSockets. And then people get very mad at you for going with WebSockets because they say, oh, you're making a new protocol for every single
00:31:04
Speaker
API or every single call i'm thinking you're doing that on the fucking rest services anyway Yeah, you know half these people are not using they're not using this I don't know what fraction of people use hatios but you know I don't even think why fieldings using hatios for his own website, you know, I imagine that you know hyper and the whole idea of hypermedia as the engine of state or whatever, you know this I mean people don't know even know what that means, you know, you're talking about abstractions, you know, i mean hatios was like
00:31:34
Speaker
Maybe it's an interesting idea, but it's like, it's totally fallen on the floor. No one's picking that up. Well, one person is the H.G.M.X. person is picking that up. Well, yeah, yes and no, I would say. Don't you think that like this whole idea of, well, okay, maybe you can explain to me why that is happening. So rather than me sort of,
00:32:02
Speaker
fighting fighting in the mist you can explain why that is why also we've had discussions of hmx recently but i think a lot of people i know i know vaguely what it is now but maybe it's for everyone get on the same page what what do you uh what are you talking about this hmx magic well i mean hatios is about the
00:32:27
Speaker
your application goes through a number of states and that is you can map that onto really a user clicking on links and it goes to go to the next state and so the next date in each of the so when you look at a web page that really is the state of an application it could be generated or something but it has all of the
00:32:48
Speaker
the links are the paths in a state transition diagram to go to the next state. And you can define the whole application as state transition diagram. And I think that is an idea that I think is because we've really turned the web away from hyperlinks and we've turned it into this client server thing.
00:33:17
Speaker
people are going back to the thinking well actually there was something in links you know that like when you you know when you can't even click you know when you can't click a link because you get a pop-up or something it's frustrating when you you know people sort of like i really love links when you you know you're the magic of being on a kind of wikipedia page and then saying oh no i wonder what you know you're looking up some
00:33:41
Speaker
Thomas Jefferson, I don't know much about how to click on him and then he was inventors something, you know, and you just you just go off and you learn, you know, you explore things with links. And I think it wasn't the
00:33:54
Speaker
wasn't the idea behind this back linked or the traceable exploratory document version of the web servicing a specific type of file, or maybe most of the web exploration kind of use case. But I'm just going to throw in a question like, OK, so what is wrong with the, sorry, it's my bike.
00:34:16
Speaker
What is wrong with the client server thing? Because that was there before HTTP, right? We have finger, we have go for it. All sorts of protocols on the web before, which basically disappeared because of the HTTP becoming the predominant thing. What do you think is the... We had lots of client applications and things written. Visual C++ and apps like we have today. What was wrong with it?
00:34:46
Speaker
It was expensive because everybody had to have a development team write each kind.
Browser Dominance and Market Influence
00:34:52
Speaker
Because it was able to create a sort of standard HTML language, it meant that you could actually run a browser. You only had to have one team write a browser. So now we've only have, was it two browsers now? Chrome and derivatives and Firefox. And so there's only, when you talk about kind of economies of human, what are you talking about?
00:35:15
Speaker
you know, and society is creating a surplus, you know, and the actual idea is, you know, kind of be as optimal as possible. So you don't want to have to, you know, don't want to have people working on the same thing and reinventing the same wheel. Well, no one's reinventing the browser wheel anymore.
00:35:31
Speaker
Because yeah but then i mean i'm just gonna be contrarian for the sake of being contrarian i guess yeah but isn't that isn't that the whole reason why when we had this you know what is it called electron based apps being you know everywhere and then suddenly one security issue an electron app now it's.
00:35:53
Speaker
There are so many apps that we don't know are using Electron, and then we had the security issue from Slack to from every weird application using this one, then suddenly need to be, they're based on the same thing. Anyway, it's not a big deal. If you end up making things so efficient, there's no redundancy at all, that you're only
00:36:19
Speaker
um you know if you're only like surviving off one food type and because that's yeah you know and that that's not that's not sustainable yeah so you know there's if there's no redundancy at all but i think that the the
00:36:36
Speaker
it wasn't that the pre-existing world, there were no security problems because we had this, they were there. It's just that with Electron, you can fix it with a patch that you can distribute in an incredibly efficient way. I'm not saying that, so I think it's a bit of a different argument. It would definitely be you can have vulnerabilities if everybody uses the same stuff and you should
00:37:02
Speaker
avoid, certainly you don't want there to be a monoculture. When that monoculture, certainly when it's built on some proprietary standard owned by one monopoly like Microsoft or AWF, that's a terrible thing. I think the hope is that when you do arrive at these things that we can all agree on,
00:37:27
Speaker
that the that that person who governs you know there's no single company that govern that agreement that that is something that is done in the public domain and we will be in the comments and the comments are governed by the comments and not by private that's the date yeah that's the danger of like to me i mean that's a danger of chrome because firefox is basically the open source version i'm not of chrome but an open source browser and chrome
00:37:55
Speaker
I know they pay lip service to have in Chromium as the open source version of Chrome, but really it's something else, you know? Well, yeah. I mean, it's Chrome. I mean, that's the problem with people misunderstanding what a standard is. A standard is something that is.
00:38:10
Speaker
you write down something that is a very simple contract that anybody can, that permits lots of different implementations. So having group buddies like the Internet Engineering Task Force and W3C, they come up with the standard. The problem with the tragedy of the browser is that the implementers have become so important like Google.
00:38:32
Speaker
and that they've raced ahead and then created the standards are so vast now that they're sort of easy to capture. The cost of entry into the market, into the browser market is like the cost of entry into making a mobile phone with the GSM that the standard is just so
00:38:53
Speaker
so huge so many standards in gsm that you have to implement that there is no way that anyone company could ever enter that market ever again and so you end up with only a very small number of monopolists in that market. Yeah i think i mean basically if it wasn't for google trying to acquire traffic from firefox that would also run out of money yeah.
00:39:17
Speaker
yeah half a billion a year towards firefox just so they can be the default search engine yeah it's a sort of a little theater little game that they're playing to sort of yeah it's like a hand of distraction to not look like a monopoly pair but that's what monopolists do they hide in shadows until they've reached such an entrenched monopoly position they can come out and they're in and they're unassailable and that's so google are waiting for that
00:39:42
Speaker
But it's not a small project, right? Building a web browser. No, but the thing is, if there was more attention, more browsers in the browser market, and there was more attention to the importance of having lots and lots of layers, then the standards would move slower, actually, because it would take longer for getting people to agree. And we would have perhaps less features in our
00:40:08
Speaker
Browsers perhaps but then they again they would be better abstractions what we've done is we've just allowed browser manufacturers or like Google to race ahead very quickly and define what the standards are and at a rate that nobody else can can catch up with.
00:40:24
Speaker
Yeah, but there is no incentive for people to build these foundational software. I mean, especially when it is such complex, right? I mean, programming languages, it seems like these days they're dime a dozen. It's very easy to come up with your programming language because most of the tool chain has already been there thanks to GNU or thanks to all the open source software being there already.
Economic Incentives and Innovation in Software
00:40:44
Speaker
you know, built out of universities, you know, because I think that it all links to the economic reality of the thing, right? I mean, the entire point of Google designing a browser to make it everywhere so they can sell more ads, you know, I think the one of the talks that I saw at Strangeloop from creator of... Yeah. Yeah. He's talking about the real estate, you know, like the fantastic talk. Yeah. Really nice one. Yeah. So, so it seems like
00:41:12
Speaker
For Microsoft and Apple and potentially some Linux distributions, open source contributors have some incentive to build browser because it's like the next critical piece in the layers of software that people are going to use. So there was a couple of other browser engines on
00:41:32
Speaker
Certainly on all these things that were there already, but for Microsoft again operating system. So it makes sense and then for for Apple over instance, but for Google. Their only incentive was to make it so popular so they could make they could be the default search engine.
00:41:48
Speaker
Right. And if we are talking about, hey, there should let let there, you know, thousand flowers bloom. But but where is the incentive? Right. Writing this foundational software. It takes time. It takes money. But I think that's Markham's point, though, is that once they are a monopoly now and it's very difficult to unseat monopolies without regulatory effort or
00:42:08
Speaker
without some other big, big company like Apple, for example, deciding to put some big investment into it. And it kind of thought Microsoft would do it, but they obviously capitulated. And that's a worrying trend, I think. Yeah.
00:42:24
Speaker
Yeah. I mean, there might be something in the web standards that allow. I mean, I think that the standards are absolutely excellent in the protocol space in HTTP. And I think that they have really stood the test of time. And what I fear is that the number of browser-based API standards that have come in that have made it very difficult for people to
00:42:52
Speaker
catch up. But I think there could be. There are some solutions that come to mind where perhaps you could have an agreement on a micro profile of HTML and CSS and sort of go back to something that is more primitive, but that would say it might be much, much easier to create a rendering engine. And then certain sites would say, OK, we are going to implement not
00:43:19
Speaker
text slash HTML, but text slash micro HTML or something like that, you know, so that they can, they can produce a very kind of cut down version. I think there's been, you know, like in OpenGL, there was the webgl, again, there's much cut down. And I think there's kind of precedence for this, where we could really just now we've got this huge number of APIs to pick from, we could probably
00:43:47
Speaker
you know, go back to, you know, another problem is that a lot of websites are trying to become mobile apps, and they've got so much React, and so much JavaScript, they have to, so performance becomes massive, and then everybody wants it to look exactly the same, because in the early days, browsers would actually do a different rendering of, you know, sometimes, you know, IE and mosaic, and they would make a different
00:44:14
Speaker
Yeah. Well, you find these days you go to websites, you know, where it works for Chrome, but it doesn't work for Firefox. Yeah. And that's appalling in my opinion. Oh, this works. So then basically Chrome is the new IE6 or whatever, or whatever it was in IE3, you know, because now people end up like writing. I mean, the things that are like bad are things like security models or
00:44:44
Speaker
you know, the kind of the extensions that Chrome has made and that Google have made, sorry to things, you know, AMP works better on Chrome, for example. You know, there are certain add-ons above that are obviously horrible. I mean, AMP is one of the worst things, but my point anyway is that
00:45:07
Speaker
Once you have these, when you have these things and they start to be impossible to use up because the distribution of, of Google is now, oh, actually every corporation will pre the operating system, like the Mac comes with a browser, but the, a lot of people will pre will pre-install Chrome on top of the existing browser. Why? Why do they do that?
00:45:35
Speaker
Well, answer me now. You know, they shouldn't have to, you know, it's ludicrous. I use Brave slash Chrome for Zencast. So that is the only thing because this, this, you know, the thing that we are recording on right now, it works better on this one. And I don't have time to debug this shit. This thing does not work on Firefox. Yeah, exactly. Yeah. It's a great example.
00:46:00
Speaker
It was exactly the same 20 years ago where everybody had Windows, and everybody had to run Office, and you couldn't get an Office document from LibreOffice that would work with your Microsoft. So everybody, you had companies that says, we standardize on Microsoft Office, and then that obviously, but that meant that- You standardized on it, yeah, great.
00:46:22
Speaker
What happened is that very little incentive for Microsoft to innovate on many years and you get stagnation of innovation. And so the real problem is that standards are the thing. We've learned that standards are slow, but they are worth being patient and to work towards. They really
00:46:43
Speaker
They've got a long tail value. And by racing ahead to try and make the web browsers a media platform, the Netflix thing, everybody wanting to import proprietary codecs into browsers, which is a good method, it then became very difficult for open source browsers to succeed. And so we're learning those lessons over and over again that if you don't
00:47:11
Speaker
If you don't work hard to be patient and to work on standards, you end up handing the whole thing to the monopolist who wins, however they win. And then that's it. That whole industry comes to a halt. And not a shuddering halt, but it is slow decline until you realize that something else has to come.
00:47:37
Speaker
you know there was no like nothing you know when everything was when all the world's information was locked into microsoft word documents you know that that was there there was no other you could send them over emails and things like that but you know it it then stagnated at the point that that's what made the web so compelling because suddenly that was a new thing that people could then access information free and and that's because of coming back to what the
00:48:03
Speaker
The hyperlink really was there, but that's the hyperlink kind of saved us from this locking it allowed.
00:48:13
Speaker
to hyperlink from one domain or one website all the way to something else. So it was a very anti-locking device. I love this idea that what's it called that or the musk boy said that I'm going to prevent links running from X to other competing social networks.
00:48:38
Speaker
That's because these guys are crazy. I mean, you know, you're right. I mean, you know, I'm not trying to blame the commercial people in the end, because I want to try and stay a lower level. But, you know, since we sort of came up to that point, you know, sometimes these links can be kind of this openness can be, you know, some people don't like it, basically. Yeah, no, but I think that the
00:49:01
Speaker
The hyperlink is the kind of most valuable tool in terms of decentralization against power, against centralization of power because the hyperlink is really a very, very simple device which allows you to
00:49:17
Speaker
breakthrough borders and walls and I can have a hyperlink here and somebody can click on they will go off to China and they can go to another page in China click a link and go to New Zealand and it's a really kind of very revolutionary device and it was so powerful it broke a whole Microsoft monopoly on information. So let's get back to I think closure then.
00:49:49
Speaker
We have, we have traversed the world. We surf the web like good old days. We, we went, I think our conversation kept clicking links again and again, then went into all sorts of directions, hyperlink discussion. So, so, you know, Malcolm, given your experience from, from CGI days and all the way up till now, you know, with the React stuff and, um, so
Atomic Architecture and Centralized State
00:50:16
Speaker
Because we wanted to talk about the architectural thing that you mentioned in your blog and a few other places already. Yeah. The atomic architecture thing. So what is this thing and what is novelty and what is something that you...
00:50:36
Speaker
experience-based, you know, oh, this is a lesson learned, so this architecture is going to solve these kind of issues. Yeah, I mean, it's not an, it's a sort of concept architecture, but it's just, yeah, like a Frankenstein architecture, which has borrowed lots of pieces that I've seen, you know, and now I'm really selling it here, Malcolm, you know.
00:50:57
Speaker
We say, hey, what is this architecture? What is new? Well, it's not an architecture. It's a sort of ugly monster that's going to kill your kids. You know how there was a big in the last few years, a big debate about monolith and microservices.
00:51:16
Speaker
Microservices have been kind of the dominant mainstream architecture for coming up to about 10 years, I guess. I think we moved from Monolith to Microservices and then back to like Decca services or something, because Micro is too small. I don't know what, somewhere in between, like Senti services. A million would be the one, wouldn't it?
00:51:41
Speaker
Yeah, anyway, yeah, I think it was not so big. Well, Netflix kind of broke out into, you know, microservices. And I think it was microservices, the main, the main influence of microservices was being trapped into, you know, paralysis by very large inflexible. And I was reading something about how AWS really was kind of
00:52:04
Speaker
but the early AWS or the early Amazon at least was so rigid and so monolithic on so difficult to get anything done that it's, you know, that that spawned people, the only way people to get things done is just work on their own. And that's how it sort of evolved as a reaction. But I don't know, nobody talked about monoliths when, you know, in the same way, nobody talked about
00:52:27
Speaker
nobody used the word waterfall before the actual folks did, right? So the monolith is just the other, it's just the thing that has been... Yeah, yeah, yeah. It's a pejorative. Yeah, it's a pejorative. Yeah. So, but there has been, I mean, I think, you know, the way that people looked at, with microservices now, just see these kind of huge amounts of, when you're debugging them, you know, you just,
00:52:57
Speaker
how difficult it is to reason about distributed state. And I can give some examples, but we've all been there where we've got
00:53:08
Speaker
You say, well, you want to see where the state is between this service, has it come back? I've got telemetry. Look at the logs and then you need some centralized log thing to look through and then some UI. It's exactly what VJ was whinging about before we started. It's a Rube Goldberg machine. That's what we've created.
00:53:33
Speaker
That can't be. And then it just I think I'm really pitching a third way, which is totally understood. We don't want to go back to everybody working on the same.
00:53:46
Speaker
code base and tripping up each other. You know, we do want isolation, we want autonomy, we want independence from each other, right? And independence, that is, through having good contractual standards between teams and so on. But where the real is saying that 99% of the complexity of microservices actually comes from the fact that they're stateful. And so it's not that they're independent is that they're independently stateful.
00:54:12
Speaker
And so if you want to simplify it again, then you rebuild the monolith, or you go back to the monolith. The only thing you take is the stateful nature of the monolith. You create a headless monolith. You don't create any shared UI or anything. And you say, well, just let's put all of the state, at least. And that's called a database. That's what we used to have. And I've seen, in my career,
00:54:42
Speaker
huge systems that are built on a single database. In fact, I last
00:54:47
Speaker
last week I saw an incredible system that we were very, very fortunate to have a three-hour demo from the Beacon people who are creating, you know, the Beacon are a company that create risks, this risk system that you can bring into your bank or hedge fund, you know, and it gives you so many foundations. But that is a system that's entirely built from, there's just one database in the middle
00:55:13
Speaker
which is MongoDB, but it could be any database and that's distributed. Then everything else works. Every product is all described in a whole bunch of Python services. But all the Python services are stateless. They talk to this MongoDB and that's how they share and say things. It's a very atomic architecture in that sense.
00:55:42
Speaker
state in the middle and then let thousands and thousands of services which are all tailored to a particular user, a particular business process or goal or whatever. You can heavily customize services as long as you just put the state in center again. So the atomicity in your proposal of blueprint or architecture is the atomicity of the state in the middle.
00:56:11
Speaker
Well yes it is, well like all these things, it's a little bit of a play on words and a bit of a juggle, you know, you're just trying to juggle with words. So you can think of the atom as being, you know, if you want to say, oh it's atomic, it's like the
00:56:28
Speaker
the atomic model with the atomic nucleus and the electrons flying around it. Yeah, like the indivisible part that we're not going to spread it into multiple things. Well, yes. So you're saying that the nucleus is where the state, you know, the protons, and then all the electrons, all the very small little services and applications that are all kind of orbiting that. And that doesn't say, and you don't have to have one atom, you might have four or five
00:56:55
Speaker
I think the atom is really the department or the group of people who all agree that they're going to save the state in the same place and collaborate. So they might be kind of all, one might be inserting a loan application and the other one might be approving a loan application. Another one might be doing some sort of reporting or something, but they're all working off the same state. And so they're in the same domain. And we know that kind of from the domain
00:57:25
Speaker
design idea that everybody's agreeing with a bounded context and it's a common shared language. You can't scale that to thousands of people, but in fact you can organize companies into different departments, as we do, with groups of people who can share stuff and talk the same language. And then those groups of people can create boundaries from there, they can share their
00:57:53
Speaker
their state or share their functionality with other people in the organisation, but through a contractual interface. And I think that the best one is the API, the web API. I think that's a really very well accepted way of transferring. It's not the best interface
00:58:13
Speaker
perhaps for fast latency and things like that. But there's a boundary to your entire system where you can provide data feeds and you can provide an API surface. You can control
00:58:26
Speaker
what outsiders are allowed to do what what business processes that they can interact with and so you get the also the opportunity by creating a contractual boundary for you to tidy up things inside you get to encapsulate your your um you might decide to have one database at the center and then you want to split it into three but you can do all of that without upsetting or communicating that with your
00:58:54
Speaker
external neighbors because you're able to keep that same API surface. But in this design, what do you call that? The necessary services like the role-based access control or identity management and
00:59:13
Speaker
So will these things, because those are the ones that other servers need to talk to, right? And usually, that's how the microservice design I've seen starting, like, oh, there is a user service. There is a role service. There is some other shit or whatever. Things start to blow up from that direction, and then invoice service, and then this one. But invoice service need to talk to the identity service to get the token to make sure that this action can be performed.
00:59:42
Speaker
in your mind, are there some services? So if I understand it correctly, every service is trying to communicate via the shared state rather than, you know, conflicting and talking to each other. Or what is the communication mode?
00:59:59
Speaker
Yes, I guess it is if I was to submit a request for a loan and I would put it through some steps and that would then go into the database. And then if I was going to use another app saying show me all loans that are being requested. It's a bit like there's workflow systems that we'd have a central deck. Yeah. You know, so by one actor might make a
01:00:20
Speaker
like an API call and that will change the database which would change the use of other actors and that's something you don't get with microservices because you get the ability to report and to say right how many loan applications have we got and what status are they in and I can do that on a central database with a different report without having to
01:00:42
Speaker
go and visit and communicate with lots of other microservices, some of which may be down or some of which may give. And the other crucial thing, I think, with having all your data in one place is that you can change it consistently. And that's it.
01:00:56
Speaker
because you've got it in one place, it's then you're then able to treat your changes as atomic, you know, so you can say, my database is in state t zero, it's at state, you know, state zero, and then it can be a t one, it'll be in state one. And so those, those, you can think about it as a reduction of, you know, replying functions over a
01:01:20
Speaker
a value and so really the whole organization is a reduction of and if you can if you can do that then that that that's I call that domain operation so you you in designing your domain you come up with a common language but you say I'm going to allow these operations to be done might be request alone or that would be an operation and it's a kind of like an event source
01:01:47
Speaker
a loan was a requested sort of past 10. It's an operation to say, this is a command, this is something I want to do. And then if that is successfully approved by whatever access control mechanism you've got in place, then that is associated with some state change, but the state changes are applied one after the other.
01:02:11
Speaker
in a queue. It's just like a mainframe batch processing or job control system, one job at a time. The reason why that's useful is because it allows you to then understand how your state has got into the state it is by being able to go back in time, say, well, we did this and went and we did this and then we revoked VJ's commission and then we saw this.
Criticism of Microservices and Security
01:02:34
Speaker
It means that you've got this repeatable story, this simulation.
01:02:37
Speaker
and you don't have race conditions or side effects or the fact that we revoked VJ's access two milliseconds before you access the Malone request, but unfortunately it didn't get through because it wasn't seen against older data and caching or those sort of
01:02:54
Speaker
stale data that we made the decision on all these amazing number of edge cases that we have to code for disappear. If you go back to creating a very solid simulation, which is ultimately a reduction, you know, that either reduce the enclosure, that's what you're saying. That is such a useful idea.
01:03:14
Speaker
reducing over, you know, a series of functions, a sequence of functions over a value. Why don't we make the value the database? And then, you know, of course, people say, well, that won't scale. But actually, you know, when you when you really if you if you're able to submit these operations asynchronously into a queue, you know, queues are fine, you know, you get like, like, it's okay to funnel everybody through kind of
01:03:40
Speaker
metal detector at an airport and everyone gets through eventually and the flight might. In the end, that's how systems actually do scale, isn't it? That's what we found. In fact, that's what really happens. If you look at the Amazons and Netflix, etc., that's what they're always doing. They're always
01:04:00
Speaker
making these asynchronous things. So that's the mechanism for scaling, actually. Yeah. And then banking is banking financial transactions. Yeah, banking, yeah, absolutely. It's all obviously kind of at one cue, one done, you know. And I think it's all about an ordered list of transactions. If it wasn't ordered, you'd be screwed, you know?
01:04:21
Speaker
It depends on how much you care about the consistency and accuracy of your data. I mean, if you just don't care about it, then that's fine. But if you do care about it and it's people's money or it's people's bookings or it's their products that you're shipping them or it's their, you know,
01:04:36
Speaker
or health information or something like that. If it is of any importance to you, then you should, the standards at which we should treat data, consistency should be a given, right? There should be no way that I can make
01:04:52
Speaker
a system where two people managed to get the same hospital booking on the same day with the same doctor and then they turn up to the hospital and I'm afraid there should be no way that you should order something and it takes your money but then we didn't have enough.
01:05:10
Speaker
That is at the factory. That should not be possible, but we make it possible because we don't treat data with any respect or care. I think that goes through a lot of the complexities that we find in the modern world.
01:05:25
Speaker
Sometimes it feels like that's a business model. Actually, it feels like over filling seats on an airplane is a business model rather than a deliberate tactic. But at least you're doing it consistently. Yeah, exactly. Yeah. I think there's enough room in the plane. Oh, let me just check. Oh, no, I know there isn't, you know, it is sort of.
01:05:46
Speaker
I overfill it by like 20% because I know there's an attrition rate of about 18 to 21%, so I'll probably be okay. Yeah, exactly. I think it's going to be fun every time my bank thinks, well, I'm going to inflate his bank account by 20% because I expect he's going to have 20% and then I spend all the money and they're like, well, give me 20% back.
01:06:10
Speaker
like overbooking the planes and then they're overbooking my bank account. Anyway, so the state argument in terms of the technical things, you know, swinging back the pendulum towards, you know, shared state makes sense. But why do you think the pendulum has swung in the direction of microservices in the first place?
01:06:40
Speaker
Because there might be, are there organizational things, or the complexity of how teams are working in companies, or how is that going to be solved? Because it's not just one dimensional problem, right? I mean, there might be different issues at hand. Yeah, of course. There are lots of reasons why people organize. It's mostly, people call it Conway's law, but I don't know. Yeah, yeah, exactly. You know that, of course.
01:07:08
Speaker
you know if you've got like if you're at a wedding right and you've got seven tables and you've got 10 people on each table are you going to have high bandwidth conversations between you know it's just obviously the way you organize and yeah
01:07:25
Speaker
But what I'm trying to say is there isn't a technical reason to do it. I'm saying there isn't a form of reason, unless you get up to kind of like Google scale. But Google do things differently because they have to deal with billions. Not everybody needs to build a Google system. In fact, I'm not even sure that that
01:07:49
Speaker
that justifies microservices, the fact that you've got to get up to Google scale. I'm not even sure that that's even true. I think that microservices has more or less come about because of an organisational failure. It's a failure of technical leadership. Conway's law is a
01:08:10
Speaker
is people talk about Conway's law, like it's some sort of kind of law of thermodynamics or something. Yeah, yeah, yeah. I mean, I think that calling these things law like physics, it's kind of a weirdness. And it's not even true. It's a Conway's idea. Yeah, it's like, it's a totally... It's an assertion, yeah. It doesn't even stand up to
01:08:34
Speaker
cursory criticism. I mean it's like saying that I could open up the Fat Man nuclear atomic first atomic bomb that was dropped on his jaw. Somehow dissect it and open up the bomb and I'd be able to figure out then the organization. Well it blew up so.
01:08:57
Speaker
I was a poor example. Of course, if you leave everything to sort of random Brownian motion and if you had no leadership at all, then people are going to come together and they're going to cobble together solutions. Of course, that is a tendency. But the point of it all is this is all about
01:09:17
Speaker
leadership. I think Mel Conway's original paper talked about the kind of the design organization, that if your design organization is distributed and then you've got a lot, but we've done lots of things by ethical leadership with the internet, for example, right? The very existence of the internet and of IP, the internet protocol and the coming together and creating RFCs and going off, that is an enormous
01:09:47
Speaker
an exhibit of evidence that counters Conway's law. And I think people use Conway's law as a sort of excuse for failure of why it happened. And I can't see anybody. I just don't have any time for it, I'm afraid. It's just because I think a lot of IT is around leadership, right? And leadership is hard. And you have to have
01:10:14
Speaker
you know that today you have to be very technical and knowledge and as an engineer to you know leaders have to be very savvy engineering wise they have to understand development and deep learning and they have to understand they have to have a lot of soft skills they have to have a lot of organizational skills and leadership skills it's a it's a tough thing to ask people to do but
01:10:36
Speaker
I think what you're doing now, Malcolm, which is interesting, is that you're kind of fulfilling a gap because the problem that leaders have is that they have to arm themselves with information. And there's a lot of information out there. There's a lot of converse law. There's a lot of other laws, a lot of microservices, a lot of tree shaking. There's a lot of stuff that's going on. And we know that the industry is sort of, it's moving at what seems to be an ever accelerating speed.
01:11:06
Speaker
towards whatever, you know, it's like, I remember listening to someone before about velocity is that velocity is fine. But the problem is you might be going fast in a completely wrong direction. And it feels like we're always like going off in completely wild directions and eventually kind of recalibrating rethinking. But what leaders need is leaders need kind of actionable strategies. They need something which says, Hmm, okay, if this isn't true, or that isn't true, or I'm not feeling this, or I'm not feeling that,
01:11:36
Speaker
I need ideas. I need something I can pick up off the shelf. I need something that I can pick up from a website and say, this is something which makes sense. It's got a rationale and actually I can action on it as well. Yes. And that's what it feels like you're trying to put on the table there. Well, that's what atomic architecture is. That's why it's got that alliterative acronym and it's kind of got a well, you know, it could have a website and you can imagine we've got some kind of
01:12:05
Speaker
ideas for some sort of like the atomic age kind of theme and things. We've mocked up a few websites on it and things. But yeah, things have to be packaged in a way that a leader can kind of
01:12:18
Speaker
read in a blog or hear in a podcast or get in a book. And unfortunately, while I'm at the stage of really socializing the idea, because I just want to get as much feedback and think about this for a while and actually try it out on different, you know, actually look for examples of atomic-like architectures. Because of course these things exist. It's just you've got to go along and you've got to give the name to the thing. You know, that's
01:12:47
Speaker
Yeah, I think your emphasis on standards is a good thing as well. Just one more thing, Vijay. It's because you were talking, Vijay, about identity services and stuff like this. But there are internet standards for this. There are rules about how to produce tokens, and there are standards for producing tokens and standards for validating them and verifying them and all these kind of things. So you might end up having some services running, but that service
01:13:17
Speaker
should be from, and there are plenty of vendors that do this, you know, so there's no reason that you have to start everything from scratch. Yeah. You know, that's the nice thing about standards, isn't it, is that you can pick a standard and then you can find an implementation. And hopefully, yeah, I mean, I think that the reason why you can't, I've sort of waited for a while for, you can't really build atomic architecture kind of 10 years ago, because there
01:13:47
Speaker
So one of the principles of atomic architecture is that you know who does each operation. When you call an operation like request loan, every API call is called with a bearer token. Now that bearer token is a standard thing on the internet. It's part of the OAuth2 family of specifications. And this whole acquisition of access tokens requires vendor software. And those things are available now, but they weren't 10 years ago. But now I think we've almost
01:14:16
Speaker
arrived at a very, we've finally got agreement in an area where we very rarely get agreement, and that is how computers should talk to each other. And we've got an agreement on, you know, that it's, you know, the HTTP API is the sort of the mainstream. I mean, of course, there are other alternatives, but, you know, to have one thing that is like the main stream, and then you've got this ability to allow applications to call APIs. I mean,
01:14:44
Speaker
they are application programming interfaces after all. You want applications to do it on behalf of users that are engaging in the business processes. We haven't really been able to do that without users getting their applications, their passwords, and that's obviously not a sustainable thing. It's only recently we've been able to do federated or delegated
01:15:09
Speaker
access from where applications can access data on behalf of users. All of these layers have had to be built before we can finally get to a point where we can build systems where we know what's going on or who's doing things. That's been a key component of atomic architecture. I think that's a big failure of
01:15:36
Speaker
microservices is simply the security issues. It's so hard to make distributed systems secure because there's so many different surfaces that you have to protect. It's very, very hard, especially we've got small independent teams and they're all building little services. It's a massive deal to ask them to make them secure as well and all the other things that they have to attend to. So it's a
01:16:03
Speaker
It's a recipe for disaster because you're giving small teams a very small amount of time to do a vast amount of requirements that you must do in order to create secure services. That doesn't sound like a good management tactic.
01:16:20
Speaker
But having these ideas, as Ray, you were talking about, having the leadership, leaders getting the information from somewhere. I think the information, what I feel is that these days, or maybe even from the beginning, is that regardless of what kind of technology it is, what kind of architecture it is, what kind of new idea it is,
01:16:41
Speaker
It is always about, this is how awesome it is, but nobody gives the context. In this context, this fits well. What I would love to see, or at least for me, as a person who can't have all the knowledge in the world, obviously, so I need to depend on
01:17:00
Speaker
on the people who are creating amazing software and telling me that look this good for this but if you don't have these problems don't use this thing you know it'll be nice to see that kind of way because that's better coming from the creators rather than some people trying to pushing this square peg to round hole or whatever and then deciding well it worked for us you know and
01:17:21
Speaker
use this for everything like Kubernetes, use this for everything or serverless cloud native architecture, everything. I think that it will be much nicer and transparent if we say, look, this is the architecture or this is the technology, this is the database, this is the programming language, whatever the thing that we're building. And when it is being built, it was built with some specific thing in mind. And then because everything has
01:17:49
Speaker
trade-offs, right? Everything has pros and cons. And the reason why microservices became a big swing in the industry, because everybody thought that is the way to build. And same with React, same with all this shit. Again, we are falling into the same grand narrative of, oh, this is the way that things should be built again.
01:18:09
Speaker
it will be nice to have one section explaining this is the problems and this is where we came to this conclusion. This might not be the solution for this one. Yeah, I agree and I disagree. I mean, I agree that you need to know what you're optimizing. What are you trying to optimize? I'm trying to optimize.
01:18:30
Speaker
data security, but the handling of data as if they're precious records to your business and the numbers need to add up. So, sort of high care of data. And I am definitely trying to optimize performance or, you know, vast scale.
01:18:51
Speaker
But then there is this kind of desire that you get this painting by numbers guy, just do this, this and this and this and just implement this. But in fact, you know, building systems, you know, is a difficult art and it requires intuition. And so you have to say, well, somebody has to make a decision, they're going to
01:19:16
Speaker
take some ideas and they're gonna have to make a decision how to apply those ideas in the context and I haven't you know I can't give you a whole set of industries or atomic architecture you know one day hopefully that that will be something I can point to that all these but that actually is probably more likely that those projects will be obsolete by the time that
01:19:39
Speaker
you come to build something. I mean, the problem is that things are always moving ahead. And so you really, every system is different. And you don't really get much opportunity to just copy and paste a system and say, well, this works over here. It's going to work for my domain. A lot of it is
01:19:58
Speaker
to a bit of trial and error. Yeah, I mean, I totally understand that. But what I'm coming at is maybe not like, oh, this is not good for something, but at least giving the context. I'm trying to look up the, there was some tool, I think it's probably built by either Airbus or one of these companies. It's a competitor to Kafka or Kafka-like product.
01:20:21
Speaker
But I was very impressed by their read me and documentation that they specifically say, hey, these are the constraints. These are the trade offs that we made in designing the system. And it is built in this way. It might work for you. It might not work for you. We don't know.
01:20:38
Speaker
It clearly explains the constraints on the context in which the solution was built. And that's what I miss in most of these technologies or languages or everything, because I would like to see what is the context to bring this idea into fruition to first version of first beta, whatever the version that you're releasing. I think that would be very helpful for me to understand, do I have the same problems and will I reach the same conditions?
01:21:07
Speaker
conclusion, then there seems to be a nice synergy in terms of the solution to use the buzzword, managerial word. Isn't it kind of like saying, where is the pain? Yeah. Because one of the things I think we're very good at, by the way, is pretending we're not suffering.
01:21:30
Speaker
And I think, you know, it was the Stockholm Syndrome abounds, you know, sort of, we're high on copium. I mean, you know, I like this sort of joke that, you know, whenever I see these kind of motivational speeches or whatever, it's just like copium-tastic, you know, copium is the new, you know, copium for the people.
01:21:47
Speaker
I think that's part of the problem. First, you're doing some humanities. The first step towards change is admitting there's a problem. That's probably what the motivation is, Malcolm, isn't it? It's like, okay, look guys, there's a bunch of things going wrong here.
01:22:08
Speaker
Yeah, well, you know, I guess what you're saying is there should be a list of things and hopefully there's enough people sort of essentially lights are coming on saying, oh, yeah, oh, yeah, I see it. I see it. It's for me. Yeah, I get it. You don't see it on the other way around. Like, no, I don't have this problems. I'm not going to take this medicine. Yeah, exactly. People should be able to say that. Oh, yeah, for sure. I guess what I'm trying to say to people is that I clearly think that
01:22:33
Speaker
the complexity or the majority of complexity in these systems is caused by the distribution of state. Now, a lot of people reject that. In fact, shared databases in microservices is seen by some people as something of an anti-pattern. That's completely fine. I'm making this assertion that, in my view, and I've had a distributed systems background and I was a core programmer, and I've been building application services. I have
01:23:02
Speaker
I've really tried to make distributed, I used to write a distributed transaction broker once and XA, transact, two-phase commit, all that stuff. I've tried hard to make this, and I just think that it's just too difficult. Therefore, that's my position. You absolutely have to, and you've got an exceptional situation. You should try, at least in the first instance, to build off a single state store which
01:23:31
Speaker
Yeah and then but i know the problems that we had in the 90s and 80s of everybody hammering on the same
Final Thoughts on Atomic Architecture
01:23:38
Speaker
database and tables and it's you know i call it a student kitchen right when everybody shared space it's gonna be tragedy of the commons so a lot of atomic architecture.
01:23:52
Speaker
after the shared state principle is actually mitigations to how to prevent the thing turning into a student kitchen and that's the like the so the um the abstraction of operation so it's not like oh you get to insert into tables randomly you don't get to know what the tables are you you get to define the operations and the operation is an important unit of currency really because it's it's at that that you attach the access control and say well
01:24:18
Speaker
when you define what it means to request alone that's the point at which you said well who is allowed to request alone so that that almost drives the conversation about what your personas are your roles are what you're going to allow people to do and so that that is the common language and that's very much borrowed from the domain driven development design people.
01:24:39
Speaker
And then after you've got that, well, then you see, well, if you've got all these operations, you just can't let every free-for-all, you can't let everybody request loans and things like that. So then you're building another level of maturity saying, well, we've got to restrict. Who can call what? And how do we do that unless we know who it is that's calling the operation? And so it goes on and on and on. And you finally realize that actually you need these operations applied one after the other if you want to preserve consistency.
01:25:09
Speaker
you know and then because you apply them one after the other you then have the opportunity to create an event log which is a sort of a bit of an optional principle but the idea of having this ledger really is a ledger set or a log but it's a maybe a public log of saying this is exactly what happened you can give it to an auditor and say trace through these steps and you'll arrive at exactly the same
01:25:35
Speaker
state as we have, you know, if you want to asking questions of the past and then of course when you're asking questions of the past you want to be able to recreate the whole state when you're trying to work out why was this loan approved two years ago, you know, in order to answer those questions sometimes you need to be able to roll back the database to that particular date in order to
01:25:57
Speaker
answer certain questions so that's where having by temporality you know which is a little bit of a hot tip to you know xddb and a bit of a marketing sort of yeah that's what i mean i was trying to you know put a question to you know to do.
01:26:13
Speaker
I don't know, contrived question to push you there. Because by design, as you're explaining the student kitchen situation, like, you know, many readers and then, you know, having this contention of resources. Because when I see the design that you made, like there is this operational setting, there is analytic side, you know, I was on the both sides and mostly on the analytic side in the past, you know,
01:26:33
Speaker
five, six years where we had to build this reporting and everything. And you're always, if you're using, quote unquote, traditional databases here, you're kind of fucked because I cannot keep quieting the production database to generate these, you know, look at reports or whatever. So obviously the then I have the, you know, read replicas and then everything that comes with all the issues with that one and CDC, all that nonsense comes in. So so.
01:26:59
Speaker
To solve this problem, to have a shared state, you need to have a certain technology in the middle, certain type of database in the middle, that is going to help with this, right? As you mentioned, like XDDB probably. Yes, although I think you can create it in a different
01:27:15
Speaker
tools and products. Yeah, but it's gonna be more work. Yeah, it's gonna be more work. I think at a certain level, you're really just saying, these are desirable qualities of the system. And this is how you go about kind of, and then there are a lot
01:27:31
Speaker
in your in any given context there might be quite a lot of work to do to you know integrate so you might want to search engine in there elastic search or something so you you just but it's a lot of these things are just ways of thinking and they're like mental
01:27:47
Speaker
And these principles allow you to at any given decision point, you can say we're going to go left rather than right. But that's really not for I'm not trying with atomic architecture to lay down a blueprint of how to build a system. I'm really just giving people a kind of map of how to make decisions and that is trying to get the benefits of shared state without all of the
01:28:11
Speaker
the obvious downsides, although I'm sure there are ones that I've not experienced which are still working in there. But the trade-offs is having shared state is well worth, I believe, the pain points, but it depends on your calculus.
Clojure's Advantages and Innovation
01:28:33
Speaker
Maybe we should talk a little bit about closure because it's closure podcast, I think. We went from web to a little bit of humanities and then ranting about everything and then going back to architectural things.
01:28:49
Speaker
you your company and you're still working in closure right i mean there are lots of plenty of closure things and and so again based on i don't know it's been 10 years or i don't know how many years you've been yeah 10 and 10 and a half yeah yeah yeah so so where do you see now i mean where do you see yourself now because are you still fully on closure or do you think
01:29:13
Speaker
Well, it's time to rethink and then, you know, create a new browser and go to the next one. Or is there anything on your radar in the Closure World and then outside? I think the Closure World is still a source of great, you know, we're going to talk about Squint and Skittle, or, you know, is this great? Yeah. And within Closure, kind of the unfair advantages of having a language where, you know, in most other languages, I've found that you end up
01:29:42
Speaker
with so many things that you're fighting against, you don't really make forward progress. So, you know, I find closure is a fantastic research language. It's also, I think, a secret weapon where kind of a company can use it and can exploit it. It can be a force for
01:29:59
Speaker
you know really innovation and you know some companies find themselves in whether they're really conservative and they've got lots of procedures and policies for you know but they say sometimes closure isn't an option for them and then there are other other companies that are less constrained and they want to use you know leverage closure. I mean we found that
01:30:21
Speaker
Closure has been a good choice for us because people who come to the closure community are often people who have experienced these problems and appreciate the value of closure and therefore they're almost a self-selecting group of individuals. They tend to be the people you'd want to hire in any team because they tend to be
01:30:43
Speaker
have initiative and have self-taught and have, you know, those are exactly the sort of people you want on your team. But for that reason, I think Closius has been a good choice. We use it internally. We use it for our own projects. It's used for XDB.
01:30:57
Speaker
you know, the version two, and it's just a great language for building things that are innovative. I mean, it was no surprise when we were on another podcast, Juxtcast, we were talking to Nathan Mars the other day. Yeah.
01:31:13
Speaker
to drop another name. You know, of course, Rama has built enclosure, although you wouldn't know from the marketing, but that, you know, you can see that that choice of language, if you're trying to build something like Rama or like XDB, then that makes a whole load of sense. If you're just trying to build something in a, in an organization and you've got to follow, you know, you're building a microservice or something like that, you know, like I don't know, you know, I think
01:31:44
Speaker
The thinking matters more than the language. The language definitely helps, but I'm more interested in the experience I've had using Closure and I'm seeing in the small that functions and having state in one place, like you're writing reframe applications. We've built some really fantastic frontends and reframes, which I just don't think we would have been able to build in JavaScript because you get really quickly into JavaScript where your early choice of
01:32:12
Speaker
libraries and things that you've made are absolutely critical, whereas going to reframe is such a safe choice. We've been creating some amazing things, but I've known that reframe is fundamentally an atomic architecture. It's got this kind of atomic state in the middle, and then all the functions that fly around it and apply changes, and it does work. And so I feel that these key ideas of separating
01:32:39
Speaker
state and functions and closure. It's just been like really are these kind of cornerstones to why it's such a nice environment. To me, I mean, to echo that, I slightly disagree with you as well, because why not? That's still our podcast. Yeah, exactly. Screw you, Malcolm. Come on. No, but I think language just matter. We're not on text cast.
01:33:07
Speaker
Yeah, they're haters. Actually, I've been on Juk's cast, so they've got all the names. You're just, you know, promiscuous with podcasts now. We've had all the big names. And then they had to resort to you, finally. They checked out everyone.
01:33:34
Speaker
Anyway, anyway, anyway, the reason why I think closure is important. I think the reason why people end up landing there is because you know, it's actually hard to do what you well, it's always possible. It's a true in complete languages, etc, blah, blah, blah. You can do what you want with these languages. But actually, if you want, for example, to have a mutability, you can do it in Java. But it's not easy.
01:33:58
Speaker
Easy you know if you want to do it in javascript you can do it you can use you can use ramda or you know a whole bunch of like functional libraries. But it's not easy and it's not idiomatic and the other people in that community aren't gonna like it they're gonna be they're gonna be kind of. Not exactly offended or annoyed but they're gonna be surprised at the very least.
01:34:25
Speaker
that you're doing things in a certain way that is counter-cultural, so you can achieve these things. Well, what I was really trying to say fundamentally cut into the chase, I know you know the answer anyway, which is that doing immutability in JavaScript and other languages, it's not as easy as it is. It's feasible, but it's not the default, it's not idiomatic.
01:34:50
Speaker
So I think languages do matter. Yeah, closure is not the sum of the parts. It's some sort of magical kind of. So, you know, it's definitely possible to sort of achieve the same kind of Zen by just doing Python or Rust or something in a closure style. I tried that. And so
01:35:14
Speaker
Yeah, it is a shame that, like, we were talking about the build steps and things, it is a shame that it's not that Brendan Eich wasn't, you know, Richard Hickey wasn't at Netscape. The thing that always strikes me though, I was just going to say quickly was that, like,
01:35:35
Speaker
A lot of language people now kind of know this about immutability. They know that it's a problem. And Rust kind of acknowledges it, sort of. And there's a lot of people in the language community know that immutability is a good thing to promote. And yet it's not
01:35:57
Speaker
common. And I know that there's various people asking questions about why not. Why isn't functional or immutability taken over the world? And it feels like a lot of the algorithms and these kind of things are still written with that kind of low-level optimizations in mind.
State Management and Database-Centric Approaches
01:36:20
Speaker
And it sort of bleeds out into what we call the real world. You know, that people are always trying to essentially say, oh, you know, it's all great with your immutability, but in the end, the real world, in the real business, we have to change things. And, you know, you just can't, it's just too hard with a functional programming language. What's your counter argument to that?
01:36:44
Speaker
if you have mutable state in any program, like at scale when you're passing references to, you know, and then it trips you up and if you're trying, you end up debugging stuff and you spend a lot of time in debugger and it's just a thing like garbage collection that really ought to be a standard feature. I don't know what the state of
01:37:13
Speaker
disabilities in python because i'm not really deep into python but i guess you know you've got date classes and same problem you have in java we've got you know these foot guns lying everywhere and therefore you have to just
01:37:28
Speaker
just it's so easy to screw everything up and making mistakes and programming and colleagues spend ages and late at night and trying to debug something. It's just there's this feeling I get when I'm working on projects that aren't using Clojure. There's a lot of time that I spend trying to figure out and debug something. It's really an issue that shouldn't exist. But it's difficult to kind of explain that really. It's that blood paradox
01:37:54
Speaker
Yeah, yeah, you can't. It's definitely it's my closure will always be my kind of go to language for trying things out and building things at any kind of scale, because it's just safe. It's just safer to do so. That said, I do have an answer. I think really the mutability that that state should be in the database, right? It should be persisted, not just persisted. It should be persisted. And for the most part, we should just not put too much state
01:38:24
Speaker
In our program so with cloud and you know the femoral services going shut down and start up on different notes and things like the cold kubernetes. Name which would mean that people are less writing state inside there.
01:38:40
Speaker
their programs anyway and there would be more stateless program. So I presume that is a trend but like I've just learned last 10 years that you know databases are probably going to come back because they do have some solutions to like maybe that you know maybe this microservices phase is going to like the furthest we'll ever
01:39:04
Speaker
go away from the solar system of databases which sort of coming back and realizing why databases were good and why there's all this new energy in databases and I think databases are the you should start with the database and grow out and it's unfortunate that that database has got a bad press because most databases were sort of
01:39:27
Speaker
built in the 80s and they're crap. But you think it's like the problem that we were talking about before with the browsers and everything that databases got to a certain point where they answered a lot of problems and then the web came along and then didn't answer the problems anymore. They weren't able to be web scale.
01:39:50
Speaker
I'm going to stop at that one. But the whole idea of MongoDB and the whole idea of no SQL and all these kind of things were kind of responses to the ossification of Oracle, essentially. I mean, the business practice of Oracle weren't very nice, but there were options with Postgres and stuff like that and other ones like
01:40:17
Speaker
I used to work for a company called Ingress back in the 80s actually, in the 90s. And there were various other sort of competitors in that space. But obviously, Oracle was the big dog. But it kind of lost ground rapidly in the internet age. And it feels like the internet as such was kind of responsible for this dispersal of state. I'd put it there. I mean, I don't know what you feel about that.
01:40:47
Speaker
where I was founded on this kind of file system concept rather than the database concept. And so there's always been a bit of a impedance mismatch. I don't think fundamentally that file systems are actually a good database for web resources. I think databases are better for web resources. And that's true, of course. Something I've been trying to push quite heavily that
01:41:14
Speaker
and resources need metadata and kind of embedding metadata in file names is a bad idea. But I think it was just two different communities really and it's unfortunate that there hasn't been more kind of web servers that have been built around databases and things tried in that area. But I think there's definitely there's lots of research happening in the database world distributed databases and
01:41:41
Speaker
No SQL and SQL, you know, new SQL. I think we haven't really, you know, we haven't seen the end of that. I think we're seeing it kind of.
01:41:52
Speaker
people will modernize the database. There is definitely a problem with kind of everything in databases being tables. And so there are kind of various approaches. And the new SQL standards have got much better support for trees or document more like JSON documents. And there's much more support for JSON. So we'll see. But it's a good area of research. And I think things will improve.
01:42:23
Speaker
But yeah, I agree with you that I think the conceptually databases are the right, we need some of the properties that databases have, like transactional consistency, which we've traded off far too prematurely, but it would be much nicer if they were much more modern. And I don't like the
01:42:48
Speaker
the old, you know, store procedures and the SQL languages. And, you know, it's a thing. I mean, SQL is everywhere, but it's a bit is it's when we used to write thousands of pages of SQL and it's all very much and doesn't compose very well. It's not very scalable language, but there are much more languages out there. But hopefully something will come along. But the other thing that I was going to say is that you mentioned there that like the file system
01:43:16
Speaker
And it's one of the things that's a bug bear for me as well is that file systems, they're kind of like a ball and chain for us.
Future Trends in Web Architecture
01:43:24
Speaker
I feel, you know, they're Microsoft tried to kill the Unix file system, you know, with its, its version of drives and stuff like this didn't really work. Um, so the file systems there, everything is basically around the Unix notion of a file system still, but, but that's still a, it's a kind of really weird abstraction, I think.
01:43:45
Speaker
And it's a very binding abstraction. It's a very tight abstraction to do with this hierarchy of things that is very much a directed, ethical graph. But in fact, like you were saying with links or hypertext, that's not the way the real world is. You don't want to be traveling all the way through to get to a particular file.
01:44:10
Speaker
You want just to have essentially a content hash. You want to have a content-based addressing rather than some sort of path-based addressing. And you can do that kind of thing in databases with all kinds of content, which is one of the bugbears I've got about the way that the web is and the way that a lot of people treat content these days. They treat it as if it's a file.
01:44:39
Speaker
which is completely like it's a completely constraining, a very confining abstraction. In fact, it's not. And I don't find it a very good abstraction because it's very much based around like the necessities of spinning disks. Yeah. Yeah. It forces one organization on you, but it really is. Well, yeah, it's like the URL.
01:45:07
Speaker
I think it's better as you know, it's a virtual point or identity. You know, the web resource is a better idea than a file, you know, it is definitely file system plus plus, you know, your idea of a web resource, you don't, you know, existing on some sort of thing in the DNS space, and you, you can
01:45:30
Speaker
navigate to it and it can be on it, you know, you can have, you can cross file system boundaries and post boundaries. And yeah, I mean, it's, and it can, yeah, content negotiation, all that stuff. I think the web, you know, the web is a great kind of improvement to the file system. But unfortunately, most web servers, that static stuff is all kind of still assuming that it's a file system, what you still has that model.
01:45:56
Speaker
I think it would be much, much better if the web was built on databases. Nobody seems to be doing that, but that for me is the natural progression of web
01:46:09
Speaker
websites is that they're kind of much more, I mean, none of the CMSs and stuff, but like having all data records that are web resources, you know, everything, all records in an organization should be ultimately a web resource. Because, you know, that was the real innovation of the web for me was I just have a URL clicker and share it with somebody or you could, you know, that will be my fancy. And you go to places today and there's still lots of
01:46:36
Speaker
pieces of information that you can't access without going through a particular path, through an application, a bunch of menus, and really information is much better organized if it has identity and can be referenced.
01:46:59
Speaker
It's a good, it's a good place to, I think, round it up because we're almost reaching like two hours. I don't want to keep clicking on another link and then going into another topic. So thanks a lot, Malcolm, for joining us again. I think, um, you know, it's a, it's always, you know, so many things I learned talking to you, you know, people like you as well, like having so much of experience and.
01:47:25
Speaker
It's really fun to see what you're thinking and what new ideas or new ways of thinking about the same problem and then solving it in a different way. And yeah, we really appreciate that. And yeah, I'm going to link to your atomic architecture post. Is there anything else that you want our couple of listeners to know about this one? I'm pretty sure we are at least the second podcast after Jextcast.
01:47:53
Speaker
in terms of audiences. Yeah, I know there's a webinar link somewhere about the atomic architecture, but I will link that from the blog post. So if you link it to the post, that'd be great. Perfect. Yeah. Super. Then thanks a lot. Thanks a lot for taking the time. And I hope we'll meet again in one of the closure events, I hope.
01:48:18
Speaker
Yeah, great to be on. Again, I really enjoyed it. Thanks, Ray. Thanks, DJ. Thanks a lot, Malcolm. Thank you. And thanks a lot, folks. And wherever you are, stay safe. Stay warm or cold, depending on whatever the hemisphere you're on. That's me assuming that there are people who are listening to this in different hemispheres, which is...
01:48:37
Speaker
bit of a stretch but well there are only two hemisphere so there is a chance that one is there one is here so good luck and we'll see you next time hopefully this year maybe next year who knows you know in the in episode number whatever you know x y
01:48:54
Speaker
The outro's a professional as well, Malcolm. Yeah, he's working on that one as well. He's doing a great job. Yeah, exactly. We're halfway there, you know? It's very much, very much. This is how much polish I have left in my box. Thanks, everyone. See you next time. Bye.
01:49:13
Speaker
Thank you for listening to this episode of DeafN and the awesome vegetarian music on the track is Melon Hamburger by Pizzeri and the show's audio is mixed by Wouter Dalert. I'm pretty sure I butchered his name. Maybe you should insert your own name here Dalert.
01:49:31
Speaker
If you'd like to support us, please do check out our Patreon page and you can show your appreciation to all the hard work or the lack of hard work that we're doing. And you can also catch up with either Ray with me for some unexplainable reason you want to interact with us, then do check us out on Slack, Closureion Slack or Closureverse or on Zulip or just at us at Deafened Podcast on Twitter. Enjoy your day.
01:50:00
Speaker
and see you in the next episode.
01:50:36
Speaker
You know what Malcolm was saying? You couldn't unpick the organizational structure from the end product. I think you probably could here. It's like, oh yeah, we know this one's a deafened.