Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Ep. 7: Verbs vacation (part 1?)
247 Plays1 year ago
And now for something completely different…
In this episode, Chris and Daniel sit down to talk about a new event sourcing package they're working on called Verbs.
Recommended
Transcript
Introduction to 'Overengineered' Podcast
00:00:06
Speaker
All right, welcome back to another episode of Overengineered, the podcast where we ask the very important question, what's the absolute best way to do something that doesn't matter a whole lot? Although today is a very special episode.
Special Episode with Daniel Colburn
00:00:21
Speaker
I am here with my friend Daniel Colburn. And say hi, Daniel. Sorry. Hello.
00:00:28
Speaker
And we've actually rented an Airbnb and we are locking ourselves in a room for a few days and trying to knock out a event sourcing package.
00:00:38
Speaker
Yes. Hello. And hopefully, you know, I would say we're trying to answer the question. What is the absolute best way to solve a problem that that does actually matter quite a bit. Take that Aaron Francis. Yeah. How are you? I am good. So yeah.
Design Challenges and Airbnb Retreat
00:00:54
Speaker
So we're in Baltimore. I was in Virginia for something. So I came up from Virginia. You came down from Philly. We're just
00:01:01
Speaker
We're in Baltimore. Yeah. I have actually, I live in Philly. I've lived in Philly for a long time and I have never been to Baltimore before. So it was kind of fun. You get to see, uh, it's like, it's Philly like, but smaller. Yeah. Yeah. Yeah. It's got Philly vibes. I like it. Um, we were totally unprepared. Yes. So yeah. So the plan, let me, let me just table set a little do it. So the plan is we are, okay. So verbs is this event sourcing package that,
00:01:30
Speaker
We've been talking about and writing code on and talking about again and writing documentation for and planning and building for a long time.
00:01:45
Speaker
You know, we keep sort of, there's like a big chunk of design that needs to be done that is blocking. API design. Yeah, API design. System design. Yeah. So there's a big chunk of design that needs to be done that is blocking, finishing it, and
Balancing Business and Collaboration
00:02:08
Speaker
shipping it.
00:02:08
Speaker
And so you're busy. You run a business. I run a business. And we have a hard time finding time to work on things together. And we were like, hey, wouldn't it be fun if sometime in September we went to Baltimore and finished this thing? So that's what we're doing. So we've got an Airbnb. We went to Safeway. We got a bunch of snacks. We've got a big roll of paper from Staples and Post-it notes.
00:02:38
Speaker
Pretty excited. Yeah, I mean, and even I think like going back a while further, right, like Daniel used to work at InterNACHI. Right. And at InterNACHI, we implemented together my first like production event sourcing project. Also mine. And we used the fantastic Spotsy package, which InterNACHI still uses to this day.
00:03:03
Speaker
But I think the two of us both just have a different set of preferences around. Well, it's funny. I was in Philly with my family a couple of years ago. Remember this when you rode your bike over and we grabbed coffee? And basically the only conversation we had while I was in Philly was like, man, we should make an event sourcing package. Right. If only we could change these 400 different things. Yeah.
00:03:31
Speaker
Yeah. Um, and so, you know, we've, we implemented event sourcing at InterNACHI.
Rethinking Event Sourcing for Laravel
00:03:37
Speaker
You have been implementing event sourcing a ton at Bonk. And, um, yeah, it's always felt like there's an opportunity to approach event sourcing that feels maybe a little bit more, uh, welcoming, welcoming, a little more artisanal to use the Laravel term. Right. And like.
00:03:57
Speaker
I think also just has different preferences that you and I maybe share around how we're going to do it. Yeah. I mean, I think they're, um, I think event sourcing traditionally has certain values and they are inherited from the world in which it was cooked up. Right. Um, so there's a lot of like DDD language and DDD, uh, conventions that are and wording even that, uh, is
00:04:26
Speaker
Sort of the default event sourcing stuff right and for a lot of our people Which is like Laravel?
Critique of Traditional Event Sourcing
00:04:38
Speaker
artisans That stuff may not like I don't know that that's an advantage for them sure right so well and it introduces a bunch of barriers right because
00:04:51
Speaker
If you don't know what an aggregate route is. Right. Yeah. Right. And this is an argument that we'll probably have as we're talking about this. But some of these boundaries that event sourcing has are there for a reason. But they also can feel really exhausting. Every time you need to add a new feature, you're like,
00:05:11
Speaker
updating your aggregate route, and you're adding a new event, and you're updating your projector, and you're having to touch these 15 different places to add one new feature. And it's because the boundaries of event sourcing are clearly defined, and you get a lot of benefit from that. But I think one of my main points coming into this is
00:05:35
Speaker
You can still have those boundaries and get that value while having an API that doesn't feel like so much busy work every time you need to do something new. Exactly. Yeah.
00:05:50
Speaker
And, you know, that's my, like, I love Voltfolio stuff. I have always, this is something actually you and I have disagreed on in the past, for example. I hate form requests and you like them. But, like, I am a big fan of, like, eliminate files. Eliminate files from features where, like, okay, when I create a new feature, I don't want to have to update
00:06:16
Speaker
the routes file, a form request, a controller, maybe a model. That has always felt bad to me, so I've always been very like, how few files can I have touching this feature? And with event sourcing, it's kind of the same thing right now in the implementations that we're both working on. It's like,
00:06:39
Speaker
There sure are a lot of files involved in doing this very simple thing. So that is the thing that we're addressing.
Key Principles for New Event Sourcing Package
00:06:46
Speaker
Yeah. And I mean, this is, this is like my current side tangent. Like I think that the composer auto loading and PSR standards, like one class per file have actually done like a huge disservice to design because
00:07:03
Speaker
I don't think that I agree that like having a new up a new form request file when you're like validation gets a little bit gnarly and you want like a place to put it or you need to like yeah like I like former quest because you know nine times out of ten you end up like
00:07:19
Speaker
Oh, I want a little extra protected function so that I can clean up this code. Or I want to just memoize something so that I'm not refetching it from the request and rehydrating it. All those little things. And I think, for me, the trade-off there is the new file is worth it. But my
00:07:40
Speaker
significant preference would be to just have that class defined below the controller because they are a hundred percent coupled. Yeah. Why does it have to be a new file? Rust is the conventions in Rust are really good. Um, but like, you know, Rust is one of those things where like, if you like install a library, like there is like an entry file that has like all of your classes to find basically, you know, which rules.
00:08:04
Speaker
Yeah. I mean, and that's nice. That's a nice thing about like working with JavaScript, like that trade off. People have found a different place, right? It's still, it's not like everything is one in one big file, but I think you see a lots, especially reactly JSX. You'll see maybe three or four.
00:08:22
Speaker
components that are basically one component but like for design reasons you want to want a couple of functions yeah yeah and you just like put that all in one file and then like all that's just in one place and I would love to see you know an opportunity to do that in PHP but I agree that it's like
00:08:43
Speaker
It just feels bad, especially when it's just like, I just want to do this one thing. I don't need like three new files. Yeah, for sure. Okay. So let's start with the list of like, okay. So let's talk about how, where we are with verbs and then like what's left.
00:09:04
Speaker
What we need to get done because like okay, so for the listener for the user This is the first so we I picked Chris up at the station at the train station went to Safeway and got snacks We went and got lunch. We've just got into the Airbnb and like plugged all our stuff in
00:09:20
Speaker
So we are this is our first like work work conversation of the trip. Yeah So we're we are setting the stage of like what are we gonna get done in the next three days? Basically, do you want me to just like read out the notes that I have from? Oh, yeah, you've been writing notes I've been driving for the last show because so you've been making notes All right. So this is what I have right now and I think some of the words here we don't necessarily agree on but I think what is this list that you're about to say I
00:09:45
Speaker
These are some core principles, and then I have the broad strokes of the life cycle. I have some notes about the major components of the system, and then a few questions. Should I just go down that list to start from there?
00:10:02
Speaker
Okay, so let's start with core principles. Okay, so like minimal overhead. Yes, right. So it should be possible to add new features with as little foil early as possible as possible. Yeah, I mean, not necessarily. I think my focus there is definitely work is loaded term. Yeah, boilerplate, right? It's like,
00:10:22
Speaker
I want you to write just the feature code and the package handles everything else. Incremental adoption, I think this is a big one for both of us for different reasons. It should be A, possible to migrate from other event sourcing packages incrementally.
00:10:42
Speaker
And it should be possible to incrementally add verbs to an existing project. It's not like you have to shift your whole mindset to event sourcing. It's like you can just implement some features. And I think part of that is like,
00:11:01
Speaker
It needs to be that someone who is not doing event sourcing every day can jump into an event-sourced feature and kind of know where they are pretty quickly. Because that's my biggest challenge with the event-sourced features at InternetG is when I have to go back over there, it takes a little while to get back into the, oh, right, I need to think about things this way.
00:11:28
Speaker
we want to design the API so that it really gently guides you in the way that doing what feels right also is doing what is right. So there's another principle I'd like to propose, which is basically in the same way that it's like you can incrementally add it for a single feature, I think it should be relatively easy to pull the ripcord and
00:11:57
Speaker
unevent source of feature. Oh, I like that. Yeah. Um, so I used to write in my freelance contracts. I used to have a section called how to fire me. Um, that basically it was just like, here's like our kill fee and like all this other stuff. Right. But, um,
00:12:16
Speaker
I like it. I like putting that on Front Street where you're like, look, this might not be for you. Here's what it looks like if it's not like before you make a decision about getting involved with this. Here's how much pain you can expect if you decide you change your mind. I love that. And I I love like I have things that in my mind, I'm like, OK, V3, V4, V5, maybe. And I would love to have like a verbs eject.
00:12:45
Speaker
if that was like possible right it's just like it's totally possible you get rid of all the event stuff and you just you know you take the state that you're at right now it shouldn't be that hard nope that'd be so cool yeah okay so um that would be sick
00:13:01
Speaker
i just added that easy to remove the next one this was a word that we went back and forth on this is what i have right now is hard boundaries right so like despite really really good dx right we want the developer experience uh to be like always perfect right i think that's that's like a maybe that should be
00:13:23
Speaker
that should probably be in here is just like, we want to do whatever gnarly BS it takes to make consuming verbs glorious, right? We are the sin eaters. Right. Exactly. Like if there's shit to be eaten, we will eat it on your behalf. Yes. And so like for me, the hard boundaries is despite that,
00:13:47
Speaker
I want to ensure that the APIs that we land on still sort of respect the event sourcing boundaries that either are good. Yeah. Are either true to event sourcing or maybe are just true to the way that we think about event sourcing. But like, you know, to people who aren't super familiar with event sourcing, it's like, there's kind of your event world and then there's like your application world and
00:14:14
Speaker
the harder of a boundary that you have between those two, the better a time you're going to have. And so that, I think, is the reason for a lot of the design decisions that we're trying to move away from. That's why there's all these different files and there's these strict processes, because they're trying to really enforce those boundaries. And I think that that principle is good.
00:14:38
Speaker
And I think that there's a way to meet that principle with a different approach to the API design. Well, and you and I have both made design mistakes while implementing event source systems. Yes. And when you make them are kind of painful, you can get kind of far before you realize you made a mistake. Yeah, and it can be hard to get out of it.
00:15:07
Speaker
it can be a little hard to get out of it. And so as much as possible, just putting some guardrails, obviously all guardrails should be jumpable, but putting some guardrails in place that encourage users to do the things that we know are best for them. Right, right. Yeah, we've learned some hard lessons, and let's try to make it harder for other people to learn those lessons. Yeah, exactly. Yeah.
00:15:36
Speaker
And that comes to the next bullet point is great conventions, right? So easy to understand conventions with great escape hatches. And that second piece is really important, right? We want to make the canonical way to do something really nice, but then if you want to do it differently, and I think that this is something that I love about Laravel in general,
00:16:00
Speaker
Like every time I've had to like do something that's not quite the way that Laravel wants it to go. There's, there's something there. There's like an empty function that's just waiting for me to implement. Or there's like, you know, just another way to do it that allows you to do it two different, different times, even though the main API like is only wanting you to do it once or what, you know, whatever it might be.
00:16:28
Speaker
And I think that that's something that we want the final API to feel like that.
00:16:34
Speaker
Yep. Yeah. I agree. So we, um, you know, I think one of the principles that we've kind of talked about is like optional magic, right? Yep. Like there's lots of places where it's like, yo, this is going to be really awesome and like really cool. And I think a lot of people would say this is like too magical or, uh, this is non-obvious or whatever, you know? And like all of those things, like I know I want them. Right.
00:17:03
Speaker
but they all need to be optional. You can actually do a very verbose, lots of different files, event-sourcing implementation using verbs, but you also can do what we're proposing, which is like a
00:17:22
Speaker
one file, like, very cool, lightweight implementation. Yeah, theoretically, you would be able to do something very similar to the other major like PHP event sourcing packages that are out there. If that's your thing, right?
00:17:41
Speaker
Or if there are certain features where the trade-offs are worth it, you can implement those using the more verbose, multiple files, aggregates and projectors and all that. You can still do all that stuff if it makes sense, but for the vast majority of features where you don't need it, you don't have to.
00:18:05
Speaker
I kind of think about this as, you remember when Adam made ZTTP? I don't, but I... Which eventually became the HTTP adapter in Laravel. I remember you talking about this. Look, I get into lots of code bases that still have Guzzle installed and still use Guzzle heavily. And Guzzle does give you a lot of granular control that
00:18:30
Speaker
is like gulls an old package where every possible permutation of how to implement like touch HTTP has been dealt with. Right. Right. I'm sure there's stuff missing from the Laravel HTTP implementation. Right. Right. But like
00:18:48
Speaker
I have not written code that needed guzzle over the Laravel HTTP thing in the last however long it's been since that came out.
Comparison with Existing Solutions
00:18:56
Speaker
And that's kind of how I think about it. I want something that feels like Laravel HTTP, not something that feels like guzzle seven or whatever the hell we're on now.
00:19:06
Speaker
Yeah, I mean, and that gets to the second to last bullet point, which is, doesn't suck to use. Does not suck to use. It feels like it belongs in the Laravel ecosystem. Yes. And maybe that's a little bit of a spicy way to say it, but like, or, you know, I'm, I think that I am just like aware of, I'm, I'm really grateful for the Spotsy package and it has been really good to us. So I, I've written,
00:19:34
Speaker
Hundreds of hours of code, right? Yeah, I if not thousands I think I want to just put a disclaimer out there that like despite me wanting something different This is not meant to be like shitting on the spicy package right sure not like we I came by my Desires honestly
00:19:57
Speaker
You know, like I put in hundreds or thousands. I'm not sure of hours of writing event source code against the Spotsy package and like it went great 90% of the time. Yeah, that package has been incredible for us. It's very solid. Yeah, like it does not usually fail in unexpected ways. Like it's pretty chill. Yeah, I just there's just like I know know what I want. Right, right.
00:20:24
Speaker
And the last the last thing kind of ties into that is like this is an opinionated project, right? Like this is not meant to be like oh You can approach things even though we want to give you the escape hatches so that you can do
00:20:39
Speaker
things that are like outside of the norm, like we're not gonna be entertaining the like, here's a different way that someone might want to do this that doesn't like fit in with the opinions of this project, like okay, that's fine if that's what you want, but like this is probably not for you then, right? Yeah, and if you, to do that, like you could do it, but to do it you'd be re-implementing huge chunks of the package. Right. And like swapping out classes or interfaces. Right, yeah.
00:21:09
Speaker
Okay, so those are the those are the core principles. I mean, I think that like adding to the top just like great DX. I'm just gonna put that there because I do think that that's like fundamentally a really big thing. Yes.
00:21:23
Speaker
Um, so, uh, moving on like the next, this next bit, the like life cycle, I don't know if we were just going to get into it. I, you know, for the well listeners, I think that is like a little deep for now. I kind of want to start at higher level. What are we doing this weekend? I don't think starting with life cycles, what we should do this weekend. Yeah.
00:21:47
Speaker
you know? Yeah. Okay. Um, so I think for me, like, what do you think the first code we should write is? I think that we need to figure out transactions. I think that we need to really nail because so this is a big, this is like the big question mark for me is the way we want to approach
00:22:13
Speaker
events being like committed to the event store, the database is through a transactional model like database transactions, right? So the idea is instead of having to specifically commit events to the database or have like each event auto commit to the database when you fire it,
00:22:34
Speaker
we want events to go into a queue that is then committed in one batch operation at the end of the request, like if you're triggering events inside of a HTTP request. And this is the happiest of all paths. Yes, obviously there are lots of edge cases, but
00:22:49
Speaker
The vast majority of the code that we care about is like a request comes in, you do some work, and you send a response back, and then things happen. When we say commit, we mean like, so like in the Spotsy package it would be persist, right? But basically what we're saying is like,
00:23:08
Speaker
okay, I want this event to now be stored in the database and for any effects that are caused by this event to trigger, any permanent effects that are caused by this thing to trigger. Right, any side effects that happen need to happen after you know that the event has been persisted because the worst possible scenario is like,
00:23:33
Speaker
an event fires, something happens, and then the event fails to get persisted, and then something in the future reacts to that side effect. We have side effects that happened for an event that never happened. Right, right. So there needs to be this guarantee that the event has been persisted before you do any basically permanent effects from that. But at the same time, we also want it to be efficient.
00:24:01
Speaker
So if you've ever played like a turn-based strategy game, right? Right. Sometimes you'll have like a couple of moves that you can make and then you like lock in your turn. Or like if you've ever played like D&D, right? Like you can do like, okay, I'm going to make a movement and an action or whatever, right? And so you like do your things and then you lock in your turn, right? And then those things play out in order, right? That's essentially what we're talking about. So we're like, you can build up a queue of events, right? And then like lock them in and then they take effect.
00:24:31
Speaker
in that order and to the degree that's possible we want any like sort of memory or context or aggregate specific state to be impacted immediately in memory yep so that like you can make more future decisions based on it
00:24:50
Speaker
inside of your request, you can kind of pretend like it has persisted already, but also know that like, if something breaks, you're still okay. Right. And I think that's the biggest challenge is like figuring out how do we, how do we like get this queue, right? Make sure that we get the like auto persist
00:25:14
Speaker
uh, like auto commit to happen at a point in time. That's like after your application code runs, but before the request has been respond responded with so that like, if something fails, the user doesn't get like a successful response, but then like an exception happens. Mm-hmm.
00:25:31
Speaker
And how do we allow for rollback if something happens? We've already basically, I have a lot of detailed notes on this. This is the one solved, or this is the one sort of unimplemented major thing, is the event queue.
Tackling Transaction Management Challenges
00:25:53
Speaker
You can fire an event and project it, and it'll get caught, and we have
00:25:58
Speaker
events that fire once and events that fire, you know, blah, blah, blah. But I think we're probably going to re-implement a lot of stuff ground up. Yeah, for sure. But this is from a design perspective, like the major thing that's unknown. And I do think it is like a big wrapper around a lot of other things, conceptually. Right. There are, I think, some traps that can, I want to be careful not to get caught in conceptual traps too early. Okay. Right. So I want to get
00:26:30
Speaker
When we get this paper on the wall and start drawing, I want to design the whole thing. And I'm more than happy to leave things and say, this is a black box that we'll solve later. And there's something that you seem to think is highest priority or very high priority that I'm actually not that concerned about.
00:26:51
Speaker
Which is like, if some events get queued, like get added to like a layer of LQ, like a job queue and others don't in a given transaction, I am way comfortable saying like, either you're queuing events or you're not. You can't mix and match.
00:27:16
Speaker
I guess, I mean, maybe I'm okay with that, but I do think that we need to account for events being queued or events or projections being queued or projections being sacredness. And I, I think that like, especially if projections are going to be queued, like that layer of like separation between when the event, when you're like the code that triggered the event happened and when the queued projection happens, like.
00:27:46
Speaker
making sure that we don't run into like race conditions. But your event world stuff should never be using the projected data anyway. Right. Yeah. I mean, the only time it's a problem is if it's aggregate data that has a race condition. Right. Right. And the only time that's a problem is if some of the events are getting queued and others are happening in real time.
00:28:09
Speaker
Well, yeah. And I think even if, um, I don't think that like updating the aggregate state should ever be queued. I think that side effects, like, so this is, this is a tricky thing is like, we're talking about new words, but we're also using the old words. I'm gonna just keep on using the old words for right now, but like, I think that projections,
00:28:34
Speaker
should be cueable or not cueable, depending on like what the, the urgency of that work is. But I think that updating the aggregate state should always be, um, synchronous. So, uh, can you, can you find a race condition?
00:28:58
Speaker
Well, so in that world, you probably don't, the only race condition that I'm worried about really is making sure that the, because we have this like event queue, right? That's gonna commit at the end of the request. We need to make sure that like the events get committed before any projections get queued.
00:29:23
Speaker
Right. Cause we don't want projections to get queued and then the events to end up getting rolled back. And now you've got queued projections that are dealing with state that like didn't actually end up getting persistent. And I don't think that's a huge deal. I just think that there's like making sure that we get the timeline, walk me through it better with drawings later. Okay. Um, okay. Anyway, so there's that. So the big chunks of the app, or do you have something for the big chunks of the app already written down?
00:29:53
Speaker
Uh, in terms of the component components. Yes. I mean, yeah. So we've got the transaction manager. We talked about that, right? This is like handling the queue of events, committing events, um, rolling back events. Um, and, and that's pretty much it, right? So the transaction manager has to talk to the event store. Yes. Right. Or the event repository, whatever you're going to call it. We're calling it store. So the event store repository in my, in my thing.
00:30:19
Speaker
is what writes the events to persistence, whatever that is, which in V1 is likely to only be the database. Could be Kafka, could be a spreadsheet, I don't give a shit. DynamoDB, it probably will be something that we look into at some point.
00:30:38
Speaker
Um, and the note that I have here is I would like to think of the event store as rights are batched by default, right? There's no single event, right? You could send a batch of one event, but that like the event store is considered is thinking in multiples of events.
00:30:57
Speaker
Uh, from the beginning and like, if there's an implementation that can't do batching, we just loop over a batch and do each right individually. Um, so, uh, I think that's fine. I do just want to, uh, I don't want to solve this now on the podcast, but I do want to figure out what happens if like an exception gets thrown in a projector or something.
00:31:19
Speaker
Right. Well, but with the event store, that should happen before any projection happens anyway. Right. The events will get assisted. That's the question, right? If, uh, if there's 10 events in a row that happen and projecting event number four throws an exception.
00:31:38
Speaker
Oh, right. Yeah. Yeah. That's the stuff. That's the stuff I'm trying to figure out. So that's a good question about, especially thinking about batching. Um, so I'm cool to say like, I'm, I like minimizing database queries. I like, you know, single in like multiple multi insert stuff. But if an exception gets thrown, then we won't be able to insert the things that have already happened afterwards.
00:32:05
Speaker
My gut says that if a projector throws an exception, that should have no impact on the event system. That if, if the aggregate throws an exception, that will halt the event chain. I think that's correct. Right. Because I think so. And so I, I feel this kind of intuitively and you tell me if I'm crazy, but like, I'm pretty sure a projector should not depend on
00:32:34
Speaker
other projected. Yeah, I don't think so. Like there's no reason like the projector for event number five should care about the stuff that was projected by event number four, right? Yeah. I think that at the point, if you, if you queued up 10 events, right, at the point that those 10 events were ready to be committed to the store, I think that all the logic that could potentially throw an exception will need, will have already happened.
00:33:02
Speaker
Yeah. I mean, there's lots of things that can throw exceptions. Well, yeah, yeah. I mean, like maybe you have a non-fillable field or something, you know, like what.
00:33:09
Speaker
No, but what I'm saying, all the logic that impacts those 10 events or impacts the aggregate state. Anything that should stop future events from happening. Yeah, that will already happen. And if there is a failure in one of your projectors, that should just be a problem with that specific event and projection. And then this is the promise of event sourcing is that you can just fix the thing that's through the exception and then rerun those events and get the historical data correctly. Yep, exactly.
00:33:37
Speaker
Okay, so the transaction manager is going to have to talk with the event store a lot, right? Because the transaction manager is basically responsible for deciding when to send things to the store.
00:33:49
Speaker
The rest of the system is mostly going to talk to the transaction manager around getting events queued or rolling events back or all that stuff. The next thing, and this is a word that we haven't landed on. So in traditional event sourcing, you've got the aggregate root, right? Which is the easiest way to describe. I haven't landed on this word.
00:34:12
Speaker
Think, okay, well, we maybe disagree. But the aggregate root is like when you have an aggregate function in your database, like, you know, sum or min or max. When you are taking a bunch of events that have happened and aggregating some state from them. Yep, you're aggregating a bunch of events into a single, it's like a reducer base. Yeah, into a state. State or context are the two words that we have been talking about. Yeah.
00:34:38
Speaker
I still like context. I like context because it implies that it is like context to me feels like a subset of state, right? It's a slightly more specific. It does not feel like that to me. Okay. I would describe context as state that is meant to be used in the context of future events.
00:35:06
Speaker
Oh, I don't think of it. Okay. So I think of context and this is, might be from like react. Yeah. That I'm gathering this, but like I think of context as state that is provided distantly from where it is originated. Okay. Yeah. That's an interesting.
00:35:25
Speaker
So I think of it as like, you might have some sort of central context manager, right? So this is why I originally was interested in context, right? You might have some sort of central context manager or a context provider is what it would be called in React, right? And then you would consume that context in lots of different places. Right.
00:35:54
Speaker
So it's sort of like a hive mind state. Yeah. But that's the thing is like context or like the aggregate route, the date on your aggregate route is used across all the, all the events that are fired on that aggregate. Right.
00:36:12
Speaker
And that's why I think, I still think it's more like context. Like state feels, it's like this is data, this is contextual data that all events in this realm of the system need to know about. So here's my big argument against context. It is a word that has been beaten to death and no one likes it.
00:36:33
Speaker
Sure. I mean, it has like .NET implications. It's just a word that has been in programming for so long and has meant different things. And there's so many cases where it's like you're just passing CTX into some function for seemingly no reason.
00:36:55
Speaker
It just feels, context to me feels bad. State is like, everyone understands what a state is. It's like, oh, there's gonna be a couple of properties and they can get updated at various times. There's probably gonna be a couple of affordances for updating them intelligently. I mean, I can live with state. I just think it's simpler and it's a little less smelling your own farts. It does mean that we're gonna have to have an object called the state store, which I hate, but I can live with it. State to store.
00:37:23
Speaker
Because you have your event store, right? You also need a state store. Because I would just call it like a snapshot store or something. Sure, maybe that's fine. But you're storing snapshots of state. Yeah, you're storing snapshots of state. And I think one of the key things that I
00:37:47
Speaker
feel pretty good about i'm not like entirely married to because i can like imagine places where maybe it'll get gnarly and we decide it's not worth it but my belief is that states should a be singleton
00:38:08
Speaker
within the application context, like within a single memory instance of your application, there should only be one instance of a given state. I agree. Unless you, for some reason, explicitly nuke it and start over. Sure. But I don't think that user land code should be doing that.
00:38:29
Speaker
I think we should make it possible, but not easy. I wouldn't do it. So I think that states should be singleton in nature. And I think that a state should always be snapshotted. Or snapshotted by default. Obviously, you should be able to turn that off. But that state is just snapshotted. And that we do that.
00:38:58
Speaker
probably through the transaction manager as well so that like at the end of the transaction you commit the events you snapshot the state once you know that means that you have basically two rights ideally yeah one right for all the events one right for all the states yep exactly okay so then then the next piece is the event dispatcher right so this is like the thing that's responsible for
00:39:23
Speaker
registering event listeners, firing event listener, firing, passing events on to the appropriate listeners at the appropriate times. Yeah. I'm, I almost want to call it the event router. Okay. Cause it routes events to the appropriate places. Sure. It's kind of like a crossing guard type of character. Anyway, I'm, I'm a little ambivalent about the name at this point, but sure. This is the thing that basically says like,
00:39:54
Speaker
an event comes to me, I know where it goes. So I know which... Where events should be dispatched. Right. I know where to route events. Okay. I mean, I'm in favor of calling it the event dispatcher just because that's a very known entity. Well, if you give me state, I'll give you event dispatcher.
00:40:23
Speaker
All right. Tentatively, I'm okay with that. All right. Very good. This is great business with you. Um, and the last one right now we're calling it the event broker. There is maybe an argument to call it the event bus, but I personally can't describe the difference between an event dispatcher and an event bus, even though I know they're different.
00:40:45
Speaker
What does the event broker do in your mind at this point? Because last time we discussed the event broker, the transaction manager didn't exist. I thought we had renamed the event broker to the transaction manager. No. So in my mind,
00:41:01
Speaker
Conceptually, events need to flow through a handful of different systems and land in the right places at the right times. So there's the moment in time that the event is fired. And from that moment, you need to do some validation.
00:41:18
Speaker
You need to potentially fetch the right context to the right state to work with the event. You need to know where the event needs to be dispatched, what listeners are listening for that event, what projections need to be fired. All these different things are going to happen. Is that not the event dispatcher?
00:41:46
Speaker
Well, yes, but but basically, there's kind of I think there's a need for this like broker object that is just responsible for saying, Okay, first, I'm going to pass the event over to the guard, oh, which is not a thing yet. But to the guard that like checks to see whether the event is allowed to be fired, right? And you have you have taken one thing and made it so many things.
00:42:12
Speaker
And even before that, first I need to load up the correct state for this event if it hasn't been provided, right? Because another idea that we have here is it should be possible for events to basically implicitly load their state if that's possible, right? So somehow you need to load the state for the event
00:42:35
Speaker
Can I tell you what I think the architecture is? Because I think it's way simpler.
Simplified Event Architecture Brainstorm
00:42:41
Speaker
And then you tell me why these added classes are necessary. So I think there is a class that deals with many events. These are, I don't know, that keeps queued events in a queue and deals with which events have happened and have not happened yet. And is aware of events as a more than one event thing.
00:43:05
Speaker
which I would call the transaction manager or the bus or something like that. And it needs to be able to roll them back and commit them and do all this stuff. Then there's a class that deals with a single event when it happens. And that is the thing that needs to route it to
00:43:33
Speaker
apply it to aggregates or states or other, to apply it to states, to project it, to send it off to the event store to get stored, you know, whatever. So to me, it's like there's a class that deals with events as a group. And there's a class that deals with events individually. Why are there four classes?
00:43:58
Speaker
Because because like I think all of the guards and validations and stuff can happen on the Event router they just if it's not valid then you don't route it there. Well, I think that to my mind there's there's I
00:44:19
Speaker
Both of those are true, but I feel like those two pieces should only, like an event should only make it into the transaction manager, into the queue once it's been like validated. There needs to be a thing that decides, hey, is this event legal to fire?
00:44:40
Speaker
And it has to go into the queue before it can go to the dispatcher because we can't start dispatching the event until we know that the event's been persisted, right? So in my mind, you've got the event queue, the transaction manager, whatever,
00:44:58
Speaker
that holds all the events and then commits them. Sorry, I don't know what you mean by dispatch here. Dispatch is kind of a vague word. You say we can't start dispatching the event. Well, we can't like pass the event on to projectors. Right, okay. We can't start projecting the event. We can't start projecting the event, yeah. And we can't start, like I'm thinking, I think the event dispatcher
00:45:20
Speaker
is responsible for passing the event to what would be traditionally called like projectors and reactors, dispatching the event once it's been persisted into the system.
00:45:32
Speaker
And I think that the event broker in my mind is responsible for dealing with the event during the period of time when we don't know if it's legal to fire it yet, right? Yeah. So I, I think of it basically as like one thing that has two cues where it's like, these are the unpersisted events or they're sorry. These are the like on.
00:45:58
Speaker
Unknown, like these are the things that are like only in memory, right safe, right? And then these are the ones that are like real world safe But even the ones that are only in memory, I mean if there's a way to just use two transaction queues,
00:46:14
Speaker
I'm all like if there's if there's basically like transaction queue dispatcher transaction queue dispatcher and it's like the transaction the first transaction queue dispatches to aggregates and then the second transaction queue dispatches to projections like is that kind of what you're picturing basically although I'm kind of I'm not thinking of
00:46:35
Speaker
multiple instances of the transaction queue. Yeah, right. It may be just one object. Yeah. But if there's a singleton transaction queue that just has two public properties that are arrays of events, right? If, if it's possible to implement it that way, obviously simplifying the system is great. Yeah.
00:46:53
Speaker
I think that when we actually sit down and start looking at the code again, you'll see why we need this broker. Because I think, let's use a real world example. For us, there's the exam system. So when you have a user answered question event, before we can fire the user answered question event,
00:47:23
Speaker
We need to see, is it a valid exam? Are they past the time limit? We have to guard against the aggregate state. At some point, we need to load the aggregate route for the exam session. We have to look at properties on the aggregate route.
00:47:45
Speaker
Yes. And we need to say, are we allowed to fire this event? All that needs to happen before the event is even fired. Right. But the event has been instantiated. It just hasn't been fired yet. Like the, the event object has been created, but like we're deciding at that moment in time, whether to pass it on to the rest of the system or like throw an exception or something. Yep. Right. Um,
00:48:12
Speaker
the work of, and then before that event, once the event is passes that like guard phase, um, it needs to be applied to the aggregate route before anything else happens. Right. So all of that, like orchestrating the event through the aggregate state kind of happens before the event is even eligible for persistence or projection.
00:48:40
Speaker
Yep. And so in my mind, having a separate object that's responsible for that, like phase of the event life cycle just makes life way simpler rather than trying to reuse. I kind of thought of it more as like an object that.
00:48:58
Speaker
guides an event through its phases. Uh-huh. Rather than like one object per phase. I thought of more of like a, uh, event shepherd that like shepherds it through the phases. Well, I, so I, I think of the broker as that, that the broker basically guides the event through those early phases, passes it onto the transaction manager. A queue of events going to aggregates.
00:49:26
Speaker
No, I think that should all happen synchronously. Okay. Yeah. Yeah. That's why, that's why I wasn't thinking about it as like the transaction manager being part of that. Yeah. I think all of that happened, happened synchronously before the event ever touches the transaction manager. And then the broker is basically responsible for think of this as like validation. I think basically all of this stuff is like validation before the transaction manager.
00:49:50
Speaker
Yeah, yeah. That's what the that's mostly the broker is just kind of loading, loading state and validating state. Right. But I just kind of think like what you would do is you would just pass it to the transaction manager and the transaction manager would be like, this is valid before I store it. Is it valid? Sure. If we wanted to just use one thought, I just think in my mind, splitting that into two objects feels like a form request to me. Whereas I'm cool to put my validation in my controller. Right.
00:50:18
Speaker
I think that object is going to get big enough that I can see it. I get it. I get it. So yeah, the one thing that I don't have on here is the, uh, like right now I have like this concept of a guard implemented, right? And all that, all that is, is just like a custom place to handle all the little validation logic that we have.
00:50:40
Speaker
and like to step back because I feel like so I do think there's like sorry go ahead I think the one thing that I think needs to get written is like a
00:50:53
Speaker
some like handler detection code. Cause like we have lots of types of handlers. So we have validation handlers, we have apply handlers, and then we have like actual event handlers. And those can be on the event itself. They can be off on States. Like theoretically they can be in like
00:51:18
Speaker
free standing handler anonymous classes like yeah whatever and so I think there is like given all of these places that you might want to look for handlers and this event instance go find all the methods that
00:51:38
Speaker
handle this event in this state. Yeah, yeah, right now. So we have the event guard and we do have that the reflector, which we're right, but that was mostly just looking for like projectors. Yeah, now it's like we need to find the validate methods, we need to find the apply methods, we need to find and we need to find only the validate methods when we're looking for validate methods, we need to find only the apply methods, we're looking for apply methods. Yeah. So all that stuff, like, yeah,
00:52:03
Speaker
I think of that more as just like utility code though, right? There's going to be other like support stuff. Yeah. I'm just so that's a big one. Spotsy has this package called better types, which has a bunch of code in it for like.
00:52:20
Speaker
figuring out what sorts of, you can say give me all the methods that accept this thing as a parameter or whatever, and it's got problems and it causes problems sometimes and whatever.
00:52:36
Speaker
It's a pretty beefy package. Uh, and that's cause there is like a lot of stuff going on there. Like, and so like, there are like lots of, I think there's actually like a lot of hidden complexity in like this. So I would want to, I want to nail that system. Sure. Yeah. Yeah. That's that's fair. Um,
00:52:57
Speaker
Yeah. And I definitely get that there is a lot of complexity there. I feel like that's one of those ones where you think it's a big thing. And I'm like, yeah. We'll just implement it. I don't know how. Have you done any performance benchmarking on reflection? I haven't. It doesn't feel like it would be slow.
00:53:20
Speaker
does it feel it feels like it should be. It feels like it should be slow. But like, if you think about like PHP attributes, the only interface to work with attributes is reflection, right? So how if, if it must be, must be fine. Yeah. And like Laravel uses reflection all over. Right. But to me, it does kind of feel like I should treat them as something that we should optimize to do it as few times as possible. Yes, for sure.
00:53:44
Speaker
You know what I'm saying? Yeah. Yeah. And so these are kind of the things where I'm like, that's where I'm starting to see complexity, but maybe it's like so fast that it doesn't even matter. I have a feeling this. So this is how I approach the reflector, like the reflector implementation right now is like, essentially, uh, we're just going to do it. And if it's a problem, we can like add a cash. Okay. Cause like, I feel like.
00:54:09
Speaker
In most production systems, we don't use event discovery in Laravel, but I know Taylor recently talked about event discovery is turned off by default because it's slow if you don't run like PHP R's in events discover or whatever the command is.
00:54:30
Speaker
if you don't cache them, right? But in 100% of production applications, caching them is just fine, right? And if you use a continuous delivery, some sort of automation, then you never have to think about it. You just add that command to your deploy script, and it just works, right? And I imagine that we can solve this with that. I'm no longer concerned. OK, cool.
00:54:57
Speaker
So yeah, I think that those, I think that those are the main pieces. Yep. And it's just like, yeah. Okay. So about the broker, whether it's necessary, I'm fine with here's the call I want to make right now. Okay.
00:55:14
Speaker
do we just say start from scratch? I think we start from scratch. I think so too. I think we look at the code base as needed. Yeah. But like we have a code base that's like mostly a functional package. But like, I think we start from scratch and
00:55:33
Speaker
Uh, I think I already sold you on this, but basically like, I think we do everything as explicitly as possible at first, which may mean not very modular or hot swappable or. Sure.
00:55:49
Speaker
extensible. Sure. And then like at the end, we go through and we say like, all right, let's like pop everything out into interfaces and like make it extensible at the end. Because I just want to limit the number of places we have to make changes.
00:56:04
Speaker
Yeah, no, I'm good with that.
API-First Approach and Developer Experience
00:56:06
Speaker
I think I think that, you know, my only we talked about this a little bit, but my only pushback there is I do want to be thinking. Yeah, there are going to be things that are obviously. Yeah. Or like there's going to be places where like the functionality is dependent on. Right.
00:56:20
Speaker
the architecture. Obviously in those places we do the thing. But I just don't wanna pre-architect extensibility points before we have built the code. API first design. Okay, do you wanna talk publicly here a little bit about
00:56:48
Speaker
how we want verbs to work, because we've talked a lot about how we want to build all these pieces, but there are a few key design decisions that have informed these, and maybe we'll leave that for later, or it might be useful for us to just codify what is good about this API.
00:57:09
Speaker
Yes. So these are where the opinionated API design things come in. Yeah. For me, the biggest one is you should be able to add a new... If you want to add a user registered, I don't know, that's terrible, but we want to add a user skipped question event to the exam system.
00:57:38
Speaker
Right. I want to be able to just create a single file called user skipped question. And I want to be able to implement the entire feature in that file. That's like, yes. Number one bingo. That's my big thing. Right. And so what that means is that event is going to
00:58:03
Speaker
have the capacity to self-project. It's going to have the capacity to apply itself to the aggregate state, to the state. It's going to have the capacity to validate whether it can be fired, given the existing state. It's going to have the capacity to trigger side effects that should only happen when it's originated, not when it's replayed.
00:58:28
Speaker
or vice versa, it should be able to say, do this thing the first time I fire, but not any other times. Do this thing only when I'm replayed, but not the first time, right? And all of that should be possible to do inside the single event while still respecting those boundaries that we've talked about, right? Yeah, so to me, for 98% of events that I write,
00:58:55
Speaker
that is going to be a single class with some properties and one or two methods. And apply and a project or whatever. And some of them not even a project. Or some of them not even an apply. That's interesting. I tend to not project more than I don't apply.
00:59:22
Speaker
There's some things where it's like, this just needs to touch the aggregate. It doesn't actually matter anywhere. But anyway, but yeah, so like single file everything, which means and like, you know, traditionally, it's been like,
00:59:35
Speaker
You put your projectors in a projector file, you put your aggregate roots in an aggregate file, which is where you define your apply methods and your validate methods. You also have these separate files called reactors, which are basically projectors, but they only happen once.
00:59:53
Speaker
You know all of that stuff like there's a lot of places so like an event and then the event itself is a class right so then you would have like an event so to implement a feature you would say like here's my event then here are my changes to the aggregate route to make you would typically do two.
01:00:09
Speaker
add two methods to the aggregate root, create the events, maybe some properties to the aggregate root, then you would add a method to a projector and potentially a method to a reactor. And for some events you would add multiple methods to multiple projectors or whatever. So that
01:00:28
Speaker
And I also think the reality is that, at least for the applications that we worked on, 98% of the time, it's exactly that and no more, no less. Every once in a while, you add a listener to two projectors. But in the vast majority of time, you add exactly one listener to one projector,
01:00:53
Speaker
You add one apply method to your aggregate root, one fire method to your aggregate root, and create one event, right? It's like four changes every time for every new thing. And they all have the same name, too. It sucks. Yes. So it's like you fire a user profile updated event.
01:01:08
Speaker
And so then you add a on user profile updated handler to the user profile projector. And then you add a update user profile method to the user profile aggregate. And then you add a on user profile updated apply user profile updated handler to the aggregate. And then maybe you also send like an email when it happens. So then you also add an on user profile update to the user profile reactor.
01:01:37
Speaker
Yeah, so you have like five files called user profile, whatever, and then you have a bunch of methods called on user profile updated apply user profile updated and then the event itself is called user profile updated. It just feels a little. Yeah. And so in this vision of event sourcing, the alternative is.
01:01:55
Speaker
we add one file called user profile updated, right? It has a constructor that accepts the properties of the event, right? And then it has an apply method that accepts the user profile state and updates the state with whatever data is necessary for future events to, like whatever your future events are gonna need to check on that state, like it'll apply that to the state.
01:02:22
Speaker
And then it'll have a projector. We have, we have a better name for it, but I'm blanking out. Oh, just listen. Yeah. Um, that will project data to wherever you want. So in this case you would do like a, you know, user where aggregate UUID or where snowflake will probably be using snowflakes here. So like user where event ID or whatever equals.
01:02:49
Speaker
Oh, so interesting. Update interesting thing about snowflakes. Yeah, I kind of want to re I just want to sanity check snowflakes. Okay, because I didn't know about you lids in Laravel before. Okay, and now I do know that you lids are I have like first party support in Laravel and
01:03:09
Speaker
Snowflakes are better. I know they're better, but maybe they're not better enough. Yeah. You know? Yeah. So that's my, that's the only like open question is like, should we use UL IDs instead of snowflakes? Yeah. I put a pin in it. I don't want to solve it right now, but this is a thought. Yeah. Um, I mean, and it should be, you should be able to do, you should be able to do whatever you want to use. It's fine.
01:03:30
Speaker
Um, okay. That is that, but yeah, I think generally speak. Oh, and also like, obviously you and I both want to write single file everything. Um,
01:03:41
Speaker
obviously you can still do all of that craziness that we were talking about. So do you want to have projector files and you know you want to have your separate aggregate files with their own validate and apply methods like well and and you can do all that in this case like I can totally see wanting to do that for like the secondary use case right. Yeah so for like I have a separate listener that is more side effecty
01:04:06
Speaker
or like you're aggregating stats. I think that's a perfect example. Like I may have like an analytics user analytics projector, right? That like listens to user profile updates and ticks up some count somewhere. Yeah. But you don't want to, you don't want to have all the stuff right there on the user profile update about its analytics. Right. You want that to be more passively off in its own file. And if you think about like, quote unquote separation of concerns, right? This is like,
01:04:33
Speaker
This is actual separation concerns. Your concern isn't updating the user profile. Your concern is keeping track of things that happened. As opposed to, so you should have one file about updating the user profile. You should have another file about keeping track of shit that happened. Or notifying the user that things happened.
01:04:57
Speaker
you know, sending security alerts when something happened. Yeah, yeah, exactly. Like, so that's that's 100%. I think both of us will definitely register projectors for sure, we will definitely
01:05:12
Speaker
have situations, I can totally see having situations where you want like the more traditional aggregate route, because that API just feels better for the system that you're doing. Yeah. Right. Where it's easier to load up the aggregate route and fire the event from the aggregate, then fire each event, each event individually.
01:05:31
Speaker
So I actually kind of want to like re harp on separation of concerns. Yeah, because like this is I love this. This is the stuff that really made us want to make this package was like so separation concerns if any of you guys like aren't familiar with solid architecture like um
01:05:49
Speaker
The separation concerns is this whole concept that gets abused to hell by Dutch people and they they love to yell at you about like whatever but
01:06:06
Speaker
Their idea, the idea of the bad people, is that separation of concerns is like, oh well like, this thing sends email, like sending email is a concern, or even worse, that like sending email through a specific technical implementation is a concern, right? They think about concerns so much more from a technical perspective than from a
01:06:33
Speaker
feature perspective. Whereas I think of concerns much more from a, but what does it do? Like, what ticket am I closing in linear? That is the concern to me, you know? And so I generally tend to group code by behavior or by action, you know? But what does it do? Yeah, I think action.
01:07:02
Speaker
Right. It's tricky. Cause I mean, I feel like that's my main, I, I, I think that if there was like, if I could wave a wand and change one thing about Laravel, it would be like the default folder structure of controllers go in like HTTP controllers and form requests go in HTTP form requests and jobs go in like, you know, yeah. And then tests go in tests and migrations go database slash migration.
01:07:29
Speaker
I want the user request and the user controller and the user resource or whatever things that you have. Those should all be as physically close to each other in your file system tree as possible. Yeah, because you're working on them all at the same time. It's a means and plus. Right.
01:07:53
Speaker
What you want to do is chop your onions, chop your, you know, your peppers, chop your celery, right? Prepare them. You want to get your oil out, you want to get your pan out, you want to heat the pan, right? Then you mix up your holy trinity. Now you're making a nice gumbo.
01:08:09
Speaker
You know? But the issue, right, is that some of us, some of us are like, well, no, you see onions and peppers both come from the ground. So these are ground concerned. No, no, no, it's more like those are in pint containers. So that goes in the pint containers section. And those are quart containers. So those go in the quart container section.
01:08:38
Speaker
Right. So I actually have a separate kitchen for all my court container items. And I mean, you know, I can hear, I can hear people saying, well, like you should just use your keyboard for everything. Why are you, why do you have the file system tree open? And like, of course, it's not about the file system tree.
01:08:54
Speaker
Yeah. Well, but a lot of times when you're coming back to old code or you're looking at code that you're not familiar with, the first thing you do is open up the file system tree and look at the files because you don't know what they're named yet. Also like for code review, like when you're looking at like a pull request and all of the changes happened inside like the permits module, you're like, all right, there's a very small chance
01:09:18
Speaker
that this had unintended side effects outside of this particular feature. Yeah. Yeah. Yeah. I mean, uh, it's cool. Go check out international modular if you haven't used it. Bro, we did a lot of good projects at module or at international. Like modular was one of them. Modular is the best. All right. So, so I think,
01:09:46
Speaker
The next thing, okay, wait, wait, I have, so I have a couple just like little bullet points. Let's just talk about them real quick. Okay. Before we stop.
01:09:53
Speaker
So one bold point I have is like testing. I think that we should pretty early on have a good sense of like what the testing story for verbs looks like and have like really good test affordances. You mean like the user-facing test disorder? Yes, yeah, yeah, yeah. So here is, can I soapbox? Yeah, yeah, go ahead, go ahead. So my whole examples thing.
01:10:20
Speaker
Yes. Okay. So I started working on this thing that was basically like, look, let's have examples. So modular has this like already existing architecture, which is like, Hey, we're basically going to duplicate the app folder of a Laravel app.
01:10:36
Speaker
in modules. So like you have your controller's directory and your, you know, or you have your HTTP controllers and you have like your routes and all this stuff like on like a domain by domain basis, kind of, right?
01:10:54
Speaker
So the the idea with examples is we basically just something very similar to modular where you have an examples folder that has like a source directory and a tests directory. Right. Yeah, and its own read me. Yeah.
01:11:09
Speaker
So the source directory, you can register models, you can register events and states and all this stuff. Controllers if you want to test the API, whatever. And then you can write tests. So basically the idea is we'll spin up that source directory in a Laravel app.
01:11:30
Speaker
Yeah. Have you seen, have you seen workbench work? Yeah. Everyone keeps sending it to me cause I talked about this on a podcast, but the idea, I haven't actually like, I haven't looked at it yet. Yeah. But something similar to that. Yeah. But I kind of want to do it our way if we have time. Yeah, for sure. Um, but like the idea that like we, you could basically like spin up a small layer of a lap and running your tests against it so that you can write domain tests because like the worst part of our tests for our first implementation.
01:12:00
Speaker
was like, it was, we're like declaring anonymous classes in the test bodies. And like, it's just all this stuff where it sucks. And it, I really like to read tests as documentation. Yeah. And, uh, you can't really read our tests as documentation. Like I remember I was at Lara Con, I was trying to show Keith how it worked and I was trying to walk him through the tests as like a way to show him that the tests are so crazy that like,
01:12:26
Speaker
No, I agree. So like, I just like the idea that like, we're going to register all of our classes and stuff in ways that look familiar. Yeah. And then we can write tests like we're just testing an application. Yeah. And what that means is our testing story will be the same as the user's testing story.
01:12:45
Speaker
Right. I mean, we'll still write unit tests for our, you know, but we'll have to consider the event tests or whatever. Yeah. But like we will consume the user testing story. Yep. So like the user facing testing story is our story. Yeah. I like that. Yeah. And I mean, I think, I think that like,
01:13:02
Speaker
Um, in the existing test suite, like the cool thing is, right. We've got these like use case tests and we've actually got like a separate, we have the same use case, right? So like the banking use case, use case, for example, shown tested both in like the traditional structure where there is a projector and there is an aggregate and there is an event and there is, you know, all that stuff. And then the same exact.
01:13:29
Speaker
outcomes are tested in the like encapsulated use case. And I love the idea of being able to show that, but show that in like the context of like the structure of a regular app. I think that's very cool. Yeah, I love that. Okay. One thing. So I've been talking about this with, uh, the couple of folks over at internet. Sure. I'm grabbing my coffee while you keep talking. Yeah, go, go town. And, um, one thing that, uh,
01:13:58
Speaker
I think that one thing that I really love about the Spotsy package is there's kind of like a nice test set up where you can just like quickly set up the world and then perform assertions on like the event system, like kind of given that these three events fired, this is what I expect. And the thing that kind of sucks about that is
01:14:27
Speaker
sometimes you have to fire 12 test events to get the state into the place that you want it just to test the thing that needs to happen right after that. And so we were talking about this idea of like state factories and like basically say like well given this current state. This matters for cedars too.
01:14:52
Speaker
Seating event source data is a pain in the ass. We just implemented some cedars for a client of ours, and the client called me and you could tell he was kind of disappointed with the cedars that we wrote, which to be fair, they were more verbose than they needed to be, because we hadn't yet written some of the convenience methods on the model that were going to wrap up a bunch of the aggregate methods.
01:15:17
Speaker
Yeah, you don't want to write a bunch of aggregate code in your cedars right seems crazy and and for like a test like in a test case like you can skip the whole like here's the stuff that happened before that got me to this place as long as like there's a good story for like
01:15:35
Speaker
Given that the like, you know, the state snapshot looks like this, ignore any of like, don't worry about what got us there. We can, we can not have that. Well, just have the state and then trigger events from that state because that's the only thing that I'm testing.
01:15:53
Speaker
So pecking order is this game that we made this event source, right? Yep. And, uh, through building this game, I taught my friends to code, right? Yeah. And so there was lots of places where I was like, willing to just not do the right thing in order to do the easy for them to implement thing. Right.
01:16:13
Speaker
And one of the things that we did was in order to make it easy for them to write tests always, I made this affordance called Boot Game that just boots up a game and creates this many teams and assigns these players to this team and does all the world building.
01:16:33
Speaker
because of that our test suite is so slow because every single test runs boot game basically of like 400 tests each of them like builds the entire world right we could easily with this verb stuff
01:16:49
Speaker
have snapshotted the boot game state and just like, boom. And our tests, like if we just converted to verbs right now and then like swapped out boot game with like implement a snapshot, like our tests would go from being like eight minutes to being like a minute and 40 seconds, I feel like. And then imagine if you had basically factory classes for your con for your state, right? So then you can do like, then you don't even need the boot game method. You could just,
01:17:18
Speaker
Well, yeah, and you also wouldn't need to just have one, instead of having just one sort of snapshot that you use for a specific state, you could say, give me the game state where they're on round three and whatever. And each of these fluent methods would progressively build up the state
01:17:46
Speaker
just like a factory, like you get your base definition, but that all happens in memory. Yeah. And then just like insert all the events and yeah, you just get the snapshot after all of the factory methods have run. And I think that's very cool. Yep. That's super appealing.
01:18:02
Speaker
Okay. Another thing is, um, I've been looking at event sauce because that's the, uh, package that I'm less familiar with. And I just want to like understand their concepts. And one of the things that they do, uh, is called upcasting. Have you looked at that? So it's, it's essentially like schema migration sort of like, you know, how Stripe very famously like.
01:18:23
Speaker
has an up method for every API version and a down method for every API version so that under the hood, I can be on a 10-year-old API and they can migrate the modern response down to whatever state. That's how Stripe handles. Basically, you can use any version of the Stripe API and it always works.
01:18:48
Speaker
And I think kind of basically like you could like version your projectors Yeah, you would version you would probably version the the state Right. And so I was thinking that one thing we might want to consider is just like having state versioning be like built-in Okay, we're like under the hood. There's always like a version to do this I'm also happy to do this in three months
01:19:14
Speaker
It could very well be that it's not worth it. But it was something that I just went to pin. Well, no, I think it's cool. I don't want to do it this weekend. Right. You know what I'm saying? I want to make sure that it's possible for us to build in the future. Yeah, they will be. And that brings me to the other piece, which is message metadata. So EventSauce has decorators. The Spotsy package basically just uses eloquent hooks because it's built on top of eloquent.
01:19:44
Speaker
But in general, I think we just want to be thinking about it should be possible to essentially register things at different levels of granularity to augment event data. So that you could essentially just install a schema versioning package and suddenly all your events have a version variable attached to them.
01:20:09
Speaker
Or like, you know, you might want to like, be able to just like automatically apply like the tenant ID to a multi tenant application. Yeah, now that we think about it, like I kind of think versioning might be a separate package that is like a plugin, right? It could very well be that we implement as a plugin. But I'm just I'm more thinking like, as we're considering the architecture, I think we want to be
01:20:32
Speaker
we want to make sure that we allow for this idea of like sort of external pieces of code augmenting the data that the events are aggregating. So it is wild how like, I'm just thinking about like, messes people could get themselves into, you know? And like, I was like, Oh, like, what if you, cause like the problem with making it a separate package is like, what if they don't know about it? And then they try and like,
01:21:01
Speaker
I don't know. They just say they just sort of like naively just like change the behavior of some stuff. Yeah. But then I was like, you know, if you did get in trouble, like I just imagine someone emailing me and saying like, Oh, I got into this trouble. Right. And now all my data is messed up. Yeah. You know, but I, even if you do get into that trouble, like as long as you know, when you merged that pull request to production, Mm hmm.
01:21:27
Speaker
you can go back, insert version metadata on all the events up until you merge that pull request and insert different version metadata on the ones after that and replay the events and theoretically get back to sane data. Like there is like a world back to sanity. Whereas like in lots of other cases, like if you mess up the way that you're inserting data into your database and don't notice for a couple of weeks, that's a big problem. Yeah, for sure.
01:21:57
Speaker
Yeah, I mean I think there's a part of me that may decide that like this Even if it's not done this week like this may be something that I like throw in There are some things where I am like Chris don't like sprint ahead on this one without me. Yeah, this one I don't give a shit if you want to implement versioning on your own. I don't care. I
01:22:20
Speaker
Cause I, I, I think there's a, there's a part of me that's just like, if that was just baked in from the very beginning and like just another method that you can add onto your event is like migrate and it just accepts the data and the version. What if I feel like version enums might be an interesting thing, but. Right. Yeah. That's neither here nor there. Yeah. So I think that it might be cool and it might be cool to have it just like baked in from the very beginning. Cause then like.
01:22:50
Speaker
I think the reason that I like that is it's a little bit of like a comfort, like a safety blanket.
01:22:59
Speaker
It's just like security blanket. It means that you may feel a little more comfortable adopting it, knowing that, hey, if I get this wrong, there's a built-in first-party path to changing it in the future. And I kind of like that. But it's not the first thing that we're going to touch.
01:23:25
Speaker
Cool. Sweet. All right. Then this was a good place to stop. Yeah. We should sit down. We might end up recording another episode or two. Yeah, for sure. No, this is fun. This is good. Um, and, uh, yeah, take a break and then we'll actually like write code. Yeah. All right. Sounds good. All right. It's been a pleasure. Cheers.