Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Ep. 4: The dreaded status column
291 Plays1 year ago
Over Engineered is all about those things that bug you but you never get a chance to "solve." Today's episode is about the dreaded "status" column.
This is another topic that most developers will hit over and over. You have a model. You need to track the status. You add a status column, and then later a status timestamp "accepted_at", and then later an "accepted_by" column—and each time you cringe and wish there was a better way.
Today we discuss a better way… maybe?
Recommended
Transcript
00:00:08
Speaker
All right.
Introduction and Guest Welcome
00:00:09
Speaker
Welcome back to another episode of over-engineered, the podcast where we ask the very important question. What's the best way to do things that don't really matter? Uh, today I am here with my friends, Shane Rude and Daniel Colborne. Welcome guys. Hello. What up?
00:00:27
Speaker
And we have been talking about this particular topic outside of the podcasting world for a while. So I'm excited for it.
Guest Introductions
00:00:37
Speaker
But before we kind of get into what we're talking about today, would you two like to introduce yourselves? Sure. I'm Shane. I am a developer and own a couple of businesses in the magic gathering space.
00:00:57
Speaker
I've been working with PHP for a little over 15 years.
00:01:02
Speaker
Cool. Um, I'm Daniel. I'm a developer. I currently run like a small agency called thunk with a couple of my friends sort of specializing accidentally in like events sourcing and like live wire and Alpine stuff. So that is kind of what I'm up to these days, but I've written a lot of Laravel and various front end JavaScript frameworks and stuff over the last couple of years.
00:01:30
Speaker
I also can't imagine that someone will be listening to this without having heard of no plans to merge. But, uh, Daniel is also the co-host of the podcast. No plans to merge. No plans to merge.com. That's right. Not.fm. Uh, no, but I do own NPTM.fm, which redirects to no plans to merge.com. So weren't you the one who convinced me to get the.fm because the.com wasn't available. Yes.
00:01:58
Speaker
Okay.
Episode Theme: Status Column Challenges
00:02:00
Speaker
All right. Well, so event sourcing is definitely, uh, one of the things that's going to come up today, but let me, let me set the scene. So, right? Like, uh, every episode we pick a thing that
00:02:16
Speaker
programmers in general probably but certainly Laravel developers are likely to bump into over and over again and choose a solution that is fine but maybe not the best and one of them that Shane and I have talked about a ton and you and I have talked about a ton Daniel are
00:02:39
Speaker
status columns when you have a model and it has a status column and you need to deal with status transitions and you need to deal with time stamps related to statuses. I feel like this is a thorn in my side every few months.
00:02:58
Speaker
And I've never really been happy with the solution that I've come up with, whether that's just a column that is the status and then a couple of timestamp columns or some sort of status column that's derived from timestamp columns. I know that there are lots of packages out there that try to do this, but it's just always annoying to me.
00:03:22
Speaker
And I feel like one thing to call out is that the status column is not really a problem. The status column itself is a solution. And so I think there is getting to the underlying problem that a status column solves, which is like, I have a model that passes through phases and I want to know where it is in that journey.
00:03:46
Speaker
Yeah, where it is potentially when it got to where it is or when it got to specific phases. Potentially also how and why it got there, but a status column won't tell you those things. Yes, yeah, how and why. I think how and why don't come up as much for me, but sometimes they do for sure. When happens a lot, I think, depending on the status and then what the status is at the current moment, I think comes up all the time.
00:04:15
Speaker
Right. Is there anything that you want to add to that Shane? Yeah. I mean, this is something I see a lot of is various contexts of having a status column. And my general approach is just that have a status column and have a timestamp for every possible value. And that I find works in the majority of cases. So like, uh,
00:04:45
Speaker
accepted at, rejected at, declined at, canceled at just a bunch of time stamps for all of the different transitions.
Illustrating Status Challenges with Magic Card Example
00:04:55
Speaker
Gotcha. I feel like when we're talking about this, we should, I feel like we should like take a step back and like define a domain that we're trying to solve a problem for, even if it's like a fake made up domain.
00:05:08
Speaker
Do you have one? I'm happy to provide something or if like Shane, do you want to use a real world example from, from the app that you're working on or do you want to use? I have an example I could make up. Yeah. I mean, I definitely could offer some examples. I was just looking at stuff like this earlier today. Um,
00:05:31
Speaker
So I'm trying to pick like, what's the best sort of model to start with? Let's say somebody wanted to sell a collection of magic cards. Is that something I see all the time in my life? Yeah, let's just say for instance. So if I were thinking through what are the statuses of that kind of what I refer to as an inquiry. So they submit a form.
00:06:02
Speaker
And at that time, let's say it's created is the status. And we've set created at, which that comes with most models. We're all used to that already. And then from there, let's say we need to price it. So we have some kind of process that runs. At that time, we might find that we run into like an error state.
00:06:30
Speaker
Let's say that their file isn't valid. So we maybe want to have a way to handle that. It could be a status. We also might find that their cards aren't really worth our time. And so we might say we don't want to make an offer at all. But if it meets our criteria, we'll say we do generate an offer. So at that time, we would say, OK, the status is that we have issued an offer. Now the customer has a choice. They can decide to accept it.
00:07:00
Speaker
And if they do so, we have the offer accepted status. And then maybe we want to see that the cards were transferred to us. And once that's happened, there's a transferred status. And then we need to issue payment. And so at that point, once we've issued the payment, we would maybe call that completed. Yeah, I think, I mean, this is...
00:07:25
Speaker
sort of the typical flow, right? It's like you've got some sort of record, it's going to flow through a bunch of statuses. I think one of the places that I notice throws a wrench in things are like
00:07:40
Speaker
nonlinear statuses, things like an error state or an issue or needs. Maybe you have a linear flow of created, offer issued, offer accepted, payment,
00:07:56
Speaker
paid or whatever but you may have like in this case maybe maybe there's like a black lotus in there there's a car that's incredibly valuable and it needs some sort of manual review right so you i think the the one of the wrenches that i want to make sure that we like identify early is.
00:08:17
Speaker
states, situations where there's like a linear flow, but there are also like statuses that are outside of that linear flow that need to be accounted for. And maybe there's some sort of logic about like, when something gets into the issue state, it needs to go through some other process before it can get back into the regular flow. So I, I think that every
00:08:42
Speaker
Every flow no how no matter how regular it seems inevitably has some something that breaks it right if you live with the app long enough so for example like.
00:08:56
Speaker
Uh, the situation you just described, right? So like you accept a inquiry, right? And so then now they need to like send you the cards, right? Um, well, what happens if FedEx loses that package, right? So now FedEx loses the package. So like what state is the order in, in a situation where like.
00:09:15
Speaker
They accepted it. Maybe we were going to pay them. Then they sent us the thing and then the thing got lost, but we have the shipping number for it. And like, it's now in a state that is just like different than any other state we had planned for, right? The state of like, am I suing FedEx? Like what's going on here? You know? Yeah. Package lost state. Package lost state. Right. Yeah. I do think that.
Handling Non-Linear Status Flows
00:09:40
Speaker
Some of the things I work with when there is a status, it really is that cut and dry. It really doesn't change. And I think, Chris, you and I have talked a lot about stuff like this because we both have apps that we lived with for many years. And so we've seen the progression of certain things needing to be maintained or changing statuses like this needing to be maintained.
00:10:07
Speaker
Definitely, it does seem like a context-driven, complexity-driven situation, where the status column with timestamps works for me when things are simple. But once, to your point, Chris, things get nonlinear or you have branching paths, it gets hairier. And maybe that does call for things that are a more sophisticated handling. But yeah, to your specific question about this collection, what if it goes missing or something like that?
00:10:38
Speaker
Again, it depends on how sophisticated you want to get. You could just say, in the case of My One Business Card conduit where we're dealing with these kinds of shipments,
00:10:50
Speaker
We might have something as simple as a returned catch-all that says, we just shipped these cards back to the customer or canceled. And something that requires more support, like we have to get involved with FedEx or something like that, is really handled offline. It doesn't necessarily need to be the concern of the web app at that point.
00:11:16
Speaker
Right. You can just kind of use a catch-all like issue or like problem status and just leave it to manual action from there on out.
00:11:28
Speaker
I mean, and that's kind of how we do that. We have a similar sort of like job, job assignment, you know, work assignment feature where things tend to be linear, but we have those like catch all just, Oh, this, this job has a problem of some sort. It's in a problem state. And like, there's now it's just up to admins to figure out what to do with it.
00:11:55
Speaker
So I have kind of like a thesis, right?
Event Sourcing as a Solution
00:11:57
Speaker
So like, I'm, I'm kind of coming in with like, uh, with like, uh, an angle, right? Which is, I've said this publicly already, but like, I think like, generally speaking, status columns are like an indication, uh, that you want to actually be storing actions instead of the acted upon objects. Right. Um, and so like, if you think about like the status columns that we've brought up before it's, there's like offered
00:12:25
Speaker
Right. And then there's like accepted and rejected and sent, right. And all of them are sort of these like past tense verbs, um, that refer to a thing that happened. Right. Well, you can say an event, an event, an action, whatever, whatever you want to call it. Right. And I think that like in object oriented world.
00:12:47
Speaker
Um, we are so, we're really mostly only comfortable with nouns. Um, we only like to store objects and like objects with attributes and like things like an order or a, uh, a post or a user or, you know, things that people, places, and things. Um, and my argument is that like.
00:13:13
Speaker
the existence of status columns is a yearning for a verb, right? And like, if you've added a status column to an object,
00:13:25
Speaker
What you're saying is, I need a way to store verbs, but I want to keep inside of this noun-based paradigm, so I'm going to store verbs as attributes of nouns. Generally speaking, I think that
00:13:43
Speaker
the fact that we hamstring ourselves into only using objects and attributes leads us into these situations where we're forced to use catch all statuses because the actual specificity of what really happened doesn't fit the model. And so we're forced into these extremely
00:14:04
Speaker
broad catch all statuses where, you know, an object may have gotten into this state in 15 different ways that aren't really the same thing. But we're going to kind of try and find an equivalence between as many of them as possible just so that we don't end up with 45 different statuses that we have to write giant branching if statements about.
00:14:28
Speaker
I do think there is an inherent awkwardness to the status column. I've always felt that, I don't know, it is just a sort of awkward thing to work around. I'm comfortable with it now.
00:14:41
Speaker
and sort of have the pattern I like to apply. But event sourcing and the concepts around it are super interesting to me. I was actually recently telling Chris that that's something I'm starting to study and I think is going to be a really good fit for something I'm working on. I guess like what concerns me is the cost in terms of additional complexity and just not really being familiar with the implementation.
00:15:08
Speaker
Yeah, I mean, part of the reason that I asked the two of you to be on this call is like, I think these are two sort of your current approaches represent the two extremes, right? Like the status column and a bunch of timestamps is really obvious and works well, I think, until it doesn't.
00:15:38
Speaker
Excuse me. And then on the other side, treating statuses as events or actions or whatever probably more accurately represents the real world, but is a level of abstraction that
00:15:58
Speaker
I think is inherently less obvious. I come into each episode imagining a world where we solve this problem at the end of it, not with any belief that that's truly possible, but I would love to come out of this imagining a world where you can get the benefits of an invented approach.
00:16:26
Speaker
but the clarity of the straightforward just have a status column approach. That's my agenda. Maybe I shouldn't have an agenda, but that's where I am because I agree with both
00:16:43
Speaker
both of you on this. It's like I have seen the power of event sourcing and it's been really, really useful to us in the places that we've used it. But I also know that when I go back to that code, it takes me a little while to get my head wrapped back around how the heck it's working, you know?
00:17:05
Speaker
Yeah, I think one thing you said, which I'm not going to give you for free, is that event sourcing is somehow non-obvious or more of an abstraction or more of a mental
00:17:22
Speaker
convolution to wrap your head around than sort of the sort of status column approach, which you described as obvious. I think, I completely disagree, right? So I think the status column, I think of the status column approach as like a convolution and like taking a thing that was a clean concept. Like if I just described to you, the customer canceled the order.
00:17:48
Speaker
Right. Like that to me is a straightforward concept. And, uh, if I said to you, the customer moved the order into the status canceled, like to me, that's like a non-obvious, uh, concept, right? So I think that the, the actual implementations of event sourcing are different than, uh, a lot of regular object oriented code that we write. And so since we live in object oriented land, so much of the day.
00:18:18
Speaker
It's a mindset shift to think differently. And if 85% of your code is just regular object-oriented code, and you've got 15% that's event-sourced, the context switching may make it seem more complicated than it is. But when I live in an app that is fully event-sourced, or when I stay in event mode for extended periods of time, I think that
00:18:45
Speaker
to me, storing actions and thinking in terms of what happened as opposed to what were the secondary effects and the ways that the world changed as a result of what happened is easier for me to wrap my head around and more obvious.
Adoption Challenges of Event Sourcing
00:19:07
Speaker
Yeah, I totally agree with that. I guess I'll clarify.
00:19:12
Speaker
I think you're absolutely right that the ergonomics of triggering the events is much better. Like you said, accept the shipment is much more clear than set the shipment to the accepted status.
00:19:35
Speaker
And yeah, when I talk about obviousness, I guess I'm thinking within the context of the standard conventions of a Laravel app. This is not how the majority of Laravel developers think about code. So if you're bringing on a new developer or if you're working on
00:19:54
Speaker
an open source project that other people are going to be interacting with or anything where other people need to interface with the code, sticking to straightforward conventions, I think is more obvious. And I also will say that I find, and I'm just going to kind of
00:20:18
Speaker
lob this up as something to dispel later, but I think a large portion of the time
00:20:32
Speaker
When I'm dealing with statuses, I am dealing with them in the context of a point in time, right? So I wanna pull all of the jobs that need to be assigned, right? Or I want to find all the jobs that haven't, you know, were assigned but haven't been scheduled, you know, or, you know, where the window between when they were assigned and scheduled is more than three days or something like that.
00:21:02
Speaker
I'm kind of like, a lot of the time that I'm dealing with models that have a status column, it's not in the impacting the status, it's the dealing with the current state of the model. And so I think, you know, I can add a helper function that's like,
00:21:29
Speaker
accept to a model that just sets the status to accepted and now I get the ergonomics of like using the verb but it's a little bit harder if I'm just thinking of like okay a model has many events and now I need to like parse through all of the events to determine like what its current state is obviously that's
00:21:58
Speaker
If that were the solution, it would not be very ergonomic. Sure. Shane, what are you thinking? I guess thinking like you're describing this event sourcing as a way of seeing how things happened over time. And to me, this sort of status column concept with
00:22:22
Speaker
if you're providing a timestamp that you're setting for each one of those possible statuses, it's not that dissimilar from it, just in a much more primitive form. You're getting a timestamp. If you wanted to know the sequence of events, okay, the offer was generated at this time, and then this happened, and then this happened, and you can sort of read it back like a story just with the timestamps. You're just not
00:22:49
Speaker
You're not working with the nomenclature that you prefer or the concepts. And so that, admittedly, is a little more awkward. And of course, it's a more brittle, hard-coded situation. But you still get to tell the story when you look at the state of the model. Yes, you can look at the current status, and then you can look back at this expected sequence. Again, it's pretty linear.
00:23:19
Speaker
Um, but you, I mean, you do have that information. One of the things you go back. Oh, go ahead.
00:23:27
Speaker
Well, I was gonna say like one other thing that I feel like starts to make it unwieldy is as soon as you need a second piece of information about any status change, right? So for example, like say I need for legal purposes to capture a user's IP address and user agent when they accepted the offer, right? So that I can have some sort of indication that it was actually this user.
00:23:53
Speaker
my only option then is to add a user accepted offer IP column and a user accepted offer UA column and store both of those things. And if I need to, God forbid, store an IP and user agent for multiple status changes, right? So like the first status change, I need what IP were they on. The second status change, I need what IP were they on. Now I might be adding, rather than just adding one column for each,
00:24:22
Speaker
Status change I might be adding three columns for each status change I think that's totally valid and I think that that's where as things get more complex like These kinds of problems arise the other place that I Like not that long ago was grappling with some Aspects of this was I wanted to have a sort of wizard flow for our customers to be able to step through
00:24:50
Speaker
the various states of sending a shipment to us and there are
00:24:58
Speaker
various sort of like disclaimers along the way where we explain, okay, here are the fee structures, please agree to these terms. Here are the, I don't know, some other information or like things that we want the user to be presented with. And we want them to be able to go back. So you're on your current state or status where you're sort of seeing this is what you have to interact with. And maybe they're just waiting for something to happen. But we want them to be able to go back and see like, okay, what terms did I agree to? And what fees did you say they're going to be?
00:25:28
Speaker
And those are scenarios where it is pretty tricky to implement that with this, you know, pretty simple, you know, status and timestamps like storing non-final data or something like data, like this isn't finalized data that we can like use in the app yet. It only really exists to be able to like repopulate the form that they filled out that they want to view before submitting it.
00:25:55
Speaker
Yeah, and we want to make sure that what they're seeing is consistent with what they experienced. So if you change your fee structures, which could happen, you know, so you need to sort of be mindful of showing the same terms that they agree to, you know, whatever. And that stuff is, is tricky.
00:26:17
Speaker
So I feel like there are two concepts that, in my mind, if I'm trying to imagine a really nice generalized solution to this problem,
00:26:29
Speaker
The two things that I'm going to reach for are state machines and event sourcing.
State Machines and Event Sourcing
00:26:35
Speaker
I think that while I'm not convinced yet that just a regular vanilla Laravel app should immediately jump to event sourcing when they need a status column because I do worry about the context switching there.
00:26:58
Speaker
I wonder if there's maybe an abstraction around event sourcing that would feel more obvious in this object-oriented world. And I think that there's been a lot of work done around state machines, state transitions, and the
00:27:18
Speaker
sort of rules, because that's something we haven't really talked about. But oftentimes, for more complex systems, you may have rules in place about what transitions are possible. Can you go from A to B? Or you can't go from A to C, you need to go from A to B to C.
00:27:41
Speaker
And so I think I would guess that state machines, well, state machines are complex, but in the simplest form, right? It's just some system of defining the different states that a thing can be in and the rules about how they can transition between those states and what to do when they do. Yeah, it's like the rules and also the process for transitioning.
00:28:11
Speaker
And event sourcing is kind of a similar concept. Maybe they are. They're kind of overlapping concepts. But I'm just going to take one second, and Daniel, then maybe correct me if I get any of this wrong. But let's just put the basic terminology together so that then we can kind of think about how we might be able to solve this.
00:28:37
Speaker
using these ideas. So like in event sourcing, we have events and projectors and something called an aggregate root, which we don't really have to think about more than the aggregate root is the thing that events happen to. And so we have this, what's it called, chain offer?
00:29:02
Speaker
or an inquiry. An inquiry, right? So the aggregate root is the inquiry. Events happen to the inquiry, and then you have projectors that take the event and project state into your system, right? So when a created event happens to the inquiry, that is gonna set the status on your inquiry model to create it.
00:29:30
Speaker
And when an offer accepted event happens, the projector will project the accepted status onto your model. And if in the case of your business logic, you need to know the accepted at timestamp very regularly, you can choose to add an accepted at column to your model and have your projector project that
00:29:58
Speaker
timestamp to the model when an accepted event happens. But for the statuses that you don't really care about the timestamps for, you don't need to store them on your model. Because you have that data in your event stream, you can always go back and grab it. You can always add a new projector later that projects that data if you need it.
00:30:22
Speaker
So the data is there, but it's not, your model is sort of just memoizing the data or caching the data for the current state of the stream of events. Does that seem like a clear enough picture of sort of formal event sourcing?
00:30:43
Speaker
Yeah, I would, I think that you leaned away from aggregates as like a thing that we don't need to talk about. I think aggregates are, I would say we don't really need to talk about projectors. I think aggregates are the most important thing to talk about, which is like the idea of building up a state from the things that have happened. So being able to answer the question like, what is the situation right now based on what has happened in the past?
00:31:10
Speaker
to me is like the most important thing that you can do with event sourcing.
00:31:15
Speaker
Sure. I guess in my mind, I think of aggregates as the implementation of doing that internal to your event sourcing logic, and then projectors are the way to get the results of that work out into the rest of your app. And so while I agree that understanding and implementing aggregates
00:31:41
Speaker
is really, really fundamental to getting event sourcing right. I think from the outside world, all I really care about is when I do, when I trigger a new and accepted event, I want my model to have an accepted status and I want that to just always work and like throw an error if that's not allowed to happen. Yeah. I mean, I think that that is, uh,
00:32:12
Speaker
that is a way to do it, right? I think there's also like a world where maybe I want totally separate models for the accepted and rejected ones, right? Maybe I want like one table that's full of accepted events because like there I care about a lot more information, right? Like there's like in those cases I care about like
00:32:31
Speaker
having a ship date and maybe a shipment ID and all these other things that might only relate to something that I've accepted. Whereas if it's declined, maybe all I care about is what date did we decline it and who submitted it? Because there's never going to be a shipment, so why have an empty shipment ID column?
00:32:52
Speaker
So there are cases where like I think a single aggregate might project into multiple different models or, uh, might project totally differently than just sort of like a model with a status column. Sure. So I guess my question is, is, is there like an in-between somewhere where
00:33:17
Speaker
you know for for for chains app like if he doesn't want to buy in you know all together to event sourcing and he doesn't want to let's imagine that he doesn't want to have to understand what an aggregate route is or the rules around like
00:33:36
Speaker
what can happen in your aggregate and why your aggregate can't understand that models exist. All the rules that are good to know when you're implementing event sourcing. Is there a world where you can just pull on some sort of trait onto a model and
00:34:03
Speaker
Under the hood, it could use formal event sourcing concepts so that you can sort of peel that back later and get more sophisticated, but you could kind of solve the problem that we're talking about of, I want to have a status column. I want to understand some metadata around those statuses, but I don't want to get too fancy.
00:34:29
Speaker
So I could imagine something along those lines. I think that. I'm a little bit. I'm a little bit event supremacist here in that like I think that you're.
00:34:44
Speaker
What you're going, you're, I can think of all of the ways where this is going to make you wish that you had just done event sourcing in the first place, right? Like you're going to try on this half, this half measure. Um, and then you're going to run into its limitations and be like, ah, why didn't I just do this the other way? Right. But I can imagine a situation where like.
00:35:06
Speaker
you know, maybe you create these like sort of action methods that accept some metadata, right? So you could just have like action and apply action methods. So like, cancel offer and apply canceled offer methods that you could just potentially stick on your model. And they would each accept some metadata, right? And then
00:35:34
Speaker
I could imagine that working. And so the main thing that you avoid there is having to have actual event objects, right? And so you could just, instead of having objects, you could have like some sort of structured data
00:35:59
Speaker
structure that you're passing around that isn't actually reflected by a class and a file that exists in the file system. And instead of having a file that represents an aggregate, you could just have like some random methods on your model that maybe you demarcate with attributes or something.
00:36:18
Speaker
To say like hey these are ones for dealing with events so like you could you could take all of the bits of event sourcing and sort of shoehorn them into existing files in a larval app. I think that feels like the worst of both that's what I'm saying I kind of think all of this feels like the worst of both worlds like.
00:36:40
Speaker
I think the idea of dedicated files and classes for events that are meant to capture a moment in time and like something happening is quite appealing. I think that what I struggle with and again, I'm sort of looking into it because I think this implementation could be really good for something I'm working on right now.
00:37:03
Speaker
The thing that's tough for me is like, where are the edges? And I think Daniel, before you said that you want to sort of go all in because the context switching can be burdensome. And for me, if I don't want to go all in, but I do want to implement this,
00:37:20
Speaker
Is the juice worth the squeeze in that scenario? So I would not say you want to go all in. The thing I was saying is that I think that the reason it feels complicated is because of the context switching, right? And it's like an ethos, right? I mean, it's like you're approaching these problems with a different mindset.
00:37:40
Speaker
Sure, but there are parts of apps that I do not think should be event source, like authentication. I don't think authentication should be event source. I think users should have a password hash column and an email address column, and then they should type the username and password into a form and submit it and then get logged in. And I don't think that we should have a
00:38:04
Speaker
user logged in event that then projects their session hash or something crazy like that. I think that for the most part, object-oriented programming is good. I know there's people who disagree, but object-oriented programming is fine. There was a time back when all of the architectural concepts that
00:38:29
Speaker
led to Rails and Laravel were being bandied about in the first place by people like Martin Fowler and all these architecture wizards, where domain-driven design, this concept, was the idea of three types of models, which are models where you care about their identity, that have an ID, and then
00:38:56
Speaker
models where you don't care about their identity, where they're basically like fungible and any model could be any other model. And then like events, those are like the three types of models, you know, in domain-driven design. And like somewhere along the line, like we just kind of got to this place where we dropped events. But I think that these things actually do live really nicely alongside each other. And I think that like
00:39:22
Speaker
things like users where like it's important to distinguish one user from another user, uh, are like identity based models and like, shouldn't be sort of brought into like an event paradigm. Yeah. The, the question just becomes how do you, if you're,
00:39:44
Speaker
if you're building an app that where, you know, this idea, the status column is sort of the first place where you're really wanting an evented approach, you know, is there, is there a way to, to implement some of these concepts without introducing quite so much complexity as can you be more explicit about like what complexity you think you can get rid of?
Simplifying Event Sourcing
00:40:14
Speaker
Well, okay, yeah, here, I'll just imagine something. Like I can imagine, I agree with Shane, that...
00:40:21
Speaker
Having objects that represent the events makes a lot of sense, right? And I think, and as soon as I say this out loud, I may change my mind, but I'm gonna go down this path at least. I think that there's maybe a world where I have a accepted event
00:40:47
Speaker
or accept, you know, except, um, I can't, I think of it, I keep on losing this word inquiry, except inquiry, accepted inquiry is probably what I would call it past or inquiry accepted inquiry accepted. Sure. Um, right. And in the constructor, the inquiry accepted event takes all the metadata that that event is going to need. Right.
00:41:15
Speaker
And I know that from a, you know, event sourcing purism perspective, this is probably terrible, but what if that event just had the projection logic right there, right? Here's the event. And then there's also a method on the event that accepts an inquiry model and applies the event data to that model.
00:41:43
Speaker
And that's the whole thing, right? So now you have, you just dispatch these events. Like events that project themselves, basically. They project themselves to, to whatever models you need them to project themselves to. And, um,
00:42:01
Speaker
under the hood, in an ideal world, there would be a proper event sourcing implementation that this is sort of like a layer on top of, so that the moment you need to do something that's sort of outside of the scope, you need multiple projectors, you need reactors, you need, I don't know,
00:42:22
Speaker
You need to make a decision about whether or not to project some data based on other events that have already happened, right? You can kind of peel this layer away when you need it. But in the beginning, I could just have an inquiry accepted event that accepts the inquiry and the metadata and then just applies and then just updates the inquiry object to have the status accepted.
00:42:51
Speaker
And that's the whole thing. Yeah, I think that's fine. I don't think there's anything wrong with that. Um, that feels good to me. Yeah. I think that basically like, I think the requiring a projector to be a separate file than an event is, you know, I think if you build, if you build something more than like maybe this one state machine, right? You will quickly.
00:43:22
Speaker
want separate projectors. But I do think that we have a really, really big event sourcing app, this game that we built. And we have one projector file. We have a file that contains all projection logic in the entire app. And it's a giant 800-line file or whatever. But it's one file. So I don't think that there's
00:43:50
Speaker
a right or wrong place to put projection logic. I think that having it on the event itself is a little bit uncomfortable because
00:44:04
Speaker
it removes the logical path to like an event fires, then we make decisions, then we project, right? But I think as like a starting point, I think saying that like events have an optional project function on them and that if an event has a project function, then we fire that project function. I think that's fine.
00:44:32
Speaker
Um, right. And then you could have like, uh, um, you know, you could optionally implement a more complex projector if you needed to. I'm just thinking about all you're doing is projecting to one model. I'm thinking about the, the user story for like, okay, I implemented this, I implemented things using this kind of like one file per event.
00:45:00
Speaker
mechanism where like the event and the projector are the same file, right? And then now I have some additional complexity where I need to make like historically informed decisions before projecting data. So therefore I need to start extracting a separate projector. I'm trying to see if that would, like this is what I'm trying to wrap my head around is like
00:45:24
Speaker
Is there like a easy story to tell about how you transition from like the naive implementation to like the more fully, uh, fully fleshed out event sourcing implementation? Well, that, I mean, that decision, that, that, um, situation, it's not even necessarily that you need to, um, extract out a new projector. It's that you need an aggregate route at that point. Right. And that's what I'm saying is like,
00:45:51
Speaker
Is there a world where I could implement an aggregate route, having a history of existing events that I had created in sort of this other implementation and then re theoretically like replay all those events and like have everything be okay. You know,
00:46:09
Speaker
I can imagine. I mean, I can imagine doing it. Yeah. Yeah. This is getting into, I mean, I worry that I'm going to listen back to this and be like, God, no one's going to have any idea what we're talking about here. I have no idea what you're talking about.
00:46:27
Speaker
Okay, a little bit a little bit so I mean I think what Daniel is getting at is Sometimes you have you need to make decisions about events
00:46:41
Speaker
based on the history of the events that have fired thus far. Right. And like, I hate to use this example, but it's the one that comes to mind and it's the one that everyone uses. Wait, I have an example. I have an example of domain specific. So like imagine, imagine, yeah, that's not banking. Um, so imagine you get an inquiry submitted from a user, right? And in the past, um, this user's, uh,
00:47:07
Speaker
I don't know, payments have been declined or like when you receive the cards, you found out they were fake, right? And so you have reason to distrust this user, right? So you probably don't want to immediately
00:47:21
Speaker
project things when you receive, like you probably don't want to immediately put them into like a order acceptance flow when the thing comes in, you might want to immediately reject them because you think they're fraudulent. Right. And so this is an instance where like these events that happened in the past, which is like, okay, maybe I won three of their, if three of their inquiries have been rejected, then we're going to reject all of their inquiries in the future. Right.
00:47:50
Speaker
That's the type of thing where like an event comes in and before we make database projections, we first want to make decisions based on previous events. So you're saying if you were to implement this in a more simplistic fashion where you are, as you described, making the projection just based on the snapshot of information, which is not enough to
00:48:13
Speaker
flag something for fraud as you're describing or something like that. Then if you later are like, I want this more robust implementation with this kind of projection that's happening based on the full sequence as you're describing. I guess the question becomes that you're asking is, is there like a clean path from one to the other? To migrate from the sort of more simple implementation to the more. And like, how do you resolve that? Like, do you have to
00:48:43
Speaker
Yeah. So I think the thing Chris is describing is actually fine. So I think what you could do is basically take those project methods that you had on your events, pull them out to a separate file where they actually become basically event listeners that listen for an event and then do something.
00:49:04
Speaker
And then you sort of put an aggregate in between those, for lack of a better word. The issue is that normally what you would do is you would listen to one event and then project another event. So there normally would be like a first event and a second event.
00:49:22
Speaker
Um, so anyway, that's, I think, let me, I want to make, I, maybe I'm wrong here, but I think there's a distinction also because in, in what you're describing, you're talking about making decisions. Like if we're thinking about the inquiry as the thing that events are happening on, right?
00:49:48
Speaker
You're making decisions about that inquiry based on events that happened to other things. Yes. There'd be different, there would be different aggregates here. So there would be an account aggregate and an inquiry aggregate. Right. What, so I think that there's another example, um,
00:50:11
Speaker
where it's all within the same thing, the same aggregate. And that's why I was going to use the banking example. If you have an account and there's a money deposited event of $100 and then a money withdrawn event of $100 and the user tries to do another withdrawal,
00:50:39
Speaker
It needs to understand that the current balance of the account is zero and not allow the money withdrawn event to be fired. And in the naive approach that I was talking about where the events just project themselves, those events
00:51:01
Speaker
you know, firing a money withdrawn event would have no understanding of the, the balance of the account. All it would have is, you know, the, the mechanism to cause the account balance to decrease. Right. And so in that moment, you do need to have an aggregate that can hold event data over time.
00:51:29
Speaker
Is that always true? Okay, you're describing this withdrawal situation where you need to have the funds available and that depends on the history. But when it comes to a status update, similarly, you need to know that it's a valid status.
00:51:46
Speaker
Yeah. I mean, you said there is like, yeah. So the naive move would be to basically just like query the account model and be like, if account arrow balance is greater than event arrow withdrawal amount, then we're good. Otherwise, you know, do something else. Um, the issue there is that you're embedding yourself even further into
00:52:17
Speaker
a non aggregate implementation, which makes your transition to your eventual transition to aggregates even more work. Right. Yeah. That's what I was getting at too is like in event sourcing,
00:52:33
Speaker
you really want your aggregates and projectors to have no understanding of the underlying data that they're... Well, the projectors do, but you don't want the logic around your events to understand the eventual models that you're projecting data into.
00:52:53
Speaker
Because if you keep that separation, then if you ever need to change something or rerun your event stream, you just re-project the events and you build up state without any problem. Whereas if your event logic relies on the current state of the projected data,
00:53:15
Speaker
you lose the ability to rerun events, you lose the ability to add new projectors and project old data to new places. You lose a lot of the flexibility, the power of event sourcing. So I think that the moment that you need to hold
00:53:35
Speaker
aggregated event data to make event logic decisions, you need an aggregate, right? You need to have a place to store that. And so like in my sort of imaginary world, I could see just having like a
00:53:51
Speaker
interface that you could add to a model that's like get aggregate and if your system encounters a model that implements that interface instead of using the model it uses the aggregate and that way
00:54:08
Speaker
When you need to start aggregating data, you can, but you still don't need it until you need it. Maybe there's a world where like, maybe that introduces some bad habits, but there are some other concepts that exist. So like, um, like spots, these package includes this concept like called event queries, which are basically like they're like aggregates, but less, uh,
00:54:36
Speaker
less declarative. So they're basically like, okay, well, what if I just wrote a good query that told you what events should be applied to this aggregate, and then use those events to build up a state over time, right? So rather than having an event be assigned to an aggregate, when you run it,
00:54:57
Speaker
right? Or when you dispatch it where you say like, I'm dispatching this event onto this aggregate. Instead, you can sort of have like a post facto. Well, I'm just gonna like build an aggregate up from historical events instead.
00:55:14
Speaker
Um, which is useful in cases where you need to like do secondary state building with aggregate or with events that were actually assigned to different aggregates. Um, right. But I don't know. So I'm thinking maybe the naive implementation, if you want to like avoid aggregates and you want to be able to like transition to aggregates with old data that wasn't assigned to aggregates when it was created.
00:55:41
Speaker
that maybe there's some sort of query-based solution that you could build or whatever. Does that make any sense, Shane? Yeah, it does. I don't know if we're any closer to me implementing this tomorrow.
Event Sourcing in Rental Systems
00:56:05
Speaker
I would love to know what the problem you want to solve is. I think it's a good one, and it relates to what you were describing before, Chris, with the banking example. But we have a rental program. This is for the online game, Magic Online. We don't have to get too deep into the nitty gritty here, but we let our customers
00:56:31
Speaker
have a rental limit. And they can then take out whatever cards they want to play with. And we have to keep track of what cards they've taken and deduct that from their balance. And then when they return them, they free up that balance to be able to take out other cards. And so keeping tabs on that is
00:56:56
Speaker
having discussed it a bit with Chris before and thought about it, because I have this opportunity to build all of this from scratch. I'm in the process of rebuilding everything. And it struck me that event sourcing could be great as a way of knowing what cards are on loan and what customer's balance is at any given time and being able to make use of that implementation pattern.
00:57:22
Speaker
Well, another important distinction is that card values change regularly. So having to have an understanding of what was the value of the card when they rented it and what is the value of the card when they're returning it.
00:57:39
Speaker
Well, needing to understand what was the value of the card when they rented it, because that's the amount that has to be returned, regardless of what the value is now. But also being able to do some sort of secondary analytics on how much our card value is changing between the time when they're rented and returned. Totally. Yeah, there's a lot of interesting information there, as well as utilization rates for our customers.
00:58:07
Speaker
being able to spot problematic patterns for people that are abusing the system that we have. So the story that can be told by having things in this event-driven thing where you can do these projections is super interesting for that reason.
00:58:26
Speaker
Yeah, that makes sense. Yeah, I think that in that case, right, like the big thing that you care, I mean, it might be useful to have like a card aggregate or something as well for other reasons, but like
00:58:42
Speaker
The big thing that you care about is like a user, like lending account aggregate, right? So you want to aggregate information about like a given user's lending account, right? And like what the rental account or whatever, right? And basically like, what is their current balance, right? Which is like their, I guess their borrowing potential or something. And so like,
00:59:09
Speaker
how much are they currently allowed to leverage essentially versus, uh, and then like all of the events that change that, which is like taking things out, returning things, you know, stuff of that nature. And, you know, I don't exactly know what the attributes that are, that you're updating are, whether there's like a, just a number, which represents their, you know, their available credit or whatever. Um,
00:59:38
Speaker
And that that number like goes up and down or whether it's more complex and it's like, it's a difference. It's a combination of like number of cards and value of cards. No, no, it's, it's, I mean, it's very simple. It'll just be, you take out the cards. We, what are the, you know, take a snapshot of their value at that moment and that needs to be deducted.
00:59:57
Speaker
So if your balance is 500 and you take out a bunch of cards and they're worth in total 400, the in-game currency, then your balance at that point would be 100. And then maybe, and this is again, sort of like a step forward, traditionally in the old web app, which was built,
01:00:14
Speaker
many years ago. This system came sort of late in the game. We were originally just sort of traditional e-commerce. And so when we introduced this rental program, you would place an order for the cards, let's say it's, you know, 100 cards or something. And when you were ready to return,
01:00:35
Speaker
you had to return the order. There was a sort of tight coupling between the cards you had taken and this one sort of order resource. And so that's another thing that's a part of the system would be great is you can place this order, that order, maybe you've read the same card. Yeah, so your one order might have four cards in it and then you might send back two of those cards and two other cards from a separate order at the same time. And some of those orders might have the same exact card
01:01:03
Speaker
at different values because they're at different points in time. And so then you want to return them at some point. So we just need a way to sort of reconcile all of that. That's so interesting. This is a really, really, I think, really good case for event sourcing. And I'll happily hang out talking about event sourcing for a while. I also could see saying, like, do we want to get back to statuses?
01:01:33
Speaker
I'm, I'm good either way. I just wanted to at least put that out there. Yeah. I mean this, in this case, like a balance.
01:01:44
Speaker
You know, it's, statuses are more like, uh, how do you say statuses are more like balances are continuous, right? From zero to infinity and everywhere along the way. Um, whereas statuses are like, there's like five of them or whatever. And we like rotate through them depending on like what the status is.
01:02:04
Speaker
Um, but I think essentially balances and statuses are the same problem, which is that like actions happen and I'm trying to reduce that down to a single piece of data, you know. Yeah. I mean, that was like a thing that I am embarrassed to say took me so long to wrap my head around is like when, when I'm using eloquent and I call some,
01:02:31
Speaker
under the hood, that's calling an aggregate method. And it took me so long to wrap my head around the fact that in event sourcing, this term aggregate root is the same as a sum or an average or a median, like an aggregate function in a database or in anything, that they're all the same. The aggregate root is just
01:02:59
Speaker
aggregating all the things into one thing. Yeah. Just applying one, applying a function on top of another function on top of another function on top of another function until the state is up to date. Right. Right. And, and status columns are the same. You're aggregating a bunch of events, a bunch of times to one single status, right? The current status. Yeah. I agree. Um, and I think that.
01:03:30
Speaker
I mean, yeah, I don't know that we are, I don't know that we're going to come up with any kind of like solution here, except that like, I think the solution is like, you know, get more comfortable with verbs, you know? And I do think there are in between things that people can do before like installing a package about it. You know, I think that they should maybe experiment with like, well, why don't you create a table in your database that is a verb?
From Status Columns to Verb-Focused Tables
01:03:57
Speaker
You know, just like any verb.
01:04:00
Speaker
um, but create a table that doesn't describe a noun that describes a verb instead. Um, and then see what the logic that you have to write in order to incorporate those verbs are, right? So I think like, you know, an example is like, you know, like a cancellations table or something, right? Like that, right? Like it's when we, we, you know, noun ourselves a verb. Um, you know, but I think that like,
01:04:30
Speaker
creating a table that is made up of times when people did a certain action and then writing the logic that way rather than
01:04:45
Speaker
keeping the actions imaginary and writing the effects in the database. I think that is a really good training wheels for this mental model of just deal with verbs more and deal with nouns less and see how it feels and see the types of queries you write. I would say that would be my baby steps.
01:05:11
Speaker
I do really like that insight that this problem that we're kind of touching on, and specifically the problem of statuses, is really trying to shoehorn a noun into a place where you're dealing with verbs. A status is describing basically the most recent thing that happened.
01:05:40
Speaker
and like trying to unwind that a little bit makes a lot of sense. I'm still not opposed to the idea of like, if you have a status column, just make each status or each way that something can get into a status, an event that gets stored in the table.
01:06:07
Speaker
And then just have that event, have like a handle method or whatever that just applies that event to the model itself. And like, even, even if you want to, right, if you want to like, not, if you want to like be, I'm not gonna like, I insist I won't event source this whole app. Right. So like, I'm going to draw a hard wall. I think even making a table called like,
01:06:32
Speaker
inquiry events, right? And so it's a dedicated table that's only full of things that happen to an inquiry, right? And maybe it has a type column or it's like, you know, a magic string that describes the type of events that happened, right? And then it has a bunch of other columns for metadata, right? Or it has like a JSON column that can accept some magic payload of metadata.
01:07:00
Speaker
I think even just building that column and sort of do it yourself when something happens, store an event in that column, and then create an accessor on your inquiry model that just runs a query of all of the events that happened to this thing that then
01:07:22
Speaker
aggregates them in real time and generates your status or something, you know, I think that even add a like on created listener to that table. I mean, you could make like an observer or some shit. Yeah, that applies the event to
01:07:38
Speaker
Cause I think in the end, I mean, the thing that I am very adamant about is a solution where you don't have a status column on the inquiries table. It's going to be bad to use like in day to day work because you want to write a query of like all of the ones in this current status, right? For like a triage view or something.
01:08:01
Speaker
Yeah, and I don't want to do some sort of sub-query where I'm querying on the events table and finding the latest event. Of course, yeah. You want to project that data. You just also want to be able to delete the entire column of statuses and recalculate it at any given time. Yeah, I do think that the metadata is a huge piece. And with this very simple implementation, you really get like the timestamp. And then to your point, Daniel, you have to like awkwardly define
01:08:30
Speaker
very specific database fields. And so yeah, like maybe a separate table, you could call it inquiry events, or it could be inquiry statuses, it could be even more specific. And it's just like, okay, here's the status update. And here's the metadata, if any, and the timestamp.
01:08:49
Speaker
I wouldn't go statuses because I would go status transitions. Maybe. Sure. Cause like you want the whole point is you want to capture not the status, but the right thing that made the status change.
01:09:03
Speaker
because you may like inquiries isn't as good an example, but in, in the system that we have in place where we're, we're, um, assigning, assigning work to inspectors and assigning jobs to inspectors. Um, you know, those go through a pretty typical, you know, created, offered, assigned, scheduled completed kind of, um,
01:09:27
Speaker
set of statuses. But if a job needs to be assigned to a different inspector, it would be great to have a reassigned event that not only changes the status, but also nulls out some other columns, nulls out maybe some of the timestamps that we're storing because we need to query on those timestamps.
01:09:57
Speaker
but something can transition from scheduled to assigned where really it was rolled all the way back to new and then reassigned all in the same moment. And so being able to represent that whole suite of changes as a single event would be way, way nicer to work with.
01:10:24
Speaker
That's a really good point. I mean, that kind of reversion is a standard thing that I typically have as part of this implementation. Like people make mistakes, they click the button, but like on the admin side, most often to advance something and then they say, Oh, I want to undo that.
01:10:40
Speaker
And so they revert or in your, what you're describing Chris, where you need to sort of reset. And, you know, there's obviously very simple ways to handle that. But oftentimes that means unless you're doing like auditing and stuff, like you're kind of losing a bunch of data, you're like wiping out a bunch of timestamps. And so what you're describing could be nice as a way to capture that moment.
01:11:06
Speaker
Right. Maybe they accepted an offer and then like you realized, Oh, this card that we thought was worth 40 cents is actually worth $200. Like you have to re issue a new offer to them. Cause you made a mistake and like you want to, yeah, you want to have knowledge of that, that fact somewhere in your system.
01:11:31
Speaker
There's, um, this is just sort of unrelated to what you were just saying, but something I've been thinking about recently is that like, get is an event source system, right?
Git as an Event-Sourced System
01:11:42
Speaker
So like, if you like use get and like do commits and, uh, rebases and merges and, you know, revert commits and, you know, do all of this stuff, like,
01:11:54
Speaker
What you're doing is basically deriving the state of your code base from the aggregate of all of the events that have happened over time. And you can delete your .get folder, and the code base is still in the same state it was before. And so there's something cool about the fact that
01:12:21
Speaker
you know, every like, every day, we kind of interact with something that's like fully event sourced. And like, we don't think about it as such all the time. And I do think that like, Git has a lot of lessons to teach us about event sourcing, right? And so like, if you just look at
01:12:40
Speaker
Like get one of the biggest things that get does is tell you whether a certain chain of events is compatible with another chain of events. And like, whether you can like combine those two chains of events or not, or if there are conflicts between them and like, you would need to resolve those conflicts in order to merge them. Right. And so like, those are the sorts of things I'm starting to think about now. Uh, we're talking about building some, some games that are more like real timey.
01:13:06
Speaker
meaning that like we might be building up event queues, like each player might be building up their own event queues that sort of get pushed up to the server in batches every once in a while. And so there's some stuff to like make sure that the batch of events that you get from player one is compatible with the batch of events that you get from player two before you sort of zip them up and apply them. And I think like git
01:13:32
Speaker
implementation of that is like really strong. And so thinking about like event sourcing as like deriving the state of your application from Git and like people doing commits instead of doing events is like a really interesting thought, right? And so when you talked about this, sorry, the reason I thought of it is when you were talking about like people making a mistake on the admin side and then needing to like revert those changes, right?
01:13:58
Speaker
Like earlier yesterday, I actually merged a PR in a client's repo, right? That like was not ready to merge. And so I had to like quickly revert it. Well, like GitHub gives you a button that it just says revert and it just opens another branch. And like opens like a PR creation dialogue to just like press two buttons and have another pull request up that undoes all of the changes that you just did.
01:14:25
Speaker
But it's kind of this forward-only, in the same way that people don't use down methods on their migrations or whatever, it's kind of this forward-only, except for Chris. It's this forward-only thing where it's like if you want to undo something, you just do more stuff to undo it, right?
01:14:43
Speaker
Anyway, these are all the thoughts that I'm just like swimming in all the time. It's just, I feel like I'm like, uh, seeing the, seeing the universe all the time. We're just like, Oh my God, there's events. Everything's events. Well, yeah, there's like the whole, what the heck is, I always forget this acronym CR CQRS CQRS.
CQRS and Event Sourcing Challenges
01:15:05
Speaker
Yeah. Which stands for hold on. I conflict. Command query responsibility segregation.
01:15:13
Speaker
CQRS is an architectural pattern for separating reading data, a query from writing data, a command. CQRS derives from command and query separation. Commands mutate state and are approximately equivalent to method invocation on aggregate roots or entities. Queries read state but do not mutate it.
01:15:35
Speaker
But isn't there, there's like, there's, there's another similar acronym that re that describes, um, like the, the type of event that's like an event that's built to be, um, nevermind. I'm not, it's not going to come to me. Okay. Yeah. That's a, that it's a whole, that's a whole other piece is like,
01:16:04
Speaker
If you get too far down this rabbit hole, then you have to start thinking about conflict resolution and asynchronous event concerns. If you ever tried to implement offline mode in an app or something like that, you need to
01:16:22
Speaker
resolve all the actions that were taken so this is a separate clients like yeah it's just talking to John I was just talking to John my thunk partner about this and he's also like the game design guy right and he has this card game that he built that
01:16:40
Speaker
we're probably going to build like a React Native implementation of it. And we're like, we need to build like a React Native, like a React event sourcing framework, really. Because like what we're doing is like, it's all events, right? A player does an event and does another event and whatever. And I was telling him about like, so in Hearthstone, when you, like if you switch applications in the middle of a turn and then come back,
01:17:05
Speaker
it plays all of the events back to you at like 3x speed to like catch you up to the current moment, right? And I was like, oh, well that like shows me the technical implementation that they actually do have a queue of events going on, right? Because if they didn't, they wouldn't be able to do that. So they have a queue of events that happen and they're just like playing them one after the other in order to like make things happen.
01:17:28
Speaker
So anyway, it's just stuff like this where I'm like, everywhere I look, I'm like, events. I see it. I see it. There's events there. These things that look like static states are actually derived from events.
01:17:39
Speaker
Yeah, especially games like Hearthstone where or magic, I mean, is a perfect example of like you've got all these very complex rules about like phases and priority and like, you know, when when two different things that sort of kind of happen at the same time happen, like which one it does matter which one happens first. Yeah. Yeah. Like in game design, I imagine that this comes up all the time.
01:18:09
Speaker
It does. Yeah. Well, I really like this idea of even if it's not a package, even if it's just like a, you know, a custom sort of bespoke implementation, this idea of just having a table
01:18:30
Speaker
of events that happened to your thing that needs statuses, right? Yeah, because then if you do decide to go full event sourcing, it's pretty easy to write a job that would turn that table into event events.
01:18:47
Speaker
Right. Right. So like you could back, Chris and I have experienced saying like, let's event source something that we have a ton of like 10 years of historical data for that wasn't event source, you know? Yeah. So like if we can do that, you could definitely turn that table into events. Right. Something that's already basically event source. Right. And then yeah, like each one of those models, each one of those events,
01:19:12
Speaker
you could do something pretty simple, like just have a callback, like a static callback inside of the saved or created event listener. And in that static callback, it just like
01:19:29
Speaker
runs a switch statement on what is the type of event, right? You like have an enum or something like that, trying to just keep it as simple as possible. So you have an enum or you just have some constants on the model itself. I don't care, right?
01:19:46
Speaker
So my feeling you'd happen to you know, I think I think this is actually like the way to go So I think like if you're like on the fence about event sourcing build your own implementation based on like some of the concepts of event sourcing but like
01:20:01
Speaker
Fuck all of the specific words like aggregate roots and projectors and reactors and like all of that. And don't use a package because then you're going to have to learn all those words in order to do anything, right? Like just start building the implementation using verbs and like you will get to a point where you're like, oh man, it would be really cool if I could like aggregate all of these events to derive a specific state and be like, okay, cool. You have now happened upon
01:20:27
Speaker
the concept of an aggregate route, right? Like you've happened upon the need for which this thing was created. And then at that point, you either build your own event sourcing package, which is what we're doing, or you just don't and just like keep it in house, or you go pull in event sauce or Spotsy's event package or ours, whenever the fuck we get it out, you know? But, you know, eventually maybe you want to like fall into some sort of a framework.
01:20:57
Speaker
But I think at first, just build some actions and see how it feels to play with them. I accept that that's a great suggestion for Shane. It doesn't satisfy my desire to just say, oh, I have a solution for this. Whenever I need a status column here on out, I'm going to pull in this package that I wrote that just solves it for me.
01:21:27
Speaker
Sure. But I can listen to that for now. I suppose I'll probably end up hacking on something after this because I do think that there's a war. I do like, I mean, I am going to take this self projecting events concept.
01:21:46
Speaker
and pull it into our package, I think, because like, I think as like, uh, cause I'm always thinking about things in terms of the, like the learning process, you know, and like the onboarding process for a new user. And I think for a non eventy person, like
01:22:06
Speaker
the onboarding process to events, if you just say like, Hey, here's an event and here's how it mutates the database. It's all in one file, like go to town. Like, yeah, I think that is a cool onboarding process. I do want to do some work on like, what is the upgrade path to aggregates, you know? Um, but I think that like,
01:22:30
Speaker
I do think that any non aggregate event, there's nothing preventing it from being a single file. Yeah. And, and I think that there's pretty easily, I think that there is a world where you can just inject an aggregate into that scenario as well. If you need it, I don't think, I think that you could.
01:22:58
Speaker
I think that you could never write a projector. I bet you could get away with only having self projecting events for a while and then introducing aggregates second and introducing a projector third. I don't think that you need to extract out to a projector. I agree. I think the projectors being like the first, like projectors being the first concept you learn about an event sourcing is like really ass backwards.
01:23:26
Speaker
Yeah, and like maybe you force that separation by letting the events from the execution perspective, I can just pass in my inquiry model into the event, but the package under the hood basically says like, hey, if this event doesn't have an aggregate,
01:23:51
Speaker
just call to array. If this model doesn't have an aggregate, just call to array and pass that array in. And that way, from your projection perspective, you're not dealing with the event itself.
01:24:06
Speaker
And you're not dealing with the model itself. You're just dealing with data. And that way from the very beginning, like you could even make it so that your system did like array accept model get key so that the thing that goes into the event itself doesn't have the ID of the model. So like your code is forced to not think about the database at that moment. I think what you're describing is a transducer.
01:24:37
Speaker
I'm glad that we got there because it is. It's all transducers in the end. In the end, it's all transducers all the way down. Transducers over time.
01:24:52
Speaker
Shane just Well in my current side project I am writing a rust audio editor and so The idea of like processing like an ever-growing stream of bits over time and doing complex branching things with them Has caused me to wonder whether I actually just want transducers. Hey, you might need them in Austin. It could be great
01:25:22
Speaker
Well, I feel like this is a good place to stop.
01:25:26
Speaker
Yep. It's been a flash. Hey, this has been fun. Yeah. This is one. I, I've been thinking about this for so long. I'm, I'm not, I'm not sure if tomorrow I'm going to wake up and be like, all right, we need to get on and have a whole other discussion or, or we're always down to spend 90 minutes talking about my psychotic fever dreams of events. Well, this has been great. Yeah. I'm going to, I'm going to be using some of this to see what happens with it.
01:25:55
Speaker
Shane, I put a link in the chat that you should look at. It's a, it's an event flow that we just did for a client, uh, for their orders system. Um, and it's kind of similar to your system. So maybe take a look, see if there's anything to glean. I love it. And I'll just be putting your proprietary stuff in the show notes. Don't worry.
01:26:18
Speaker
I will. All right. Uh, we good. Anything else? All right. Here we go. Hell yeah. I just triggered the outro event. Yeah.