Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Ep. 3: Referencing specific database records in your code
248 Plays1 year ago
Season 1 continues with a discussion of how to deal with special database records that need to be referenced directly in code.
We've all been there before: you've got a specific vendor that you need to write a custom command for, or a certain category that needs special handling, so you either hard-code the ID or slug and shudder slightly before moving on with your life. In this episode, we imagine a better—perhaps the best, even—way!
Recommended
Transcript
Introduction to 'Over-engineered' and Season Themes
00:00:07
Speaker
All right, welcome back to another episode of over-engineered, the podcast where we ask the very important question, what's the absolute best way to do things that don't really matter? I'm Chris Morel, and I have my friends Bogdan Carchenko and Skylar Katz back with me today. Continuing on the theme of what we're calling season one, but with a little different twist. Welcome back guys.
00:00:32
Speaker
Hello. Hello, everybody. And I will say, Chris, it's not exactly useless. You know, this does have a point. I think we're doing the Lord's work here. Yeah. I mean, it's been, it's been really fun.
Excitement for New Topics: Data Migrations and Database Management
00:00:48
Speaker
And I, I do feel like at least for myself, I've gotten some clarity on the way to approach certain things. For sure. For sure. Yeah. I'm super excited to talk about the things that we have lined up for us today. So we'll see.
Handling Database Records: Case Study of PlayStation 5 Stock Tracking
00:01:02
Speaker
Yeah, so this is like, this is kind of following on the topic of, you know, the last couple episodes were data migrations or migrations, schema migrations and how you manage your data around there. And this is sort of related in that what we wanted to talk about today is you've got data in your database, you've got records in your database that you need to reference and code and how you do that. So to sort of set the stage,
00:01:33
Speaker
Imagine that you're that there was this layer cast series that I was watching about building a stock tracker. And so the idea was, you know, you want to track the PlayStation five stock at a bunch of different vendors.
00:01:47
Speaker
And so we need a record in a vendor's table for, for example, Best Buy or maybe Amazon, or I'm trying to think, I was gonna say Circuit City, but I know Circuit City hasn't existed for many years. And we need, we're also gonna write, you know, custom commands that will scrape those sites for stock and they need to be specific to that individual vendor because each site operates differently.
00:02:17
Speaker
So the code in our code base needs to understand
00:02:23
Speaker
specific records in the database, right? And there are a bunch of different ways that we've approached it. And each time I've always been like, this feels not great, but it's fine. It works. And that's the whole, that's the whole theme of this show. So I thought it'd be a perfect thing to pick up. So I guess maybe first let's just, just run through some of the approaches that we have done and maybe what we don't like about them.
Referencing Database Records: Approaches and Challenges
00:02:52
Speaker
Yeah, that sounds great.
00:02:54
Speaker
I think, uh, let's see, I've, I've done things where you're just like fine by slug and pass in the slug best, best dash by, which is okay, except that maybe somebody doesn't do it by slug and does it by name somewhere else. So then you're like fine by name, best space, by we've also. I've also done like a constants on the, on the vendor model. So you like vendor, colon, colon, best buy.
00:03:21
Speaker
Uh, to like, at least then it's, it's kind of in code and can be renamed, but it always feels gross to do that. Or like, what if, what if you don't have it in the database yet? And then, and then what happens in, uh, in the code? Yeah, that's a big one every time. I mean, I think that that's really what.
00:03:44
Speaker
always becomes the frustration for me is whatever solution you come up with is usually fine until you get to the testing story where the data doesn't necessarily exist yet.
Legacy Data Handling and Testing Challenges
00:03:54
Speaker
And that's sort of where it dovetails in with our previous conversation because maybe you have solved this problem a different way by having a perfect way to always have the data that you expect in your database, although,
00:04:13
Speaker
You know, I still am not certain that that's not a pipe dream. So I, you know, if you're in a scenario where, you know, your test, your test database always has those records in it, that's maybe not a problem, but then you're dealing with the same issues of like, do you want to seed a bunch of data that you're only going to need in certain tasks? But yeah, yeah. I mean, more recently I've used slugs.
00:04:39
Speaker
In our code base, which has to account for a lot of legacy data, we have done plenty of just hard coding, special IDs, you know, and maybe storing them in an enum. That feels super gross and also becomes a problem with testing. And I know that, you know, for example, like Sushi is a good, we talked about that a little bit, I think on the last podcast,
00:05:06
Speaker
Maybe not, but it kind of came up around this idea of seeding databases, like Sushi is just a way to have an array of content that you can query through Elquent, but I don't really feel like Sushi works here because it's a separate database
Database-Driven Solutions: Pros and Cons Debate
00:05:24
Speaker
connection. You're not gonna be able to do like, you know, where has on that relationship or, you know, it's always gonna,
00:05:34
Speaker
by design be isolated from the rest of your database. And then like the other thing that I think is a really interesting approach, I don't know if it's
00:05:46
Speaker
I don't know if it always is solving exactly the same problem, but in a bunch of places, more recently, what I've found myself reaching for is actually storing the fully qualified class name of the class that handles that record.
00:06:05
Speaker
in the database. So instead of fetching the vendor in our code, we just fetch the vendor and then instantiate the object that is responsible for handling that vendor, which, you know, in a lot of cases you can probably do because
00:06:25
Speaker
You know, for example, for our example, instead of having like a custom best by command, like a scrape best by and scrape Amazon command that you run on a schedule, you could just have a command that fetches all the vendors, instantiates the scraper for that vendor based on the class that's stored in the database, and then execute the scraper
00:06:53
Speaker
But then, you know, you're kind of back to needing to move more logic into your database. Cause then it's like, Oh, well I want to scrape Best Buy every hour and I want to scrape Amazon every 15 minutes. Right. So now do I have to go in and, you know, add a, some sort of timing info to the database so that I know how frequently each scraper needs to run.
00:07:21
Speaker
the more you move what really ought to be done in code into the database, just there's a bunch of overhead associated with that. So I think it can fall apart. Are there any other approaches that I'm missing?
00:07:36
Speaker
Yeah, I mean, so I was going to comment. I mean, this is going to piggybacking on what Skylar mentioned is just having some sort of like enum class or like a bunch of constants that kind of reference, you know, so-called slugs, right? So in this vendor example, we'll have like a Best Buy slug and maybe Amazon and Target and so on in Circuit City.
00:07:54
Speaker
And I think I've done it both ways, where you would go and say, give me this vendor by this reference enum. And you can get that model out, and then you can do whatever else you want to do with it. I think there is another step.
00:08:15
Speaker
iteration of this is to use scopes. You could just have a scope that says Best Buy because it's very specific. But you're still in the hard-coded world of things where you're passing a bunch of slugs around and everything like that.
00:08:30
Speaker
So I think ultimately what you were getting to Chris is having a database-driven solution. Because otherwise, even if you have some sort of handler, it's still referencing some sort of class that will need to get newed up and something needs to happen. But if you do have, for example, these values like you alluded to having
00:08:59
Speaker
some sort of like timing, like when should Amazon get scraped? You know, when should Best Buy get scraped? And then we don't even need to know the concept of Amazon or Best Buy or Circuit City. Like we just know that this is just data that's driven through a database. So I think that's probably the happy world, you know, somehow in my mind.
00:09:24
Speaker
Yeah, I mean, I don't know. I guess I'm just not convinced because in a lot of cases then you're, you know, you're just dealing with having to add
00:09:37
Speaker
you know, add a bunch of junk to your model that really should, I don't know, in my mind really should be code. Maybe that's not true. And obviously there are cases where, you know, if you want the scrape timing to be configurable by the end user, right? If you're like building this for a company and someone in the company needs to
00:09:59
Speaker
be able to adjust the scrape timing for some reason. Then of course you're gonna need to push that to the database anyway. And arguably you're gonna want some sort of much more configuration driven system in that case.
Configurable Scraping Strategies and Database Storage
00:10:17
Speaker
But even then it still needs to understand like how to scrape Best Buy because Best Buy is, you know, gonna need to be scraped differently than Amazon. You know, it can all,
00:10:28
Speaker
I mean, I guess you could have like some sort of wild, like ex-path based configuration in the database, but I think that's again, kind of a pipe dream, like, you can't cover every scenario.
00:10:43
Speaker
I would agree. I think like most of the time you would be dealing with like an API, right? You're not like going on their website and maybe doing some scraping and you know, maybe there's API keys involved or even if it is scraping the website, maybe you need to be logged in or something. So I totally get that there's like specific handlers for each vendor. But it's almost like well, if you have a job in the background that says like,
00:11:05
Speaker
Hey, find me all of the vendors that I need to go scrape today. You get the vendors back and then you can determine like, okay, well this slug, let's say Amazon, like going back to like this constants idea.
00:11:21
Speaker
And this, for Amazon, you have to go new up this class. And the benefit of that versus storing the actual class name is that if you do refactor your code and move the class name around, you don't have to change the database entry because you have a unique slug that references a vendor.
00:11:41
Speaker
So I'm not saying everything has to be in the database. I think ultimately we'll still have to be like delegated to something, but I think it's pretty reasonable to say that like, Hey, you know, I want to say, for example, like turn scraping for Amazon off today.
00:11:57
Speaker
and that has to be like a database configuration. So I think having another database configuration that says, what's the frequency of the scraping or whatever the situation is in the database is not too far fetched. Oh, Skyler, we can't hear you. That's interesting. Apparently, I muted myself in the software.
00:12:25
Speaker
I mean, I was going to say like one, one potential approach is like, I almost think of it as like a repository of, of like you have a best buy model and it just extends the vendor model, but you can call like new best buy or best buy fetch or best buy get or something that, that that's going to create the underlying eloquent record. If it doesn't exist in the database yet, maybe it's like first or create.
00:12:55
Speaker
And then you can work with your best by class, but it is just a, it's gonna, the return type is the, is just the regular eloquent vendor model. And you're able to kind of like use that in the code.
00:13:11
Speaker
That's actually, I'm constantly fascinated. Have either of you used parental, the single table inheritance package? I'm constantly interested in this idea and I've never really used it for anything. I've heard of it. I feel like I've looked at it. Hey, even though I was trying to remember the name of it, so I'm glad that you remembered it, Chris. I think is that a Titan package? Mm-hmm. Parental. Yeah, I know someone who uses it.
00:13:40
Speaker
And it, it solves some problems for them, but I think it's a little,
Parental Package and Single Table Inheritance
00:13:46
Speaker
you know, it's also adding some abstractions or I think, you know, my sense is that there are some rough edges. I don't want to speak, speak for that for it too much. Cause I haven't had the experience with it, but I feel like when I was sort of source diving, I just looking at the code, I got just worried that as soon as you started to color outside the lines at all, like if you're starting to do
00:14:11
Speaker
you know, sort of more complex relational things where one or more of the models are using parental, you know, doing like the relation create method or something like that. I wasn't sure if it handled all those situations. Maybe it does, but I do like conceptually this idea of, you know, having your vendor model and then having
00:14:40
Speaker
a couple of specific implementations of the vendor model that get instantiated based on some criteria in the database. Because that, it would be really nice to be able to say like, when I care about it, I can just say like, all I care is that this is a vendor. But in this case, I care that it's specifically the Best Buy vendor. And that Best Buy vendor class can not only
00:15:08
Speaker
you know, have its own identity, but you could also implement custom functionality that's just for that vendor. I think that's pretty, it's a neat idea, but I've never really explored it that much.
00:15:20
Speaker
Yeah, I will agree. I think now that you mentioned it, I feel like that is actually a really good solution. I think the question is, does this work in reality? Some of the things that I could think about, do you have a default attribute on the Best Buy model that extends the parent vendor model that constrains it to the Best Buy slug, more or less?
00:15:48
Speaker
Is that how it would work, essentially? In my mind, it doesn't extend the actual eloquent model. Maybe it extends some interface that's your vendor, your prefixed vendor, specific vendor that has just one method that's get or fetch or whatever that is. And it's going to do the thing that's get by slug, best by. But it's in that one place.
00:16:17
Speaker
Maybe it's like first or new so that if for some reason in your test, you don't have to create it ahead of time. It's just going to create that on the fly.
00:16:28
Speaker
And everywhere else, you just can get that. And that way, you don't have to worry about, oh, man, I'm overriding an eloquent method, or I'm doing something crazy like Best Buy without scope, and then find slug target. And you're doing something crazy ridiculous in your code base. This is just all it does is gets
00:16:52
Speaker
The model and then your your return is your vendor model. You're not having to deal with with anything else Yeah, no, I well, I think that that's getting closer to the solution that I've I've been leaning towards That before before we get down get sort of further down the this path
Reserving IDs for Known Records: Feasibility Exploration
00:17:14
Speaker
though the one other thought that I had is
00:17:18
Speaker
And this is probably an awful idea, but again, we, we have a bunch of cases where we have just literally hard coded IDs. And that's just because we're dealing with legacy data that already existed. Uh, and there is no slug column and for whatever reason.
00:17:40
Speaker
you know, at the time, the choice was made to just use the hard coded ID. And I think, yeah, I don't know. But anyway, a part of me was like, what if you just, what if you just by default for a table where you knew this was going to be the case, or maybe you just decided by default across the board,
00:18:03
Speaker
you just, in your initial migration, set the auto increment value to a thousand or 10,000 or some number where you just reserve, say, the first 10,000 IDs to known hard-coded IDs, right? So if you just set the auto increment to 10,000, then any
00:18:29
Speaker
automatically generated records, either if you're just seeding data for tests or if you're creating data through a CRUD interface. The first record that you would create would be with ID 10,000 in one or whatever, 10,000. Then you could just say, if I need
00:18:52
Speaker
to know that this one specific vendor exists. I just hard code ID one for that vendor. And then I just know that vendor one is always gonna be this one. You still kind of have to, you still have to come to the same sort of first or create type solution, but it does allow you to avoid having some sort of unique slug
00:19:21
Speaker
I don't know if that's really beneficial. I mean the performance improvement of doing like an integer based lookup is so minimal, you know, with an index slug column that it doesn't really matter. I just thought it was kind of an interesting idea.
00:19:37
Speaker
Yeah, no, it's definitely an interesting thing. I feel like we've all been there just reserve some, uh, you know, subset of, uh, IDs in a database and, you know, then you can reference the IDs and then, you know, you have to make an entire podcast series to talk about why that's a bad idea. Um, I, I did want to piggyback on what, uh, Skyler, uh, mentioned as far as like having this. Repository per se. I feel like on one hand, I kind of liked this idea, but I think in reality,
00:20:07
Speaker
you know, most of the time you're diving into like a series of relationships, right? You may be looking for something, right? And you want to be able to say like, well, I want to get all the products that Best Buy has, right? So it's like, you know, give me all the products where has, you know, some sort of vendor of the thing Best Buy. And I feel like then you're going back to the place where you're just dumping a bunch of these
00:20:37
Speaker
Repository calls, just say, just give me this, give me this. And I still feel like going back to more of a scope solution is more elegant.
00:20:49
Speaker
rather than having another class. Because that way you could just say like, well, here's like the five or 10 vendors that we have, and there's the scopes, they exist. You know, and then that's just how you reference them. So I don't know, that's just my initial, you know, thinking behind, you know, just fetching one record, you know, you know, I feel like it's
00:21:15
Speaker
That's great when your database is simple, but I feel like once you have a very complex relation structure, it could get a little gnarly. You're going to have to pass that stuff around in different closures because you're going to pull it up initially and so on. Yeah, I mean, so there are two things that that makes me think of. One is I think that this pattern
00:21:45
Speaker
particularly applies to models that are almost singleton-ish in nature, you know, where arguably you only need to load it from the database once for as many times as you would need it. That's not always the case and I think a lot of times
00:22:11
Speaker
You can kind of shoot yourself in the foot by passing the same copy of a model around and accidentally mutating something or loading a scoped relationship or something like that. But there's a part of me that thinks this type of model, you can almost treat it as like,
00:22:37
Speaker
something that you can resolve out of the container as a singleton, right?
Managing Special Records with Directories
00:22:41
Speaker
And like, even, you know, you could use the container as the mechanism for fetching it perhaps. And then, yeah, I mean, the direction that I've been kind of leaning, and we have sort of a very naive version of this in the code base now is this, what we're calling directories, right? Where it's essentially just a class that
00:23:06
Speaker
knows about these special records. And it just does sort of a first recreate where it's looking for the record by slug, and then it's creating it with the default data. And that way, everywhere in the code base, you can just call the directory. Most of the time, it's just gonna be loading the record from the database, but from a test perspective, you're loading it, or you're creating it at the same time.
00:23:35
Speaker
And that's kind of like what you were talking about, Skyler, except for it's just a single object responsible for fetching any different number of named items. And I feel like there's an opportunity for
00:23:53
Speaker
I think that this is a place where there's an opportunity to abstract this out into a package that handles a lot of the stuff that we were talking about. Because I can imagine sort of in this world like vendor colon colon directory arrow best buy and that gives you
00:24:13
Speaker
some sort of object that you can then either get a singleton instance of Best Buy, right? If you just want to have that vendor and you know that you're just, you're kind of going to treat it as a read only model, you could call get maybe on it to get a new copy of that particular model. And both of those under the hood would do a first or create
00:24:40
Speaker
But then I imagine a way where you could essentially have some helper functions on that object where you could pass in a builder and have it add constraints to that builder. And maybe even through some like macros,
00:24:59
Speaker
or some of the automated hooks that eloquent gives you, you might even be able to sort of have magic dynamic scopes for any of those named directory items. I feel like that, like in my mind, there's a way to do this. I don't know if there are any major roadblocks for it, but I like that idea of having just like a single place that's responsible for
00:25:29
Speaker
anything that can be mutated. And then, you know, like if I'm going in there and changing the slug, uh, in my directory, I need to also create a migration or something that's going to update the slug in the database. If there's a chance that that model already exists with the old slug, you know, like it gives you sort of this one point of control over the, that connection between the database and the code base.
00:25:56
Speaker
I mean, that's, it's an interesting, it's an interesting approach. I feel like there's like, you know, we, I feel like the approach that I was, I was taking in even the directory approach is like, how do I get this one model specifically? And then Bogdan's bringing up this approach of like, well, I need to like query off of a specific model. And I don't know that the, the repository situation that I was recommending, like doesn't really solve for that. It only solves for like resolving this one
00:26:25
Speaker
model out of the database for one-time use. I think the directory, it has the ability to
00:26:34
Speaker
to add on the builder and create more dynamic relationships. I also think when you're in a situation where you're pulling in a lot of different data and doing a lot of queries, maybe it makes sense to go back to hard coding the slug. Or it's probably dynamic in nature anyways. Someone is requesting from the UI something specific. So you have the ID or the slug already from some form field.
00:27:04
Speaker
and you don't need to like encode specific instance. I think there are lots of cases, I don't know, I've already, certainly, excuse me, I've come across cases where you, you know, you want to add that constraint, right? Even in, I don't know, in that scraper example, I can imagine perhaps the,
00:27:32
Speaker
you know, the Best Buy scraper is going to want to fetch all the, all the prices that are related to the Best Buy vendor, right? So it's going to want to do like select from prices where vendor ID equals and, and like get a number. And so being able to do something where you like do, you know, prices,
00:28:01
Speaker
related to directory or whatever,
UI Design and Database-Driven Configurations Integration
00:28:04
Speaker
you know, and that's like a macro that then you just pass in this directory item and let that be responsible for understanding how to add the like where in or where constraint. I can imagine there are a lot of places where you'd still wanna do that.
00:28:22
Speaker
Yeah, I will also say, like, I think there's probably a few points. Like one, uh, you know, we need to like query this data. I feel like the other piece of it is also just having a reference to, uh, the idea of Best Buy, right? And it's kind of like maybe in, even in your UI or even in that scraper, you may want to like show something sparkly next to the Best Buy for the Best Buy vendor, right?
00:28:47
Speaker
So I feel like just that use case of pulling one item out is still valid. I don't think it's not dismissed by the scope idea or anything like that. I think that is probably 90 percent of the use case is that let me find Best Buy so that way I can go and present this Best Buy or use the Best Buy scraper or whatever the case is.
00:29:14
Speaker
Yeah, I'm not dismissing those ideas at all. Yeah, I think there are lots of cases where you may, you know, you may have like, to that point, hard coded UI, that's like, there's a best by tab at the top of the screen. And maybe, you know, maybe your system can scrape 15 different services, but you know that mostly you care about these three vendors, because those are the big ones.
00:29:43
Speaker
And so in your UI, you wanna show a Best Buy tab that has a timestamp of the last scrape and maybe the number of records. And you wanna have four tabs, one for Best Buy, one for Amazon, one for Circuit City, of course, and then the fourth that's just all other vendors, right? And that's something that, sure, you could build a whole database schema around
00:30:10
Speaker
you know, featured vendors or like, you know, have like an is featured flag on a vendor and dynamically build those things. But like, a lot of times, I don't know, to me, that just often feels like you're adding a bunch of junk to the database just because you where you would have usually just, you know, added three tabs to the UI. You know what I mean?
00:30:40
Speaker
And so to be able to just say like quickly, you know, give me the latest, uh, run for these three vendors and then give me, you know, a sum for all vendors that aren't these three vendors in, in the controller and then just hard code that UI. And that way, if I want to show the best by tab with like the branded blue and the Amazon tab with the branded yellow, like I don't have to then go into my database and add a.
00:31:09
Speaker
vendor highlight color and vendor text color and you know like And and obviously all of that's contextual because you may have a you may have an app where you do want those things But I'm kind of talking about the times when you don't right when you you need it, but you don't you don't need it to be super configurable and
00:31:33
Speaker
Yeah, I feel like you make a good point about just the context of things, right? Even this idea of should we use constants in three places to determine what is Best Buy or Amazon or Circuit City, or do we build this very, very complex
00:31:50
Speaker
you know a system you know some sort of directory structure or like the parental or use a parental package you know i think it all you know. Does always come back to like you know what is kind of i think the best bang for the buck right and i really feel like you know if if you do need custom colors you're gonna you know.
00:32:15
Speaker
have a lot more database tables than just, you know, whether or not this thing should run at a certain time. You know, those are kind of like just very small little peanuts comparing to what you're actually dealing with.
00:32:31
Speaker
But yeah, I still think having a good database-driven solution is probably the most optimal way of doing this, rather than going back to my initial point where I made a beginning of the podcast.
00:32:51
Speaker
If we were to really over engineer this, how can you make it so that your app doesn't really have any understanding of what is Best Buy, aside from some of the custom logic that does the scraping or the API stuff. But a lot of this stuff needs to be configurable
00:33:17
Speaker
You know, that's my position at least because that way, you know, if there is a code change where you're like, well, we don't want to present Best Buy on this tab. You know, it's a configuration change in the database and it's, you know, something that is a no code solution effectively once you deploy it. So, okay. I just had a thought.
00:33:42
Speaker
Like I wonder if we can combine the best of sort of both worlds here in that.
Linking Classes to Database Models Using Traits
00:33:52
Speaker
If I have a trait that I can attach to any class that basically designates that class as being tied to a database model somewhere, right?
00:34:08
Speaker
And that trait just exposes a few of the functions. So kind of going back to what Skyler was saying, instead of having this single directory that has lookups for a bunch of different things, you'd create individual classes for each of the things that you want to look up. But each one of those classes really just pulls in a trait. And by default,
00:34:38
Speaker
what that trait would do is essentially do a first or create on where some column name equals this object's fully qualified class name. And then you'd have a inverse of that that is a cast
00:34:59
Speaker
Right. So, you know, maybe by default, we've used the term, we've used the name handler a bunch. So I'm just going to say, let's call it handler. So by default, if I have a vendor model, I could add a handler column to it. Right. And the handler column, I would just do a, what's it called a, you know, a custom cast for that column that, uh,
00:35:31
Speaker
handles all the instantiating the correct version of the handler and making sure that the class name actually implements the methods or the interface or whatever, all the validation and stuff around that. So if I pull the best by vendor, I can just do best by arrow handler and that gives me this like best by class, which can have a bunch of vendor specific custom methods on it.
00:36:00
Speaker
At the same time, I can just do that, you know, that best by class, colon colon singleton. And that will check to see if there is an, or, or I guess, I mean, we could even, you could just implement them as singletons in the container. So like, I could just do app best by class. And then on that class I can get.
00:36:29
Speaker
a copy of the Best Buy model from it.
00:36:31
Speaker
I can get a shared instance of the Best Buy model. I know in some cases, and I don't want to talk too much about the specifics because it's a private code base, but I know of someone's code base where they have this vendor concept and it's very central, and they may need to have a reference to that vendor in
00:36:59
Speaker
10 different places during a single request. You don't want to load that vendor. You don't want to do a first or create 10 times for that single vendor when all you're using it for is to get the ID and the display name or whatever. I do think you want that concept. But I could see having this fully qualified class name in the database,
00:37:29
Speaker
which I, you know, I don't, I don't think I buy your concern there about it. And cause like, we already do that, you know, with polymorphics so much like renaming a model is just, you know, it's like a thing that you have to take into account. So if you ever wanted to rename one of these handlers, you just have to know that it has to go, you know, hand in hand with a migration or something like that. I think that's just the reality. You could also implement a morph map.
00:38:00
Speaker
Right. Yep. Situation. Exactly. Again, though, if you decide to change the morph map, then you've got to change stuff in the database. Like, I don't. Yeah. Eventually, when you have to change something, you have to like renaming models is fraught, right? It just is a tricky thing. And, you know, there are tricks we have. I know we have that.
00:38:28
Speaker
command that scans the entire database for polymorphics and can check it against the morph map and make sure there's no stray things with old names, but it's scary to run that.
Refactoring Challenges with Database References
00:38:42
Speaker
Yeah. I mean, you know, I'm not saying that it's not doable or there isn't a way to prevent it. I just feel like it's another layer of, you know, in, you know, you deploy it, it's all fine. And then in a year or two, they're like, Oh no, I'm going to restructure, re-architect my folder structure. Right. The good old folder structure. And now all of a sudden, you know, it becomes, you know, like, um,
00:39:06
Speaker
an undertaking to move a class from one directory to another. Yeah. You know, so it's, you know, it's just like another thing. Well, how can you kind of minimize some of the risk long-term like going, I think you could call, you know, you could call the handler, like you could call handler and the cast could like look up the, the like known directory of where the handlers are. And if one exists with the name of the mob, like there's waste. I guess.
00:39:34
Speaker
Or you could even have a handler. I mean, you could just have like a property on the handler that's like old names, you know, and it could just do a, you know, instead of that, then it would just do a. Where in, you know, and it could still do, it could still be.
00:39:59
Speaker
I think it would still, it could still be pretty performant because it could just do like where slug equals or where slug in. And that way, if there is a match on the, if there's a direct hit on the current slug, then right.
00:40:15
Speaker
Yeah, I'm sure there's a way to figure it out. I did want to circle back about this Singleton idea and this code base that's private. What is the current solution? And maybe that's a good way to kind of, how does this problem get solved today? And what would be, maybe the Singleton idea is something viable for this example that you mentioned.
00:40:44
Speaker
I think in that code base, there's like a config file with a bunch of IDs. I think that that's, and then those are just, there's some sort of helper function to either just return the already fetched copy for that ID or fetch it fresh if there isn't a copy for that request.
00:41:09
Speaker
I think that's how it's done. I'm not entirely sure, but it's something like that. I know it's using hard-coded IDs. I see. Interesting. Yeah. There is, I suppose, another world of just using a, instead of hitting the database all the time, maybe it's like some sort of cache mechanism where this stuff just gets loaded into
00:41:33
Speaker
you know, Redis or, you know, some other store and which is a lot more performant and, you know, just has like, you know, bare minimum information such as ID, slug and, you know, display name that you use commonly. And that's just what you interact with. Maybe that's also like a solution to this. Yeah. Yeah. I mean, I think.
00:41:59
Speaker
Yeah. Yeah. I mean, of course then you're, you know, you're dealing with the downsides of cash too, but I definitely think, I mean, I think if, if, if we were doing this as a package, I would definitely imagine that you would have some sort of caching mechanism that was an option where, you know, it was just like configurable and then
00:42:24
Speaker
Yeah, you would just add some sort of model hooks to, to clear the cache whenever a new, new record was saved. Potentially. You wouldn't even necessarily have to do that in this case, but I mean, you could just do like a request, a request cache where it's just like an array of data that's being cashed. And then the request ends and it's gone. Right. You only fetch it, right. You just fetch it once for the entire request.
00:42:50
Speaker
Yeah. Yeah. Cause actually going back, the reason that the config file with hard coded IDs works is most of the cases you just need the primary key to do other queries, right? Like I think a lot of the times you're just doing like,
00:43:12
Speaker
you know, give me all the prices for Best Buy or, you know, give me the crawl history for Best Buy, right? Like, and so just having the primary key and understanding it is enough. You don't actually have to pull the Best Buy record from the database. If you just know what the primary key is, you only kind of need that singleton concept
00:43:37
Speaker
if you want to pull it from the database first before you do that that query so that you don't have the hard code IDs anywhere.
00:43:45
Speaker
So what if this is kind of crazy idea? So like some, I love it already. This, this handler approach and there's some, there's some method that's like primary key. And then there's a, that's going to return the primary key and there's a command that you run when you deploy that's going to, to like do the crazy lookup to like of all of the things to get the primary keys.
00:44:12
Speaker
and then basically like store it in some cache. And so every time you call primary key, like it's not having to do a database query. It's just returning the thing that already exists. Like maybe, maybe it's writing everything to a, to a, just like a JSON config file, or maybe it's pushing it to Redis, but
00:44:34
Speaker
Like the primary key method is going to pull it out of the cache that the command runs on deploy. And that way, like if the ID is different in your local environment, you just run the run the command to populate the primary keys.
00:44:53
Speaker
doesn't really matter which environment it's in. And then you, you always have your, your primary. I don't know if that makes sense with that. I mean, I think the problem with that is it like what happens if I add a new vendor between deploys.
00:45:09
Speaker
I mean, maybe there's some, there's some observer model event that's gonna like cue off the job to recalculate it. I mean, I guess like if something is important enough to be, if something's important enough to be a vendor that has all this special code, then you can't really just add something to the database. Like this is a special thing. You've got to push code out. So yeah, you're never like,
00:45:36
Speaker
The nature of this problem is such that that's never going to be an issue, right? You're never, if you're adding a new vendor that needs to be referenced in code, you're, you're by definition doing a new deploy. There's no, there's no way. Adding this record in a migration or a cedar or wherever based off the last two episodes.
00:46:05
Speaker
So yeah, I don't know. It's interesting. Yeah. I mean, I, I like that just because then your criteria, you can use slug, you can use name, you can use whatever that you can use some fully quite qualified class name, but essentially, you know, it could, it could be the type of thing where I don't even know, you know, I guess using the cash, you could just,
00:46:32
Speaker
have the primary key method, look it up in the cache, and then you could have a command that warmed the cache. And if you don't run the command, it just does it once and then stores it in the cache. But otherwise, that would just become sort of like an optimization, just like root cache
00:46:59
Speaker
you know, all those, all those artisan commands that are just for sort of production optimization, that would let you cache all those IDs once on deploy. But if you don't do it, it's fine. It would just still do that first or create one time, the first time you need one of those records in code. I like that.
00:47:22
Speaker
Yeah, that's a pretty decent solution. So where do we come from? We started at writing strings straight up in queries to constants, to scopes, to what is the final solution? What are we thinking is like a really good hands-on approach without over, over complicating it. Without over-engineering it? Yes.
00:47:50
Speaker
I really like the idea of some object that has a bunch of generic helper functionality around the common use cases.
Helper Methods for Dynamic Model Access
00:48:09
Speaker
Getting a copy of the model, getting a singleton copy of the model, getting the primary key and foreign key name for the model, probably.
00:48:20
Speaker
as well as applying the model's constraint in a belongs to and has type, you know, both sides of the relationship, right? So you would be able to just
00:48:37
Speaker
sort of transparently pass a query builder into it and say like, add a belongs to constraint to this query builder for this thing. And it would, it would then be able to transparently add like where vendor ID equals three, right? Or, I mean, I guess it's only that side of the relationship that you'd have to have to worry about, you know, like best buy with prices.
00:49:07
Speaker
get and like, it's going to know, it's going to know to be able to do that without executing two queries. Right. Right. Yeah. Well, it would still execute without executing three queries and still execute the best by, well, no, I guess if it's a singleton, you yeah. If it, yeah. If it doesn't exist, I think that would necessitate multiple queries, but right.
00:49:33
Speaker
And yeah, it would take a little bit of time to get the more complex sides of that, right? But it feels to me like you could, because models let you boot arbitrary logic from a trait, right? So you could just have a trait that you could drop on any model that would
00:50:03
Speaker
handle like sort of dynamically setting up scopes for you. So then you could still kind of just automate your scenario of like vendor colon colon best buy. And that would just set up the query builder. And you could publish a
00:50:27
Speaker
custom caster for the, if you wanted to connect it by like the fully qualified class name in the database. But if you didn't want to, you could just override like the attribute value on this thing to be slug or ID or whatever you wanted. And that just becomes the lookup table or the lookup value. And that seems like it covers everything that we've kind of talked about, doesn't it?
00:50:56
Speaker
Is it doable? I think it's doable. Okay. All right. Cause yeah, I feel like, I mean, it's, it's great to talk about in concept. I feel like, you know, it's, it's nice, but I feel like, uh, and maybe this is a challenge to some of our listeners. Like, is it possible to create this?
00:51:14
Speaker
you know, thing that is a package that you could just somehow expose some of these methods and, you know, uh, use them, you know, share it with people. Like, I dunno, is, is that too much to ask for? I mean, also I'd be curious to know if this is like, is this a common problem for lots of people or.
00:51:37
Speaker
Is it just like a problem that like due to the nature of the, of what we work on comes up more often? I think it's a common problem, but I don't know. I'm sure it's a common problem. I've run into this in legacy apps that, that aren't like non-Larevel apps that
00:52:00
Speaker
We're trying to pull into Laravel or just PHP if you had to figure out how to get this stuff out. But also, like, Greenfield and Laravel applications where we needed, you know, we had a particular model that was super important to the business. You couldn't just add one without also adding a bunch of database, I mean, without adding a bunch of business logic.
00:52:25
Speaker
provisioning this model and doing all sorts of things. So like, I don't know. Not that I've experienced hundreds of applications, but across the spectrum of things, it seems reasonably important.
00:52:42
Speaker
All right, so let's sanity check. There's the vendor case, I think this feels great for, but here's another real world case that we have at InterNACHI, right? We've got courses and we've got exams and there are many courses and there are many exams and many of them, while important, don't have any particular special meaning within the organization.
00:53:10
Speaker
But we have one specific exam that is a core membership requirement, right? And we have to deal with that in a bunch of different ways around checking eligibility for certain things or compliance reasons or, you know, well, for all sorts of different reasons. And the same is true for courses, right?
00:53:32
Speaker
Um, something like 12, 12 courses that every certified member has to take over the first year of their membership. And we need to, you know, we need to check on their compliance and, um, you know, present, you know, present those courses in specific ways across the website. And, and for the courses.
00:53:58
Speaker
I mean, we still do have the hard code references to those courses in a bunch of places, but for the courses, we also do a lot of that dynamically based on database tables. But, you know, there's one exam, you know, that I know the idea of by heart because, you know, we've had to reference it.
00:54:22
Speaker
a thousand times, right? Would this work for both of those cases? I think I can't see why not. Yeah. I mean, I definitely think it would work.
00:54:37
Speaker
I also maintain my position of this special exam ID is only special now. If it were to change, there has to be an easy story to transition to another ID. You can't just go back and update all of those places, or even if it gets consolidated into
00:54:59
Speaker
some sort of class that references this thing in some sort of magic way. I ultimately think having a real database-driven thing is kind of the answer, where it's like, hey, give me exams that are special to do this stuff. But ultimately, I think it will solve it.
00:55:25
Speaker
Yeah. I mean, in the case of the exams, the way we do it is awful, right? It's only, it's only that way because of legacy reasons, right? And we've run into so many weird test failures where in testing, you know, that now we're accounting for this, but for a while it was like, you know, a test would just happen to factory up an exam with like a special ID.
00:55:51
Speaker
And then a query would fail because that ID was explicitly excluded from the query for some reason, or something would go weird because that ID was specially reserved.
Legacy Systems and Special IDs Complications
00:56:04
Speaker
And I'm pretty sure now for that table in our testing migrations, we do set the auto increment to like some high number so that all the known sort of special IDs don't accidentally get created. But that was like a lesson that had to be learned. So I'm not saying that that is a good way to do it by any means. But I think given that like not every application is greenfield, like
00:56:33
Speaker
I think this solution would work for that scenario, even though, you know, if I was starting from scratch, I wouldn't necessarily do it that way. Right. But does that also expose the issue? Like, you know, I hate to kind of dive back into it, but you know, the fact that we have to reference this ID over and over and over again, and there was this conflict, say, in testing.
00:56:57
Speaker
you know, because we're kind of like creating this unique constraint and excluding it somewhere, like, you know, it just kind of seems to me that, you know, it's almost like a code smell, right, of relying on a very special ID or relying on a certain thing, you know, because, you know, it,
00:57:16
Speaker
Especially, I mean, when you have maybe one application, that's one thing, but if you're deploying your applications in multiple places, say, think about like WordPress, for example, right? Like there has to be like a very dynamic way to be able to get this data in and out without referring to like a very specific thing. Maybe like a slug is fine, but I feel like when you look at IDs and it gets a little complicated.
00:57:44
Speaker
Yeah. I mean, there's a part of me that that's still intrigued by the idea of just, you know, having reserved IDs with some, some auto increment constraint and just, you know, just having an enum that every time you need a new
00:58:04
Speaker
a new vendor or a new course that's like referenced in code, you just add that, you just do the next number, you know what I mean?
Slugs vs Fixed IDs: A Flexible Approach
00:58:14
Speaker
But generally, I think probably using slugs for anything new is gonna be better. And then you just dynamically load the ID from this system. So all your queries still can use the ID, but you're not relying on that
00:58:34
Speaker
that ID is only unique in the current deployment, whether that's local testing or CI or production. Like you don't have to rely on it being a specific ID. Yeah. Yeah. I like that. All right. Well, I'm curious. I'm actually, it kind of feels like we landed on a solution that I'm, I'm pretty happy with. I mean, maybe in trying to implement it, it'll turn out there's a major gap.
Conclusion and Listener Feedback Invitation
00:59:02
Speaker
But I'm curious to see, you know, if in a week it turns out we missed something very obvious or there's a totally different concern, but like, I don't know, maybe we solved it. Wouldn't that be cool? I would, I'm interested to hear back from people that listened to the episode. Uh, if anyone listens to the episode, like what are the things that you're screaming at right now? Because we've just missed, missed the whole thing. And you're like, this is not solving anything that I'm having to deal with.
00:59:32
Speaker
I'll be curious. It's been really cool to get feedback so far. And I feel like some of the responses that we've had have been really insightful and very, very, very well thought through. So yeah, I'd be curious to know if there are any other approaches to this type of thing that none of us have even thought about.
00:59:56
Speaker
Yeah. I mean, it might be a coincidence, but I saw in Laravel news, someone released like a package that deploys things one time, you know, when you deploy and maybe it's coincidence, but maybe they're a listener to the show. So I think we can take credit. I think we should take credit for sure. Yeah. All right, sweet. Well, any, any closing thoughts before we call this one?
01:00:22
Speaker
We're good? We're good. Let's do it. All right. Well, here we go.