Designing compute for the edge: Out of the cloud and down on the ground,

Transcript

Speaker: Our founding vision was the pod, like the benefits that a smaller, you know, a self-contained data center in a box could deliver to a commercial entity. But we didn't know how to do it.

Speaker: It's like ah we had ideas, but we had to go and do R&D to figure it out. And in the end, we adopted a compute standard that is the cutting edge, miniature, low power, rugged form factor of the U.S. military.

Speaker: Now, it's an open standard. It's not proprietary in any way. It's already used around the world, but it was developed over 15 years. by the US military. Because why? Because they have a vested interest in things that are tougher, smaller and have more compute crammed into them, right? For drones and tanks and on what have you. Then we're the only company in the world that's taken that out of defense and aerospace and adopted it, evolved it with new IP and brought the cost down a lot to make it relevant for the commercial edge computing market

Speaker: This is Bare Knuckles and Brass Tacks, the tech podcast about humans. I'm George K. And I'm George And today, our guest is Jeff McMillan, founder and CEO of GECCO. That's spelled G-E-C-C-O, which stands for Green Edge Computing Corporation.

Speaker: And this might be one of the first times that we have interviewed someone working on hardware. So Jeff's team is building... I guess what you could call portable servers. We're talking these tiny suitcase sized things that you can take out into the most horrendous terrain to get compute out at the edge.

Speaker: And it's pretty rad. They've been able to reduce the, I think the cost by like 90%, the size by 75%. It's insane. It was just an incredible conversation about what does it mean when you can take you know, not huge ass data center, but sufficient compute out into like really wild places.

Speaker: Yeah, I think what's really fun and innovative about him, George, is that he has built a technology that actually um is the manifestation of of a working theory. Like I know i've I've had, I think he's been on the same page as me for almost the same amount of time.

Speaker: ah Pretty sure actually we both came in same conclusion at the same time. The future is in small language models. this future The future is in localized distribution. And it's in, you know deployments that are actually dealing specifically with a certain use case that don't rely on a massive backend infrastructure, powering it with the water, with the supplies, with the needless, needless oceans of data flows.

Speaker: um And I think Jeff has has come up with a good technology set that actually enables that ah for critical organizations that willing to invest and they have the capacity to. So it's a lot of military, it's a lot of healthcare, care but it's always been either military healthcare driving innovation anyway. Correct. So I think...

Speaker: I think this is one of those really cool episodes that if you're if you're a technologist that's actually interested in, no hype, but what is actually at the cutting edge of innovation that will likely have the best probability of being the thing that saves us from economic collapse, I think Jeff is one of the guys who are building the right kind hardware that's going to get us there.

Speaker: Couldn't tee it up any better myself. Let's turn it over to Jeff McMillan.

Speaker: Jeff McMillan, welcome to the show. Thanks, George. It's a pleasure to be here. Thanks for having me. We are very pleased to have you and excited to talk about a hardware startup rather than the usual SaaS stuff. um So, Jeff, why don't you give our audience just a very quick overview of what Gecko is doing, and then we will get into it.

Speaker: Yeah, sure thing. So um let's see. Well, I'll just start by saying that GECCO stands for Green Edge Computing. And that's important because we set out to try to do something different in the edge computing market, which seemed like a pretty big opportunity in terms of the way that the tech landscape is unfolding.

Speaker: We've seen a huge shift towards cloud over the last 15 years, and that's a pretty mature market. And Edge has always been sort of there on the other side of the computing coin, but is sort of the the younger cousin, if you will, to cloud. um But there's been a lot of improvements over the last decade in software, hardware, and user experiences that have made it so that enterprises are looking to the edge and saying, you know, maybe we could do our operations better or more cost effectively there.

Speaker: And I felt from a series of my experiences in my last startups that that market wasn't being served. There was a big wide open opportunity for someone to come in, bring some innovation and create an edge computing platform that was actually designed for the needs of edge, which are different than the needs of cloud.

Speaker: right cloud and data center you have a big building it's ah it's a fortress perfect power security temperature all that um pretty easy to run operations because you just log in and do your thing as a as an end user in the real world at the edge you've got to deal with constraints around temperature and vibration and shock and water and people and security and all that real life stuff So why isn't the gear, you know, why are you trying to cram 19 inch server in the back of a 7-Eleven?

Speaker: Doesn't belong there. So we wanted to create something different. And that's, that's really the genesis behind the thoughts on Gecko.

Speaker: And, and then we had to figure out how to make it happen. How do we make a product that's rugged somewhere between 80 and 90% smaller? consuming much less power, generating less heat, and just kind of plug and play, you know, fire and forget. That's that's what we set out to do about five years ago.

Speaker: Nice. Well, that's ah that's a very good summary. And then for our listeners, I always joke, you know the cloud is really just somebody else's computer. And so I think most of them are familiar, as you said, with data centers, or maybe they're not. There's these large, huge concrete steel buildings that look fairly anonymous from the outside, but they have seen racks of servers. And what you are talking about, and also I will say the edge has been overused to the point, but you are talking about like,

Speaker: kind of a suitcase shaped compute cluster that can be taken at the edge, literally mean muddy field, whether it's a war fighting situation or, you know, a remote a healthcare care situation, but the ability to do ah computer operations in the field.

Speaker: um I think there's another... ah wrinkle here, which we'll explore a little bit, um which is data sovereignty, right? Like we have seen, I guess, a a new consciousness arise that when data becomes the fuel for the AI revolution, that countries start to become really sensitive that their citizens' data may also be sitting in ah cloud, quote unquote, again, somebody else's computer that might be overseas or in another sovereign territory. um

Speaker: So can you talk a little bit about the role that maybe data sovereignty also plays in in being able to shrink compute down into something that can be more portable?

Speaker: Yeah, I think I can. um You've probably heard a lot from different folks on data sovereignty. It certainly is a hot topic, not something I ever thought would be even remotely a household name, but it's kind of floating around these days.

Speaker: You know, probably one of the best official models would be something that some of our First Nations partners are talking about, which is OCAP, ownership, ownership control, access, and possession of data.

Speaker: And those categories will sort of cover the gamut of how to create policy around what sovereignty means. But I find that that doesn't, like, it's hard to resonate with that. That's very complicated.

Speaker: I like to think of sovereignty as, an individual or an entity's ability, or in the case of the context of your question, a country potentially, their ability to do what they want with their data and their data infrastructure when they want.

Speaker: man and not have someone else messing with you. So an example would be, and there's ways that others can mess with you almost regardless of how you architect your attempt to architect your situation. So for example, if you had a province in Canada that has a significant amount of compute in it because it has a number of large data centers and there's through some type of political shenanigans, they get strong armed by a trading partner or something to do a certain behavior, then some percentage of, say, Canada's compute might be sort of now under someone else's control or denied access, or, you know, you're using it as some kind of leverage.

Speaker: But it might only be five data centers, so it's not that hard to sort of squeeze them. But if you, instead of having 10,000 racks of servers in one building, had 10,000 compute nodes spread out across a large region, each with the equivalent of one rack,

Speaker: under the control of 10,000 or maybe 5,000 different entities, whether that's universities, First Nations groups, municipalities, hospitals, what have you, well, good luck strong arming 5,000 different groups to do what you want, right? Some some might be strong armable, but many are gonna just kind of tell you where to go.

Speaker: And when you look at the idea of sovereignty in Europe, for example, a number of years ago, they announced their own sovereign cloud. But they called it the distributed sovereign edge cloud.

Speaker: So they threw in those extra words of distributed and edge. And really what they were trying to do is create 10,000 nodes that are distributed, partly because they have a lot of different countries that would argue, well, who gets the data center, right? So they spread it out.

Speaker: But also if you imagine five data centers powering all of Europe, where do you think the five first five missiles will come from? right and head to when there's a conflict but if there's 10 000 and half of them are secret or hidden or inside of other facilities you wouldn't even know they're there that's a much harder target so i like to think of sovereignty as again the ability to kind of control your own destiny and do what you want to do with your data. And the last thing I'll say about that is we hear a lot in the news about sovereignty of, you know, a country, but there are many groups within a country that have their own sovereignty agendas.

Speaker: and So for example, first nations want to support Canadian sovereignty, but they also to their own sovereign compute platform within Canada. And then within their own groups, they have separations. And then you have academia and then you have hospitals and you have, um ah you know, commercial entities that all similarly want to have their own sovereignty within the whole.

Speaker: But I have to, I have to, sorry, before I officially get a question, i have have to call a question a little bit on this because, you know, I also spend a lot of time discussing data sovereignty.

Speaker: like I'm in Ottawa, so I deal with the Canadian government actually trying to fight this issue as well, like with the AI Secretariat, with the FedGov folks I'm sure you know as well. I have to argue about the sovereignty statement because I still believe there is a political and and and Westphalian a level philosophical meaning to sovereignty of nationalizing what your your data is, right? And I think when we talk about segregation of certain things,

Speaker: if If, for example, yes, we we allow the indigenous populations in Canada who should have their sovereign rights within their territories and within their treaty bound territories, especially. um If there is a national security issue and that is still on sovereign Canadian land as well, respecting the difference between indigenous sovereignty within our borders in the Canadian context.

Speaker: We still as like, you know, law enforcement or as public safety, and there are MOUs in place to to mechanize this, they still have a right to access that data. If there's an organized crime investigation happening and goes on the res, they can still reach into the res.

Speaker: What I think shouldn't be allowed is by the US Cloud Act, for example, you know, a US court ordering a Microsoft or any other kind of ah cloud hosted technology to hand over data that is Canadian in source by the by the legal definition of PII.

Speaker: I think that's where I see a differentiation. i'm I'm not sure like if you're saying something different or if that you're also aligned with that whole idea. i'm I'm definitely aligned with it. And it all weaves together. I mean, that was the wake up call because we've been talking about data sovereignty in a different way for the last decade, but we tended to call it data residency. And then everyone got comfortable with this idea like, oh, as long as Microsoft or Google whatever put their data center in Canada, then the provinces could buy the services for education and healthcare and whatnot, because, you know, it was in Canada.

Speaker: We're all good. Yeah, okay, it's an American company, but it's in Canada, we're good. And then the wake-up call that came over the last year was this whole Cloud Act and that sort of sort percolating into the general consciousness that, oh, there's this kind of kill switch concept.

Speaker: And the same thing has happened in defense with the idea of, you know, the F-35s and stuff. There's these ideas that have now entered into our consciousness that weren't there before. And now we want to protect against them.

Speaker: So, yeah, I mean, and we we play part to the hardware infrastructure aspect, but there's also a lot with the software. And that's that's a great position. We're kind of wondering, like when it comes to the edge pod, you're pitching that to both military units and as well, health care organizations, health care clinics.

Speaker: I mean, what does it mean for society that, like you know, the same technological solution to operate like without a cloud dependency per se applies to both a battlefield or a theater of operation and your doctor's office down the street?

Speaker: Well, I mean, it's it's really interesting because i've I continue to keep learning new ways to see how my own company's products can benefit or fit into the larger whole. For example, when we go and we talk with mines, some global mining companies that are doing all kinds of cool AI-enabled digitization, and it's not hard to conceive of the fact that they would be in relatively...

Speaker: bandwidth constrained or denied, or at least bandwidth expensive environments if they have to send everything by satellite. Like that's a pretty easy one. But if you look at, you know, a large enterprise like a Walmart or something that has thousands of stores,

Speaker: 99% of them are on fiber because they're in metros and they have good connections. And still 3% of them are down every day because of some, you know, you know, upgrade to a router that went awry or a mop bucket that tipped over on a internet table. Like, and their IT folks are always trying to get them back online some percentage.

Speaker: And those are the ideal scenario because they have, a big store, room for IT equipment, fiber connections. So you can see why the Home Depots and the Walmarts and and the world still maintain multiple servers on site for local on-site operations, even though they have a fiber connection to the cloud.

Speaker: And actually now we're seeing that that that rate of movement of applications and and workflows is actually moving more in the direction from cloud to edge than in the other direction. And you can talk about sovereignty and you can talk about all these other things, but honestly, I think it's just economics.

Speaker: Like, you know, it's, I call it the Netflix effect. It's like, oh, our cloud spend used to be $7.99 a month. And then we sort of blinked and it was $28.99 a month, like two years later, but for the same services.

Speaker: Now the cloud services are amazing, but you don't need them all at the edge in most cases. right So folks are looking and saying, geez, you know, my $50,000 a month cloud spend is now a month.

Speaker: What could I do with 400K a month in terms of buying edge infrastructure for on-premise compute? and and And then, yeah, there's an upfront cost there, but, you know, they start doing the math and're like, I could pay that off in 13 months.

Speaker: And then Gecko comes along and we just have a better way to do that. But I mean, edge as a category is solving that problem with or without Gecko. And then we just have sort of a better way to do it for those environments.

Speaker: I take your point that the part of your name there's a sustainability play there, which we'll talk about later, but also it does come down to the economics, right? If you're running a unit out in the field, like sustainability doesn't become a buzzword, it becomes just a priority to keep costs down, right? Which is um kind of the more compelling.

Speaker: argument for that innovation. um You had mentioned this decentralizing idea, right? Like instead of the five data centers in Europe, let me spread it ah out and make it much more difficult to target.

Speaker: That removes some of the dependencies we've seen with critical points of failure. Like, i don't know, in the US, we've had like three major outages this year, and it's always either AWS, Microsoft or Cloudflare essentially.

Speaker: um And so when you remove that single point of failure, I can also play devil's advocate and see that it might, um especially with AI applications running and increased need for oversight and governance coming online, that it is harder to audit or regulate once you're kind of processing in independent boxes instead of a handful of data centers. what What can we say to that from ah from your perspective?

Speaker: Well, I would say that I think I agree with you and just to almost to put an umbrella statement over it is if you're going to decentralize something, you do add an an extra layer of complexity just to to really boil it down. Because, you know, when you think about a data center, it's actually more of a business concept than a technical concept, right? You you look it and you're like, do you know, it makes sense to aggregate things like power and cooling and security.

Speaker: together and just put a whole bunch of computers in one big building so that we can manage it from a business perspective. And then when you go to distribution or distributed computing, you kind of go in the other way.

Speaker: So you're now reintroducing that you have um these other aspects that you have to manage. But the software and the procedures are starting to exist nowadays for doing that.

Speaker: And this is where you bring back in that sovereignty or another word I prefer is like autonomy layer to allow more localized groups to have control and autonomy over it. But they have to take on the extra burden and the ah extra procedures and the extra follow-up or, you know, overhead, for lack of a better term, that comes with decentralization. But what they get in exchange for it is a distribution of benefits, right? So if you put all the computers in one big building, one company makes money off of it, right?

Speaker: When you distribute it, you get to distribute the power consumption and all the not in my backyard type of stuff. you know, licensing and and whatnot is much easier if you're just putting down like a 10 kilowatt rack versus a 10 megawatt facility. Right. But you're also distributing the the positives as well. Different folks get to benefit, you know, so if a university does a distributed computing play, then maybe they need 40% of that for their own day-to-day things.

Speaker: And then they monetize the other 50, 60%. Interesting. Yeah. Right. So they can't do that in a cloud context. But I have to ask them, because that that in the software world correlates to the open source versus enterprise model, right, of purchasing. It's the same kind of idea where in open source, you're dealing with a a decentralized software that generally has like a maintainer, right? That's the old joke of like the internet is built on the work of volunteers years ago, and it's basically on popsicle sticks holding the entire thing.

Speaker: i don't think distributed computing is that per se, but, you know, there there there is... I think dependency then on a you know, potentially a technology like yours, right? if If it actually becomes something critical or, and I'm making the argument for you and that,

Speaker: the the solution technologies that are getting out there right now and actually trying to provide this capability to our economy, to to the the folks in our enterprises. um I think it's critical that you being Canadian owned and operated and headquartered here is probably a massive advantage. and I think, you know,

Speaker: going into to kind of consumption of resources as well. We look at every efficiency gain in computing, like it's been historically absorbed by more usage rather than less total consumption.

Speaker: So I think, you know, there is that nationalized angle there in ensuring that we are specifically helping the Canadian, the Congare, Canadian air crisis. um What makes edge computing sustainability claim different from that of like, you know, just the, the,

Speaker: rather earlier stage of it where you know you're you're getting more into still having to rely on massive amounts of resource consumption to be able to power the ah capabilities to deliver this type of compute.

Speaker: Like how are how are you guys getting more efficient essentially?

Speaker: Yeah, well I think there's a couple sort of levels to that answer. I mean, from the gecko perspective, we set out to make a technology that was 90% smaller because one of the insights that we had before founding the company was that miniaturization brings a whole bunch of efficiencies from a power cooling and ruggedization perspective. So how do we sort of amp that up into something that's productized?

Speaker: Right. So, so we do that. We have a product that's smaller. It uses 70% less power, but it's not as powerful as like a big rack mounted server. But the difference is, and people say, well, then what's the point? Well, the point is, thank you very much, that we're right-sizing the compute for the needs, right? Like you don't need a 60 core Intel Gold chipset in a 7-Eleven.

Speaker: You need literally like, like we have 16 core servers and most of our customers are buying the eight core version. Even you expect that it would go up to the 16 quarter. No, they're happy with, with eight.

Speaker: So now you're starting to like, you know, this whole adage, we grew up with it, reduce, reuse, recycle. A lot of people don't realize that those three words are in that order for a reason.

Speaker: You start by reducing, if you're trying, if the goal is like a better world or whatever, you start by reducing your initial requirements. Then you reuse what you started with, which has been reduced.

Speaker: and And then and only then at the end you you recycle. So that's what we do at Gecko. We start by reducing the footprint. And then we make everything modular and reusable, create a circular economy. And then and only then later at the end of life, stuff will get recycled.

Speaker: But it's already 90% less stuff to recycle in the first place. So it's all just becomes like a virtuous cycle. And then I think if you take that up a layer to what you were talking about, distributed computing and trying to meet the needs of the the country, well, then there's two categories. There's the commercial aspect, which is you know, we just help get more compute out in the world because there's an insatiable appetite.

Speaker: And we just do it more cost effectively with more resilient technology that has more of an eight to 10 year lifespan versus a three to four year lifespan. Another sort of benefit from an eco perspective. And then there's the more complex distributed computing at the government level, sovereignty.

Speaker: and yeah, you're right. um Both of you are right that there's more overhead, more complexity on the software side for managing that. But if you go in again, back to the Europe example, All the efforts over the last three years that they're investing in are actually almost almost exclusively the software layers to make that sovereign and an open source and like the way they want it.

Speaker: Getting all the systems to work together. That's what they're investing in is all that inter, the the glue logic. Make it work. Yeah, I appreciate that. ah For our listeners' sake, the principle that GA was talking about is called the Jevons paradox, right, which is that as something becomes more efficient, instead of consumption going down, it ends up going up, which is actually kind of where we are, even with the massive data centers, right? They have become...

Speaker: more efficient, but the demand has gone up. And so now we are in this bind where we're building, you said, 10 megawatt. We're building five gigawatt data centers and things that are ridiculous.

Speaker: You mentioned something that was very interesting to me, so much so that I changed my question. um You said, hey you know, you have mining clients that are, you know, contracting with you because what they're trying to do is some cool AI analysis or something decisioning on site in obviously very remote, rugged locations. It's hard to like lay fiber towards, right?

Speaker: This ah is of interest to me because i think as you mentioned, if you reduce the the amount of compute required, it also sounds like if you're going to run inference on these distributed machines that the models are necessarily smaller, more focused and have sort of a tighter business application. Like you're not trying to...

Speaker: jam the machine god down into a smaller box you're just saying like well this business has these applications and they need to run that compute at the edge in weird places i imagine that must also be true of military uses right like you're in very far frontier you're not laying cable supply lines yeah and you're not in that location for long so right setting up infrastructure might not even make sense Yeah. So can you speak to a little bit, have have you seen that from the client side? Like they what they're trying to do is just run smaller inference models, but they' that's why they need that compute?

Speaker: I want to say yes. The answer is yes, we are seeing that, but there's a pretty big caveat and the caveat is that it's all pretty rudimentary right now. Now part of that is because you're dealing with buyers or end users that this is all pretty darn new for.

Speaker: So they're, they're into like the baby steps stages. And part of it is that the infrastructure wasn't there, so they can't just jump in head first because the pool is empty currently. So they they don't want to jump into that empty pool yet.

Speaker: So we're coming along and positioning ourselves saying, look, you know, putting compute at the edge is tough. Putting GPUs is even harder. So let's work together on that. um In the case of mining, what we're seeing is just a general trend towards digitization.

Speaker: So like simpler level than even AI, right? Like this is a traditionally non-tech industry. that is starting to say, oh, how can we improve our bottom lines and improve mining safety and and extraction? ah How can we improve the overall business model using technology?

Speaker: You know, so I didn't necessarily say AI because they start with even kind of the precursors of AI, like simple vision, machine vision, machine learning. It's it's a stretch to call it like like it's certainly not even LLMs or generative or anything. Right, but it could be it could be focused machine learning. but yeah. chemical analysis And they want to go there. The the challenge is is that these mining companies, some of them that we're talking to are the actually the equipment manufacturers who service the mines, like big tractors and whatnot.

Speaker: And they're getting hundreds of different vendors coming to them and saying, we can solve this problem. We can solve that problem. We can solve that problem. And each one wants to put a little tiny computer inside the cab of their machines.

Speaker: And before they know it, have a hundred computers. The real solution is to move to server and only take in software in containers. Now, are you seeing this? so I have to ask because in the Canadian case, are you seeing this like in conversations you're really having with folks who will say are working out in the patch and and let's say northern Alberta or are these individuals that you're you're having these conversations with folks who are more doing like um offshore platform mining or essentially oil works?

Speaker: Like where are you seeing um really where this kind of interest is coming from because it's two completely different physical processes to extract the commodity, right?

Speaker: And where are you seeing more interest in the the utilization from? So we're not working today in with any mining entities in Canada.

Speaker: Well, I shouldn't say that. We're working with a couple of global operators who are machine vendors, who have certainly have a presence in Canada, but it would be a stretch for me to say that there's a direct Canadian tie there.

Speaker: And we're also, so those are the machine makers who wanted to just put some compute in the mine, not the machines. And then that didn't work because even wireless within a mine is notoriously unreliable.

Speaker: So now they've come to the conclusion that they need to put the compute right on the machines. These are big machines. Yeah, yeah. and And then they can't do that because there isn't anything existing.

Speaker: The other entities that we're talking to are actual national scale mining entities. that are part of specific countries that are just looking at their overall operations and having the same exact problem, but they're not seeing it from the machine vendor side of it. They're seeing it from the opera, like it's my mind and I just want to deliver certain capabilities.

Speaker: And it keeps, you know, the computers keep breaking and just things aren't working the way they'd hoped.

Speaker: Yeah, I mean, look, I mean, we also look at like forward deployed defense computing as well, as we've mentioned here, beyond just, you know, resource extraction. It seems that defense computing and and small business edge computing are converging now onto the same kind of hardware category.

Speaker: Do you see, like, does that convergence tell us something where compute infrastructure is generally headed? Or would you say the defense sector is just an early adopter subsidizing the technology for everyone else?

Speaker: I think it's generally heading that way. I mean, you just look at your iPhone and the amount of power that's in it, right? And even though we probably would never need more, in 10 years from now, we'll have yet again, two or three times more than we have today in our in our mobile phones.

Speaker: It's just a general, I think, trend of modernization, like of society moving up the tech ladder. Defense tends to lead in some respects. I mean, what we did is we wanted to, do our founding vision was the pod.

Speaker: Like the the the benefits that a smaller, you know, a self-contained data center in a box could deliver to a commercial entity. But we didn't know how to do it. it Like ah we had ideas, but we had to go and do R&D to figure it out. And in the end, we adopted a compute standard that is the cutting edge, miniature, low power, rugged form factor of the U.S. military.

Speaker: Now it's an open standard. It's not proprietary in any way. It's already used around the world, but it was developed over 15 ago. by the US military. Because why? Because they have a vested interest in things that are tougher, smaller and have more compute crammed into them, right? For drones and tanks and and what have you. Then we're the only company in the world that's taken that out of defense and aerospace and adopted it, evolved it with new IP and brought the cost down a lot to make it relevant for the commercial edge computing market, which

Speaker: Jay, to your point was this convergence that we saw of the behaviors of military and the commercial sector converging. But really when I say commercial sector, I mean more like the edge side of it because they have similar characteristics, right? Like that need for a smaller, ah a military term would be swap C, size, weight, power, and cooling.

Speaker: It used to only matter in defense. Now it's mattering in edge computing just across the board. Yeah, that's interesting. So you took this bet five years ago and that's what founders do is they try to see where things are headed and anticipate that.

Speaker: If edge computing succeeds at what it's promising and I know we live in a world where the big hyperscalers kind of maybe stop being the default place of choice.

Speaker: what If everything goes right, what do you think that the edge computing industry looks like in 10 years? You know, who who's winning and who's losing? Well, I think anyone who's selling into the existing cloud space, certainly like the hardware vendors that support it and supply it, they'll do fine because they'll just be able to adapt.

Speaker: And they can acquire companies that do edge computing if they, you know, it's the IBM model of, oh, the winds are changing and we're not going to be good at it, but we'll see who wins. Let the little kids duke it out and, you know, acquire them. So they'll do okay. Okay.

Speaker: I think what we'll see is ah maybe not an exact equalization, but right now I would characterize the edge computing market as being somewhere in the $300 billion dollar range compared to the cloud market that's in the kind of $1.8 trillion dollar range.

Speaker: So considerably bigger, five, six times bigger. I think we're going to see that gap closing because it's human nature for money to move where economic benefits derive.

Speaker: So for example, it's hard to get your hands on GPUs right now. So a bunch of smart people are going to just go, well, then what can we do? Well, we'll start to make GPU workloads run on CPUs. We'll start to make ah cloud ah AI models more efficient. And we've seen hundreds of percentages of efficiency increasees increases in foundational models just in the last two years.

Speaker: and And why are they doing that? Well, they're doing that because they just want to keep doing things while there's a shortage of, of you know, GPUs and memory. So what we're going to see is inferencing.

Speaker: The first wave of AI was all about training. Training is those big NVIDIA machines go in a data center. They do not belong at the edge, right? so So you put them in a data center and you use them the way they're meant to be used.

Speaker: They serve a purpose and it's a good purpose. But they just train the models. You can run a lot of models on your iPhone. So that, and I would call iPhone an edge.

Speaker: it's It's certainly not, you know, cloud-based and some big giant servers and whatnot. You hold it in your hand. So it's that edge interface between the analog world, which is us squishy humans, and the digital world, which is everything in the net.

Speaker: So, you know, we're going to see the next wave of AI is going to be inferencing and agentic. And those interact directly with humans and human mechanisms out in the real world.

Speaker: So that's where we're well positioned for. That's what we've been i yeah i i don't mean trying to line up with. don't mean to go fully sci-fi and we're nearing the end of the show, but when you say that, the only thing my CTO brain envisions is like, oh, you're basically talking about cyborg warriors. you're You're basically talking about battle droids, essentially.

Speaker: huge ah You're not wrong. You're not wrong. But I was, because i you said battle droid, I was going to chime in to say it sounds... uh also uh jeff increasingly relevant to the role of robotics in the field right like you are going to have to take you can't like pipe llm style but models into robots that are at a considerable distance from the unit, right? And the more capable as the robots get, like you were going to want them in further and further distances, whether it is in like, you know, they were experimenting with rescue operations, they're experimenting with um different things, but like they're going to be away from a Wi-Fi signal essentially, right? And so you're going to have to like extend the compute into these different places.

Speaker: It is scary to think about some of, you know, GA, like it is scary to think about some of where it's going, but it's definitely going that way. I think we all think that every time when we see one of those little Boston Dynamics robot dogs, we're like, oh, cute, when's it coming for me?

Speaker: it's Like, I think we all think that the first time we see it and the second and the third time, while also intellectually understanding that it could add all kinds of value to society. So where's the, where's the happy place?

Speaker: Yes. I'm not sure. Yeah. Well, that is as good a place to leave it as any. Jeff, thank you so much for the time. And it's really interesting what you're working on. And again, we hear from a lot of AI folks. We hear from a lot of SaaS vendors. So it's interesting to have a conversation about getting down into the guts of where the calculations are actually going to take place, which is perhaps in a reduced size box in a muddy field somewhere.

Speaker: Mm-hmm. Yeah, well, this was a fun conversation. I enjoyed it. um Thanks for bringing me in to have it. Yeah, we'll talk to you soon. Thank you, man.

Speaker: All right, crew, let's leave you with some questions. I maintain this hypothesis, right? I asked Jeff about it. He hasn't quite seen it. I think we're going to begin to see it.

Speaker: As companies try to apply AI to their business processes, my question to you is, are you right-sizing your compute and your model design to your needs? I have been, for the last few months, posting every Wednesday about...

Speaker: These case studies of where practical AI and very focused models are achieving incredible things, including classifiers that run for two hours on a desktop, right? And don't require a data center the size of New York City.

Speaker: um So yeah, right size your applications, right size your focus, right size your compute. And you can probably do things that you have never imagined before. And maybe even more so if you got this edge computing cluster that you can drag out into the field.

Speaker: Yeah. And the my my question kind of rolls down to, you know, are you actually right-sizing the investment that you're making and trying to participate in these edge technologies and the AI space? Like, are your use cases correct? Are you actually, you know, spending money in a way that you can quantify? Because I think, you know, do you need to spend all the money on a big large LLM procurement and all the bells and whistles that come with it or can you find a local McCain case know Canadian manufacturer like Jeff or

Speaker: If you're in America, a local you know state level manufacturer, someone who's not just representing the big conglomerates, someone who's a local innovator trying to do good things, can you find someone with a solution? and Are you willing to give them a chance?

Speaker: Because I think there are good people out there like Jeff who are trying to create environmentally conscious, sustainable technologies that are not purely profiteering, that are not trying to to gouge investors or gouge customers.

Speaker: And I think, you know, they. Or just speculate because it's a real fucking object in three dimensional space. They're doing they're doing good things and they're good for your business. They're good for the environment.

Speaker: And for God's sake, just stop giving your money to the empires. um Just think about that. That's the that's the question. All right. Take that forward and we will see you next week.

Speaker: If you like this conversation, share it with friends and subscribe wherever you get your podcasts for a weekly ballistic payload of snark, insights, and laughs. New episodes of Bare Knuckles and Brass Tacks drop every Monday.

Speaker: If you're already subscribed, thank you for your support and your swagger. Please consider leaving a rating or a review. it helps others find the show. We'll catch you next week, but until then, stay real.

Speaker: Yo, this is Bare Knuckles and Brass Tacks. I am I messed it up. I'm going to do again. it again but the blooper reel.