Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Data Centers Draining Our Energy: What Can Be Done?
41 Plays5 months ago
In this enlightening episode of Green IT on a Cloud, host Rob Gillam sits down with Sean Varley, Chief Evangelist and VP of Business Development at Ampere, a company renowned for its efficient, scalable, and sustainable data center and edge solutions powered by cloud-native processors.
They explore the intersection of AI, cryptocurrency, and sustainable technology, discussing how these innovations are reshaping the energy landscape. Delve into the challenges posed by data center energy demands and discover the innovative solutions driving the industry towards a greener future. Tune in to hear Sean Varley's expert insights and learn how Ampere is leading the charge in promoting eco-friendly IT practices.
Recommended
Transcript
Introduction to Sustainable Practices in IT
00:00:04
Speaker
Welcome to Green IT on a Cloud, a podcast that celebrates the success stories of sustainable practice in IT. Going green does not have to mean business sacrifices. In fact, cutting waste can make your company lean, efficient and more productive. In this podcast, we feature interviews with experts and innovators who are shaping the landscape of sustainable technology. In today's episode, we're going to explore the fascinating intersection between technology and sustainability.
Introducing Sean Varley and His Role
00:00:33
Speaker
Joining me is Sean Varley from Ampere Computing, a company that's well known for its cutting edge CPU technology. Hey, thanks for joining me, Sean. It's great to be here, Rob. So it would be great if you could just give a little bit of an intro and a bit of a background. Absolutely, yeah. So I came to Ampere about five years ago. I currently am the Vice President of Business Development and Chief Evangelist for the company.
00:01:01
Speaker
which is really a fancy way to say that I kind of look after our ecosystem and platform readiness, right? Like all of the pieces that go around the Silicon platform that help us to go forward in the market and really tell our story to the world and to our partners and customers.
00:01:24
Speaker
That's great. So we kind of had a chat before, and I know you've got a pretty broad background in processing development.
Why Power Consumption Matters
00:01:31
Speaker
So it'd be really interesting to get your view on what drivers you see influencing your part of the industry to move towards sustainability. Yeah, thank you for the question. That's a really good question. As I kind of look back on it,
00:01:47
Speaker
And I spent many years at Intel before I came to Ampere. And this is an industry that really didn't have a concern about power for decades. The years of Morse law and the way that
00:02:03
Speaker
Moore's law was giving us performance and essentially doubling performance and halving cost every two years, which was kind of the foundational aspect of that law, kind of gave us some bad practices or at least biases and sort of well-worn paths that were essentially lending themselves to the economics of that industry.
00:02:32
Speaker
And that kind of sort of willful, you know, following of that path really kind of led to lack of attention to the power that was being consumed by these devices. And so it sort of kind of created the industry that we have today, which is now an industry that has sort of runaway power.
00:02:56
Speaker
And, and, and has now bumped up against thermal, you know, other thermal laws of physics that, that are not allowing us to, you know, kind of go any further in performance and, and overall, you know, kind of functional capacity without really radical changes in the way that we
00:03:18
Speaker
Essentially dissipate heat in these platforms through ever more esoteric and more complicated ways to deal with the heat that is produced by these chips so what i find today is that you know that sort of lack of attention for decades.
00:03:36
Speaker
has now kind of created in many areas, a greater awareness for how can we do this differently?
Power Demands on Data Centers
00:03:45
Speaker
And how can we start saving power and paying attention to the power consumed by these massive machines that are doing all this sort of background computing for the cloud and the internet?
00:04:02
Speaker
And that is, I think, I mean, probably get more into this, is really pressured by the overall data center power draw and concerns around that power drop in especially many municipalities and regions of the world.
00:04:19
Speaker
because data centers are becoming such a huge consumer of power that they're having trouble supplying that power on grids. And also, there's lots of debate about the power being taken away from other commercial or residential concerns. So it's really starting to ratchet up the pressure. You really see this kind of pressure coming regionally, but it's really a global concern.
00:04:47
Speaker
I can remember going back a few years, I was at Rackspace, the power capacity of the data center was pretty much calculated on 4U servers at that time. So when they went down to 2 and I guess they're even smaller now, basically ended up having empty halls because it simply couldn't provide
00:05:05
Speaker
the power to fill the halls up with the density as the density went up. So it's an interesting challenge. You mentioned something about industries. Other industry, everybody hears about AI and obviously these massive GPUs and also crypto and the amount of power that they're consuming trying to do that is crypto mining.
Sustainable Solutions in AI and Crypto
00:05:29
Speaker
Are there any industries that are coming to you or you are reaching out to that you're seeing as this is where we can really make a difference in terms of the sustainability?
00:05:41
Speaker
Yeah, there's two ways to answer that. There's people that are coming to us and then there's industries that are scaring people so bad that they're looking for any kind of solution to help. You mentioned one of those, which is AI. I think the projections now around, these are fairly, I guess,
00:06:05
Speaker
balanced perspectives like the International Energy Agency, the IEA just came out with one that kind of forecasted the power requirements for
00:06:14
Speaker
the build out of the AI age, showing more than a doubling of data center power consumption in the next five years, just to try to stay on track with where AI is. And just put that into perspective, doubling the consumption of data center power worldwide is really just not feasible.
00:06:37
Speaker
in that amount of time, right? It takes time to generate new sources of energy, especially new green sources of energy. And so that industry has really put a real fright into data center operators, new service operators on how we could actually even sustain a build out for that kind of power consumption.
Telecom's Power Expansion Challenges
00:07:03
Speaker
And we'll probably talk a little bit more about why AI is so power intensive, but to come back to your another part of your question, which is, you know, who's coming to us? There are industries that are absolutely coming to us, including data centers, you know, but as I said, that interest is, is, from my perspective, very regional.
00:07:22
Speaker
If you're an island nation, you know, maybe on the Pacific Rim, there are many of them, Pan, Singapore. You can kind of think of South Korea as a bit of an island nation.
00:07:33
Speaker
You know, all of these types of areas, Europe in general, which have experienced a lot of cost increase first, you know, the price of energy really skyrocketing are coming to us in the data center space saying, hey, we don't have space or power. You were mentioning space earlier, and that's another aspect of this whole thing. But those sorts of industries are coming to us saying, hey, how can we significantly
00:08:03
Speaker
cut or manage the amount of power that we're consuming for any of these workloads, really. And then the last one I'll mention are telcos worldwide. Telcos worldwide, if you think about it, have to essentially provide radio service for their customer bases wherever their customers are.
00:08:25
Speaker
And so they have very little choice in putting that IT infrastructure out in areas where they have very little leverage over the utilities and the cost of their power. And of course, in all of those regions, that cost has gone up drastically in the last three to five years.
00:08:49
Speaker
they're facing really kind of a crisis where the cost of them being able to deliver that service, I'm talking about like cellular service worldwide, has been increasingly radically and they can only pass so much of that on to the consumer.
00:09:06
Speaker
So they're getting pinched really hard and they're coming to us saying, hey, we really have to drop the power consumption, especially of these newer technologies like 5G and in the future 6G. We have to drop it because the technology gets more intense, requires more compute and of course then more power.
00:09:26
Speaker
and they're faced with, you know, sometimes hundreds of thousands of sites that are all drawing power to deliver this service. So you see it in many different areas, but there's a few of them. Yeah, that seems that's interesting that it is, you know, quite, you know, that there are industries that are certainly, you know, challenged to be greater than the others that, you know, the telco industry has always been, you know, quite a resource-hungry industry.
00:09:53
Speaker
Everybody wants more connectivity, so the challenge just forever grows. It's interesting you're talking about different areas.
Collaborative IT Sustainability Efforts
00:10:01
Speaker
One thing that's interesting to me is, I've spoken to a lot of organizations involved in sustainability, and one of the benefits that a lot of organizations see is collaboration. So working with, you're a part of the solution, but it's a big problem and it takes
00:10:17
Speaker
many people to solve a big problem. So I'd be interested in other collaborations with other organizations in the industry that are all working towards green practices in IT.
00:10:30
Speaker
Yeah, there really are. I've got several of these sorts of areas. And first off, the collaborations are all over the entire ecosystem of building that platform I mentioned earlier that's part of my responsibility at the company is from silicon all the way up to the application, which is the bookends of a technology stack, so to speak, you really have
00:11:00
Speaker
all sorts of areas to impact a greener perspective. Even software has a role to play here. Just think about us if you were able to create software that consumed a lot less memory. Well, memory, especially as we get faster and faster in memory speeds, DDR3 to DDR4, DDR5, every generation consumes more and more power. If you can do things in less memory, if you can take
00:11:30
Speaker
essentially collapse your footprint. All of those things have meaningful impact on the amount of power drawn by any given application. But there's a lot of collaboration, especially focusing on silicon and hardware, doing things more efficiently. One of the classic debates here is general purpose processing versus, say,
00:11:54
Speaker
acceleration or very purpose-built processing, which you see in the form of accelerators. You can think of like a GPU as an accelerator, but there's also even more power-efficient accelerators that do various things. One I would kind of highlight is transcoding. There's so much video on the internet today in any typical online service,
00:12:22
Speaker
that now translating and transcoding video to end point devices for the consumers viewing pleasure becomes a real computational challenge because there is a lot of essentially math going on behind
00:12:39
Speaker
compressing that video and transcoding it for streaming. And so we actually partner with a number of different companies here that do accelerators for video and for video of transcoding. You know, you find these almost anywhere, you know, backing up big services like YouTube or Meta, you know, any of these large video oriented social media platforms and things like that.
00:13:09
Speaker
So collaborating with those types of companies to offload the computational capacity needed for transcode of video is part of the thing. And then what you can do with the CPU cores, the general purpose CPU cores that you freed up from having to do that task is then layer on new technology.
00:13:31
Speaker
Right. So it's kind of a question of do you bank the savings that you got by reducing the power of the computational aspect or then do you add functionality? And so this will be a dynamic that is constantly probably being worked by different
00:13:52
Speaker
different service providers and application developers worldwide as we go forward. If you can save some of the power on one thing, then maybe you can spend your savings elsewhere. Maybe doing AI, for example, on a video stream.
00:14:14
Speaker
An example that we actually do an active demo on now is we use cycles saved from the transcoding to actually do closed captioning, right? Voice to text, closed captioning with the general purpose CPU processors that were saved by offloading that transcode.
00:14:33
Speaker
So lots of different areas to kind of think about, but one of the main ones, and I predict that this will be a huge part of the evolution of silicon in the next five to ten years, is the combination of purpose-built processing for specific tasks like transcode that I mentioned,
00:14:54
Speaker
or AI or other things, and then general purpose. Those two things combined together in different weighted amounts will be kind of the balancing act of combination products going forward in silicon. That's interesting. So the consumer will look at which areas they're consuming the most, and they will get a silicon specifically made to balance their loads out, is that how you see it?
00:15:23
Speaker
Exactly. Yep. That's exactly right. Yeah. And any, you know, there's, there's many areas to be, to be accelerated essentially. Right. Anytime you come upon, like I like to tell people when I'm working with them, you know, rule of thumb is, is if you're spending over 30, 35% of your time in computational time, I mean, and doing one task, then you should consider looking at some sort of an acceleration.
00:15:49
Speaker
You know, if it's less than that, then don't bother. Just use general purpose processing is far more flexible, far more malleable, and it's not really worth looking at the cost and effort of doing particular accelerations. But if you're over that, it's probably well worth your time to start looking at offloading that type of a task.
00:16:11
Speaker
That's interesting. We kind of touched on the AI and some of the emerging trends that are challenging us.
Ampere's Focus on Efficient Cores
00:16:21
Speaker
Are you seeing in the green IT space, what do you see coming along? Your things are moving so fast, I guess. How do you keep ahead? How are you making sure that you are ready for what the next challenge is going to be? Yeah, foundationally, our company
00:16:39
Speaker
Being founded to essentially provide sustainable computing sort of views our role in this as a fundamental differentiator or fundamental mission, which is to build ever more efficient cores from a power perspective, right?
00:17:00
Speaker
So in efficiency in this case is measured by the amount of compute it requires to do anything essentially divided by how much power it takes to do that computing. So that comes out in all sorts of metrics, queries per watt or spec rate value per watt or many other types of throughput analysis per watt. And we are fundamentally,
00:17:29
Speaker
Always concerned with delivering more of that throughput per watt through our cores. And then our other sort of philosophy on this is to scale out. In other words, we add cores instead of pushing more out of one core.
00:17:48
Speaker
Why? Because pushing more out of one core typically increases. There's only almost one way to go with that, and that's increasing the power consumption of that core. Because you're going to push it for performance by either increasing frequency
00:18:06
Speaker
which is going to drive the power up radically. And it's not a linear thing, right? Once you get past, this is an exponential or logarithmic curve. And so you get past the knee of that curve and you start to increase power radically after a certain point in pushing frequency.
00:18:25
Speaker
And then if you push it in other ways, like higher parallelism or greater vector width and things like this, greater instruction depth in cues and things like that, then you're adding more and more silicon area. And so that's going to consume power as well. So our philosophy is to
00:18:43
Speaker
build the most efficient core you can and then provide a lot of those cores. And that's kind of what we call cloud native computing because that's a scale out method of getting performance out of a processor.
00:18:58
Speaker
So that's not unlike, by the way, GPUs. They actually have very small cores, tensor cores that are much smaller and they're built very purpose built for essentially matrix multiplication math and things like that. So these are specialized cores, but more in the line of smaller and more of them.
00:19:22
Speaker
And so that's an area where you see, trending-wise, because AI is so computationally intensive, this sort of need for more cores, but then you can ask yourself, well, why is AI such a power consumer? And this starts to become a balance, right? Because GPUs are also, not only do they have a lot of cores, in many cases, sometimes,
00:19:48
Speaker
thousands or tens of thousands of them in one chip. They're also very, very memory intensive. So there's a lot of data being moved in and out of these devices, which all of these things increase the power draw. So AI now becomes this looming power consumer in the compute environment. And we're only at the very front end of this AI wave because
00:20:17
Speaker
Up till very recently, the AI tasks have been building models, which is a thing called training.
Managing AI's Power Consumption
00:20:26
Speaker
Your training models and that's very, very computationally intensive and requires a lot of precision. So those sorts of tasks have now built these massive models that are changing the world. These Gen AI models like chat GPT and others that are building this incredible capacity for machine learning and also machine decision-making.
00:20:53
Speaker
But what you're seeing now is now the transition between the making of those models and the use of those models and then when you get into the use of those models, this is a task called inference.
00:21:07
Speaker
and inferences where the decisions are made, this requires less precision and it also requires less computational horsepower to really do the business of decision making because you're essentially querying a model that has already been built.
00:21:26
Speaker
And so this is an area where we can start to balance the computational requirements needed to do the inference tasks. And that is much more modest type of action. It doesn't require quite the computational intensity in that precision that has been built for training.
00:21:52
Speaker
And this is where we have been telling the market, look, you can do this inference on
00:22:01
Speaker
a general purpose processor just fine for certain types of models. And then if you do need to get to higher and higher throughputs and batch sizes and things like this, then you can use an accelerator, right? And an accelerator doesn't necessarily have to be as power intensive as what you find with today's GPUs. So there's a lot of nuance that has not really been, I think,
00:22:28
Speaker
fully vetted or debated or understood in the industry today about kind of this trend of AI and how AI will get used by enterprises around the world as this technology kind of curve builds out.
00:22:45
Speaker
Yeah, it's a really interesting discussion around where are we going with this technology? How do we best address it? It sort of leads in nicely to my question. We've talked about technology a lot. We're in the technology space, and technology is all excited.
Integrating Green Practices into Business Plans
00:23:03
Speaker
But one of the biggest challenges certainly I face, and I've spoken to quite a few people,
00:23:09
Speaker
He's trying to get business to adopt the greener practices. We can all see the value of AI and it's going to bring great benefits. But when we try and then couple that with, okay,
00:23:22
Speaker
build a business plan, where are you going to put your green practices within that business plan to enable your AI product to be more efficient as well as being better for the planet? I'm interested in how you kind of face those sort of challenges. What do you see from a business perspective when you're talking to business owners? What do they say when you come with the
00:23:44
Speaker
that we can save you power, but this is great for the industry. Is it just purely a dollars and cents conversation, or do you get that feeling that there is a move in the industry to perhaps do things which are more sustainable? There's definitely a desire to do greener practices. I am encouraged. I travel the world in my current role, talking to people in different
00:24:14
Speaker
countries and regions. And I am encouraged by the general sort of ownership of the problem that I see coming from a lot of the customer base that we deal with. And it could be somewhat self-selective because we are so such a part of our value proposition is around sustainable computing.
00:24:38
Speaker
Maybe the people that don't care don't take the meeting. All jokes aside, I do think that there is, I am encouraged by the uptake and the rallying point for us to try and do better at this.
00:24:55
Speaker
But I think when it comes down to really answering your question, it does come down to dollars and cents. Everyone has a budget. Everyone has a priority list. And trying to balance things that are challenging these businesses on a daily basis, there's always going to be that trade-off analysis of can I get to that thing, which I would like to do,
00:25:24
Speaker
Before i've got to do these things that are you fires that are burning and i can't i can't ignore. And those usually come down to service stoppages or getting to features that are gonna be more competitive because they're being outflanked by competition or.
00:25:42
Speaker
all these other concerns, but power is becoming a major concern, kind of rising up that list. And it may not be number one today, but it's certainly starting to be number two or number three. And this comes back to some things like you were talking about earlier, Rob, space constraints. As you look at these data centers trying to actually effectively use space,
00:26:10
Speaker
higher power consumption servers essentially leave major gaps in racks, right, which is a completely inefficient use of space.
00:26:20
Speaker
And one of the major costs of data centers is space, right? They're all tiled. And every rack will consume a certain number of tiles in a data center. And so every time you have to go spill over into another rack, it costs that data center in order to be able to provision that rack. So if you can do things in fewer racks,
00:26:42
Speaker
which is something that our company enables and we have metrics.
Cost Benefits of Sustainable Practices
00:26:45
Speaker
I would invite your audience if they're interested to go to our website and look at all those sorts of things because there are metrics here that help. But the fundamental space usage, power usage comes down again to dollars and cents. Because if you think about it, it just makes sense. It makes from a green perspective in two ways. You can consume less power, but you consume less money
00:27:09
Speaker
And of course, the United States, our money's green, so both are green. And that becomes the typical motivator, right? Saving power saves money and that becomes a much more enticing and interesting imperative for businesses worldwide as they look at making decisions about what to do next or what to do with their precious resources.
00:27:36
Speaker
That's great. And I think that's a good way to end because it does link how we sort of started this, which is going green should effectively make you more effective and should save you money. So anything that is focused on that is a good thing and it's helped me, it's taken me in the right direction. So it's a great way to finish it. Thank you so much for your time. I really appreciate your insights. It's been fantastic. I really appreciate you joining us today.
00:28:04
Speaker
You know, it's been my pleasure, Rob. I really appreciate it. And thanks for having me on the podcast. Yeah. If anybody is interested in learning more about these kind of green technologies, encourage them to go through you and your sort of podcast or come directly to us at Ampere Computing. Great. Yeah. Thank you for that. That's it. Thank you. Bye-bye. Thanks.