Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Perspectives: Scaling AI
724 Plays5 days ago
Arvind Krishna, Chairman and CEO of IBM, joins Stuart Riley, Group Chief Information Officer, HSBC, to explore AI – what it takes to scale it, the risk considerations, the opportunities for highest impact, and the cost profile. They also discuss expectations around quantum computing and the transformative potential of this technology.
This episode was recorded behind the scenes of HSBC’s Global Investment Summit in Hong Kong, March 2025. Find out more: grp.hsbc/gis
Disclaimer: Views of external guest speakers do not represent those of HSBC.
Recommended
Transcript
Introduction and Welcome
00:00:05
Speaker
Welcome to Perspectives from HSBC. Thanks for joining us. And now on to today's show.
00:00:14
Speaker
Welcome to this edition of HSBC's Perspectives. I'm Stuart Riley, Group CIO of HSBC.
Exciting Times in Technology
00:00:21
Speaker
And today I'm delighted to be joined by Arvind Krishna, CEO of IBM.
00:00:27
Speaker
Arvind, welcome. Stuart, it's great to be here with you. Nice to see you. Look, I guess you're right in the heart of everything that's
AI's Long-term Impact vs. Short-term Hype
00:00:33
Speaker
going on at the moment. And what a fantastic time to be in technology.
00:00:37
Speaker
don't know about you, but I find it so just a pleasure to work in this industry at the moment. So it's super exciting. I'll start, if you don't mind, by um using the old adage that in technology, people often and ah sorry overestimate in the short term and underestimate the long term.
00:00:55
Speaker
I feel like AI is going to make that adage truer than ever. i mean, how are you seeing that play out? Look, absolutely. To your point, people always overestimate what it could do in two or three years.
00:01:07
Speaker
A lot of that is to do with adoption and all of the inhibitors which we know are there in most enterprises and governments. And they dramatically underestimate the 10-year point, sometimes maybe the five or six-year point. yeah You know, i the internet is the classic example.
00:01:22
Speaker
I'm old enough to remember in 1995, people were predicting 10, maybe 100 million total adopters over the next decade, it's a river in the billions.
Phases of AI Development
00:01:32
Speaker
yeah But when they thought about what it could do in the first two or three years,
00:01:36
Speaker
Not quite there. yeah yeah Yeah, massively overestimated. Yes. Yeah. and um And so obviously, i mean, AI now, I guess most people think of generative AI really starting probably with the launch of OpenAI Chat GPT in what, 2023? So we're sort of two years into that journey.
00:01:56
Speaker
Where do you see that we are today and and the next steps? Yeah. So I'll use a baseball analogy. I know a lot of your audience doesn't follow baseball. Just think of it as a nine innings game.
00:02:08
Speaker
ah You know cricket is two innings, so it's a nine innings game. I kind of say is in the first innings. So you don't quite know how the game is going to play out, but you're on the field.
00:02:18
Speaker
is going, the players are out there, you know who's there, but you don't know who's hot, who's going to win, ah
Scalable AI Use Cases
00:02:25
Speaker
really. yeah By analogy, cloud is probably in the fifth or sixth inning, so it gives you some sense that's mature, you kind of know what's going to happen. yeah So that's where you are.
00:02:35
Speaker
But you've got to be on the field, you've got to be playing, you've got to be worrying about which use cases will scale. And I think from a big, long arc of technology, think of AI like the Internet.
00:02:47
Speaker
By the way, there's only eight big technologies. It is not that many. you know If you begin with the steam engine and think railways, go forward all the way to semiconductors, which enable computers, internet, AI.
00:03:01
Speaker
So it's kind of in that yeah big, big arc of importance. Yeah. And I've heard you say on numerous occasions, giving advice to CEOs and boards, suggesting don't experiment with hundreds or thousands of things, focus on the things that can scale.
00:03:17
Speaker
Can you give me examples of that? Sure. Look, so what happens is it's new, it's exciting, you read a lot about it and before you know 100, 200 experiments blossom all over the enterprise. yeah Now to make AI really give value to the enterprise, you gotta worry about your data.
00:03:34
Speaker
You gotta worry about putting some guardrails. You're gonna have some ethics around it. You go to integrated inside an application. All of that takes investment and takes expertise. Is it tens, is it hundreds of people?
00:03:47
Speaker
Well, a hundred times a hundred, I don't think you're gonna quote, spend 10,000 people on an experiment. So the reality is 90% of them are gonna fail. yeah That leaves a really bad taste.
AI in Risk Assessment and Professional Aid
00:03:58
Speaker
I kind of say it's mature enough. Understand which ones could scale and impact it at enterprise scale. Put all your investment behind those three to five. You'll learn.
00:04:09
Speaker
Now you'll get better at it. And by the way, two or three years down the road, you might have 100 going, but start with three to five. Yeah, kind of get some wins on the board. runs in the boat and scale. Scale is really important. Yeah.
00:04:21
Speaker
what What sort of examples are you seeing people focus on at scale? I think there is four use cases in buckets. Lowest risk, easiest one is customer service. No question about it.
00:04:33
Speaker
You know, like even in a bank, I'm sure there are people who call in to say change in address or there are people who call in and say, look, I forgot something around my account. Or they call in and say, I forgot just my online credentials.
00:04:45
Speaker
I think at least half, if not all of that could be handled with AI. yeah Those are, I call them low risk because you're not making a credit decision or something. yeah If you think it's a higher risk, that 10, 20%, it's not more than that.
00:04:58
Speaker
And your call center or in any online experience, want that to a human, but all lower, number one. Number two, I would say your enterprise operations, especially around the back office, you know invoicing, payments, but i what I call it HSBC paying its vendors, like even people doing maintenance in a building.
00:05:17
Speaker
yeah You do have that. All of those things I think could be automated at least half away with AI. Yeah. And in your main function, I think elements of automation, which I know you are a big fan of, Stuart, whether it's run books, it's code books, it's problem triage, programming.
00:05:36
Speaker
yeah i think you could easily get to see 20, 30% productivity improvement, yeah which allows you to handle all that tech debt a lot better. And the fourth one, which is unique to a bank, I think is around risk.
00:05:48
Speaker
worrying about um how do you onboard our customer, all of the paperwork that you have to do. yeah Why can't you take half that burden
IBM's Use and Future of Gen AI in Coding
00:05:55
Speaker
away? i think are the four buckets. You mentioned risk at the beginning of that in the context of whether deploying a Gen.AI model was risky in a context or not.
00:06:06
Speaker
How do you think about assessing that risk? like how do you How do you think about whether something is risky with a Gen.AI model? look I kind of think of Gen.AI as we're trying to mimic how a human being operates.
00:06:21
Speaker
I don't know about you, but in my experience, humans are not always great at having perfect recollection. They think they have it perfect, but when you go back to a record, they're often about only about 80% accurate, yeah yeah but they have complete conviction that they said that or they behaved like that maybe six months or a year ago. yeah Why would we expect Gen AI to be different?
00:06:44
Speaker
Okay, so that's the case. Now you have to go back and say, look, if it's customer service and if it's giving an answer, verify the answer. yeah The moment you have got a verifier built in, you're taking that what is called hallucination or I call being incorrect, you're taking it down from 20% down to less than one.
00:07:02
Speaker
yeah At that point, you're probably better than your human agent. Yeah. yeah It's not going to be perfect like 100, but it's going to be 99% accurate. So setting the benchmark as how does a human respond or behave in this situation and can we better that? Can we better than that? yeah And actually think of it this way, using Gen AI maybe in conjunction with a human, are you making your newest, most inexperienced professional up to the level of an experience? That's quite a win by itself. Yeah, yeah.
00:07:29
Speaker
And of course you free up people to do other things that are more valuable. complicated things and in the end, facing up to your customers is what you want people to be doing. Yeah, yeah. Spending valuable time with them rather than administrative or bureaucratic tasks,
Future Cost Reduction and Domain-specific Models
00:07:40
Speaker
etc. Correct. right Yeah.
00:07:40
Speaker
And you mentioned um coding assistance. I'm a huge fan of Gen.AI for coding. um Actually, I've used many of the models to to do my own coding in my spare time to to play with these models.
00:07:53
Speaker
How much efficiency are you seeing in IBM for your engineers? I'll give you three numbers. i know people get fascinated by the 30, 50, 90% that some people talk about. Yeah. I'm going to shock you.
00:08:04
Speaker
But this is across a base of 25,000 engineers. okay We are seeing 6% today. Okay. So pretty low by some of the numbers that get quoted. Right. But I like 6 because I think that that's an honest and real number as opposed to this thing. Yeah, the inflated. Right.
00:08:19
Speaker
I actually believe that that 6 will climb to 25 to 30 over the next one to two years. Yeah, not that long. Yeah, now that people are getting used to it and it gets to 25 to 30 by taking on a lot more languages, a lot more tasks, not just coding.
00:08:36
Speaker
So my 6% comes from just actually a very simple one and two line completion. yeah Yeah, yeah, sure. But I can get now to more procedures. I can get to full documentation. I can get to test case generation so I can see it climbing to about 30% quite well.
00:08:50
Speaker
whitewell yeah yeah Then I did an interesting experiment inside IBM. I asked people, what about patching? Because that's different than writing code. yeah yeah And suddenly they found amazingly, there was a little bit of work.
00:09:02
Speaker
It was really, really good at doing patching. yeah Because that's a very, very unique and specialized task, as you know. yeah And one that a lot of people don't exactly love. not fun. It's not fun. yeah But it's essential.
00:09:14
Speaker
Yeah. And there suddenly, I think there it could be 50 or 60 percent. actually Yeah. Yeah. So it's choosing the right task. It's choosing the right task. And if you think about application maintenance, which is patching, it's half the bank. A significant part of our expense for sure. Yeah.
00:09:28
Speaker
but Today, it's still like certainly very expensive to train a large model. Obviously, DeepSeat changed that picture a little bit, but it's still a pretty expensive capital outlay ah to train a large model.
00:09:39
Speaker
um Inference costs are coming down, but actually if you're doing it at scale, it's still significant GPU costs, e etc. What do you think happens to the cost profile in the next few years? I'm going to give you a prediction that will sound counterintuitive and surprising.
00:09:54
Speaker
I think in five years, the total cost will be 1% of today's cost. Wow. So 100 times cheaper. Wow. And I know a lot of people say, well, that's semiconductors. No, semiconductors normally gives you about 2x every two years.
00:10:09
Speaker
So that's maybe 5x over five years, not 100x. Yeah. But the semiconductors, competitiveness in the semiconductor market, because when you only have a few suppliers, prices tend to be high. Sure. I think more will come.
00:10:22
Speaker
then I think there will be a lot of algorithmic advantage. And the piece that very few people like to talk about, instead of using a hundred or a 500 billion parameter model, these are massive by any scale in technology. yeah I think you'll find that for 99% of what you want to do, a three, seven, 10 billion parameter model is actually better.
00:10:44
Speaker
yeah and with less hallucination than a big model. yeah That says you got to be able to use the smaller model. So small model, defined or distilled down, for a domain, specialized, locally deployed.
00:10:58
Speaker
Locally deployed and locally deployed means right there you get another 2x. Let's add it off. Semiconductors gives you 5x. Algorithmic gives you another 5x. Small model gives you a 5. ah Five times five times five is 125. Yeah, yeah, yeah.
00:11:11
Speaker
yeah yeah yeah Okay. And actually, ah again, you you sort raised an interesting topic because I think i thought the DeepSeek moment was was genius in many ways. What they achieved is is is amazing.
00:11:22
Speaker
But I thought it also reflected the fact that these things hadn't been optimized. you know There had been no need to that point to optimize. And so it really demonstrated to me that and the optimization journey is only just beginning.
00:11:35
Speaker
I want to poke a little bit of fun off my own industry, the tech industry. If you think that you can draw a big moat by convincing everybody, look, it's really, really expensive, don't bother to chase us. yeah There's no motivation to go optimize.
00:11:48
Speaker
If on the other hand, as tech has shown again and again, people might start that way, it's not like some other industries. then the barrier falls because once one proves it, then everybody runs after the... Everyone's running. simple That's why that was my middle five, the optimization piece.
00:12:03
Speaker
Yeah, yeah. and and And again, you touched on models getting smaller um and therefore cheaper to train and run, etc. Where do you see the balance between and training versus inference, you know, and and the shift potentially?
00:12:19
Speaker
The value for those who deploy, and that in the end is the only value that matters, is all in inferencing. So if over time, inferencing is not 10 or 100 times larger than training, you got a problem.
00:12:32
Speaker
yeah This is an implosion going to happen. So that's clear. But that may take five or 10 years to get to that point. So that's part one. The second point, I actually believe very strongly that on the large versus small, we are going to need a few large models.
00:12:48
Speaker
Think of them as teacher models or the ones you use when you have no idea what you might exactly be wanting. There's space for a half dozen, maybe a dozen, but that order.
00:12:59
Speaker
yeah It makes sense. They can get their return on tens of billions of investment. Yeah. And then there will be hundreds and thousands of much smaller models built using millions, not billions.
00:13:11
Speaker
Yeah. And i I would predict actually that on the small models, HSBC could well have maybe a dozen of the smaller models yeah that you use that are unique to
Quantum Computing's Potential Future
00:13:21
Speaker
you.
00:13:21
Speaker
Yeah. And that unlocks your enterprise data to be used in that context. Yeah, yeah. Do you think Quantum comes into all of this? Look, Quantum is the first time we have ah actually a new computing architecture. Mm-hmm.
00:13:33
Speaker
in 70 or 80 years. That's why it's so exciting. Even AI, it's really, really exciting, but it's built on digital deterministic bits. It's not actually that different.
00:13:44
Speaker
A funny way to say it is AI and computing is great. It looks at the past, it looks at data, and then it can do things that are remarkable, but they don't do the future. Quantum is going to help us design the future.
00:13:57
Speaker
So when we look at areas like materials, like risk, like optimization, quantum is going to give us techniques which we can't even dream of into those worlds.
00:14:07
Speaker
And so the things at risk, which you thought were just too expensive to compute, can be done. Yeah. And this is going to be here upon us quicker than we think. Like, i I would predict by the end of this decade, this, not next.
00:14:22
Speaker
I've heard you say that before, and and I'm going to play devil's advocate a little on this. I feel like I've gone through my, I've been in this business for sort of 30 years now. I feel like quantum's always been five years out. What makes you think that this five year out is closer? No, I don't think it's five. I think in 2015, we would have said it's 10 to 20. Don't know exactly, but it's 10 to 20. Yeah.
00:14:42
Speaker
it To me, this is engineering, so I'll just put some numbers on the table. okay You need a machine that is big enough, because if it's small, who cares? You can simulate it on something.
00:14:52
Speaker
So big enough. Two, it can't have too many errors, and quantum inherently will have errors. But you've got to get the error rate down to where it's very, very much more usable. Yeah. And three, quantum, because it's using tiny amounts of energy, tends to dissipate or become noise very quickly.
00:15:09
Speaker
So how long can the machine remain coherent? So that's three. There are not five, 10, 20 things, those three. ye fix those three and Fix those three and you're in business. So on the scale point, 10 years ago, people were talking about four, six, eight qubits in scale. Okay, who cares? That's a toy.
00:15:27
Speaker
Today, we are in the hundreds. We actually have 75 quantum computers on the cloud, not simulators, actual quantum computers. 13 of which are in the hundreds of qubits.
00:15:40
Speaker
I need to get hundreds into thousands, so a 10 times advantage. yeah Error rates right now are sitting around 10 to the minus three. Need to get them to 10 to the minus two.
00:15:52
Speaker
At that point, error correction works. At 10 to the minus three, 10 to the minus two is hard, 10 to the minus three we can get there. yeah And coherence time needs to improve by about another 10 times.
00:16:04
Speaker
We think these are actually three to four year problems. So I was going to say because when you look back over the last five, 10 years, the advancement that's been made is then roughly equivalent to the advancement that needs to be made once again to get there. Is that?
00:16:19
Speaker
No. No? I think we needed to do 10,000 times advance, if I take it over the last 15 years. Yeah. A thousand of which has been done. The last 10 is to be done. Okay.
00:16:32
Speaker
So that's why my confidence and convection. So if we've done a lot more. These have been improving about three times each year. Yeah. So actually, and and jumping slightly, but still in and in the theme of quantum.
00:16:46
Speaker
and given your prediction, we really ought to be very much today working on cyber techniques that are kind of quantum proof, which are readily available today. So we we know we have that technology, but but I think we need to be deploying those more rapidly given the prediction that you're making. Is that that your assumption? 100% Stuart.
00:17:08
Speaker
So you might have a lot of data that you think is protected today. But when quantum comes along, it may be able to read that quite quickly. yeah So it's not that it's going to sort of come backwards in time to today.
API Strategies and Enterprise AI Success
00:17:24
Speaker
But if you have something encrypted and you think it's safe because it's encrypted, yeah that could get read and that's a problem. um what What haven't I asked you about AI that I should have asked you? Is there anything?
00:17:35
Speaker
Look, I'll just mention one thing always. I think the scale point we mentioned and talked about briefly is really, really important. Pick a few that can scale because the learning inside the enterprise of then how to do it is really, really important.
00:17:49
Speaker
yeah And really focus on deployment much more than the invention side. Yeah, yeah, yeah. And I guess in the in the world, everyone's using this term agentic today. um you know, I find agentic, it's a useful term in a way, but I mean, what it really means is that the AI model is calling other APIs to to enact something.
00:18:08
Speaker
Do you see that as a significant development or just a natural development of of where we were heading? ah To me, it's natural. Because I'll use the word, whether we use the word process management or we use the word workflow.
00:18:21
Speaker
Yeah. In the end, no application was ever a single program. It was always a workflow that went across multiple systems. Yeah. And the reason we decompose it, because as good engineers, you've learned a long time ago, decompose each task into something that is not too small, but not too big. Yeah. Because then it's fixable. Manageable. And now you can put it all together to do something that is a lot more remarkable.
00:18:41
Speaker
Yeah. Yeah. That's what EGENTIC is. So it's not a surprise. To me, it's a natural development. Yeah. But we should also be careful because EGENTIC can also go ory and bring up costs pretty high.
00:18:54
Speaker
Yeah. So we should also be careful at where we begin to apply it. But if you put it in a constrained box, I want you to worry about getting a mortgage application approved. Don't make every decision, but can you do some of those pieces using a GenTech? That seems a natural use case. So you're breaking down specific processes and looking at which part of that process makes sense.
00:19:14
Speaker
Correct. And I guess this um this highlights the importance of an API strategy. So for for the audience are not familiar with API, we're talking about application programming interfaces. So effectively how one piece of software talks to another.
00:19:27
Speaker
But if you're a company that doesn't have a great API strategy, doesn't have great APIs, it's going to be pretty hard to take advantage of this type of technology, isn't
Conclusion and Partnership Highlights
00:19:36
Speaker
it? think you're going to end up at a competitive disadvantage to those who already have that strategy in place yeah and can begin to use it to be able to deploy AI and agents both.
00:19:47
Speaker
Yeah, yeah. mean, this is where I see a significant advantage for HSBC because we've got the breadth of services and products that already exist. We've got APIs. And so we've now got this ability to digitize those businesses fairly rapidly.
00:20:03
Speaker
Well, look, thank you so much for joining us today. It's been a real pleasure to speak to you. As I say, the engagement with IBM has been fantastic. So we look forward to continuing to partner with you. Stuart, it's always great to talk to you. Thanks.
00:20:14
Speaker
i mean Good to see you. Thank you. Thank you for joining us for this episode of Perspectives. Make sure you're subscribed to HSBC Global Viewpoint to stay connected.