Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
The Macro Brief – Putting AI through its paces
45 Plays3 months ago
Mark McDonald, Head of Data Science and Analytics, examines the latest applications for AI - from analysing speeches to hosting podcasts. Disclaimer: https://www.research.hsbc.com/R/101/jsBFmVn. Stay connected and access free to view reports and videos from HSBC Global Research follow us on LinkedIn https://www.linkedin.com/feed/hashtag/hsbcresearch/ or click here: https://www.gbm.hsbc.com/insights/global-research.
Hosted on Acast. See acast.com/privacy for more information.
Recommended
Transcript
Introduction to Global Insights Podcast
00:00:02
Speaker
Welcome to HSBC Global Viewpoint, the podcast series that brings together business leaders and industry experts to explore the latest global insights, trends, and opportunities.
00:00:13
Speaker
Make sure you're subscribed to stay up to date with new episodes.
00:00:16
Speaker
Thanks for listening.
00:00:17
Speaker
And now onto today's show.
00:00:24
Speaker
This podcast was recorded for publication on the 9th of January, 2025 by HSPC Global Research.
00:00:29
Speaker
All the disclosures and disclaimers associated with it must be viewed on the link attached to your media player.
00:00:34
Speaker
And don't forget that you can follow us wherever you get your podcasts.
00:00:36
Speaker
Just search for The Macrobrief.
AI's Influence in 2024 Financial Markets
00:00:42
Speaker
Hello, and a very happy new year to all our listeners.
00:00:46
Speaker
I'm Piers Butler, and welcome to The Macrobrief.
00:00:49
Speaker
The hype around artificial intelligence was one of the big stories in financial markets in 2024.
00:00:55
Speaker
And it looks like it will be more of the same this year as the technology continues to take big leaps forward.
00:01:01
Speaker
So on today's podcast, we're putting AI through its paces.
00:01:04
Speaker
I'm joined in the studio by Mark McDonald, head of data science and analytics, who set AI a series of tasks, pitting it against humans, including me.
00:01:15
Speaker
We're also going to take a look at how the rapid growth of AI could affect the outlook for venture capital this year.
00:01:20
Speaker
So Mark, welcome back to the Macrobrief.
00:01:22
Speaker
Thank you very much for having me back on.
00:01:24
Speaker
So yes, pitting AI against the humans.
00:01:28
Speaker
We talked about a recent piece of work where you actually set AI against one of your team members.
00:01:33
Speaker
little bit in competition in a way.
00:01:36
Speaker
And there were some interesting conclusions on that in a way that it was quite positive, but AI could go in a complete tangent and so one had to be careful about some of the results.
00:01:45
Speaker
And actually it was the combination of AI and the human that produced the best results.
00:01:49
Speaker
Yes.
00:01:49
Speaker
So what have you done this time?
00:01:51
Speaker
Yes, this one's a bit different.
00:01:52
Speaker
Again, we're trying to see to what degree you can use AI practically in financial market research.
00:02:00
Speaker
This one is different.
00:02:01
Speaker
The previous case, we were testing out AI's ability to do data science.
00:02:05
Speaker
is it coming for my job?
00:02:07
Speaker
This time we kind of broadened out the scope and so is it coming for other people's jobs perhaps?
00:02:12
Speaker
And we set it the task of following the Fed.
00:02:15
Speaker
The piece was called Can AI Follow the Fed?
Analyzing Federal Reserve Speeches with AI
00:02:18
Speaker
Now obviously to some degree the answer is definitely yes.
00:02:22
Speaker
It would only take a fairly small amount of experimenting with feeding a Federal Reserve speech into ChatGPT to see that it can often do a very good job of this.
00:02:31
Speaker
But similarly, it would only take a little bit further experimentation to see that sometimes it does a much poorer job and sometimes you give it the same speech twice and it gives you completely different answers.
00:02:43
Speaker
And so really for something to be useful from a business perspective, you need to be able to engineer it into a sort of reliable and repeatable process.
00:02:53
Speaker
And so that's what we were looking at here.
00:02:55
Speaker
So we were trying to build an AI tool that we would give it a Federal Reserve speech and it would eventually analyze it along the directions of is it hawkish, dovish or neutral.
00:03:08
Speaker
It would give a summary of the monetary policy discussion that was in there.
00:03:12
Speaker
It would assess how relevant it was to the monetary policy outlook and it would explain its thinking.
00:03:18
Speaker
But rather than having kind of speech in and analysis out and trying to do it in one step,
00:03:23
Speaker
We instead split this into two steps and would rely on AI's ability to turn text into structured data.
00:03:31
Speaker
And this helps you to kind of corral the output, corral the output of the AI.
00:03:36
Speaker
in a more reliable manner, and it enables you to sort of engineer the task
AI's Performance and Human Comparison
00:03:42
Speaker
a bit better.
00:03:42
Speaker
So that's kind of why we went for this slightly more complex approach.
00:03:47
Speaker
So I guess, Mark, the big question is, did it work?
00:03:50
Speaker
Yes, I mean, at a high level, I would say the results are pretty good.
00:03:55
Speaker
We got Ryan Wang, our US economist, to do a compare and contrast and to analyze some of the same speeches himself manually.
00:04:05
Speaker
and to assess the outputs of the AI tool on the same task.
00:04:10
Speaker
And his view was that it was already performing at the level of a hard-working junior employee.
00:04:16
Speaker
And that is quite impressive.
00:04:18
Speaker
Despite this, he's not concerned that it's coming for his job just yet.
00:04:23
Speaker
And part of that is a function of the way that AI works.
00:04:30
Speaker
When you compare the AI performance to the human performance, what you see is, as I alluded to earlier, you can ask AI a question, then ask it the same question and get a difference.
00:04:41
Speaker
And it won't be the same.
00:04:41
Speaker
Yeah, exactly.
00:04:42
Speaker
So is this this issue of consistency?
00:04:44
Speaker
Consistency, yes.
00:04:46
Speaker
And it's one of the things that when people see an AI tool give what look like inconsistent answers, that worries people because we're used to the idea of computers being deterministic.
00:04:58
Speaker
Just so you interrupt, I was sort of listening to other podcasts about AI, in particular, self-driving cars.
00:05:03
Speaker
And everybody's terrified about self-driving cars because they can occasionally have accidents.
00:05:07
Speaker
Yeah.
00:05:08
Speaker
In a way, are we judging the AI too harshly because humans have lots of accidents in cars and actually statistically speaking, self-driving cars using AI are actually quite safe.
00:05:19
Speaker
So similarly, I mean, humans give inconsistent answers to data.
00:05:23
Speaker
So are we a bit too obsessed about this consistency point?
00:05:26
Speaker
Precisely.
00:05:28
Speaker
Obviously, the AI would give different responses on multiple runs of the AI on the same question.
00:05:33
Speaker
But we asked five of our colleagues to do a sample of the same analysis that AI was doing and it showed even worse variability from one human to the other than from one run of the AI to the other.
00:05:48
Speaker
And so I think what AI is doing a good job of is it's replicating human performance.
00:05:53
Speaker
The average output of the human and the average output of the AI were very similar.
00:05:57
Speaker
when aggregated over several instances, and the variability was comfortable from one to the other.
00:06:05
Speaker
However, we're fine with the idea that you can ask one human to analyze a piece of information and ask another human to analyze the same information and get different answers.
00:06:14
Speaker
And we're fine with that.
00:06:15
Speaker
But not with AI?
00:06:17
Speaker
Well, because if you ask the humans again tomorrow, you get the same answer.
00:06:20
Speaker
Right.
00:06:21
Speaker
And that's the key.
00:06:22
Speaker
It's the same person.
00:06:24
Speaker
And at some point down the line, if one of them capitulates on their view and changes, that capitulation is informative in and of itself.
00:06:31
Speaker
Whereas with asking AI to give you advice, that's very tricky because it will give you a sample of plausible advice from humans.
00:06:40
Speaker
And unless you're in the situation where you'd be happy to phone up 1-800-Strategist and get matched with a random strategist,
00:06:46
Speaker
then you're probably not going to be happy with an AI-generated opinion and AI-generated advice.
00:06:51
Speaker
And so I think that sort of use case for AI, it seems like these tools are particularly ill-suited to that.
00:06:58
Speaker
And that's not something that's going to change as the AI tools get better.
00:07:02
Speaker
It's not a performance issue.
00:07:03
Speaker
They're actually already performing very well at doing, on average, what good humans
AI-Generated Podcasts: The Future of Content Creation
00:07:07
Speaker
do.
00:07:07
Speaker
And also, strikingly, it's the speed at which they do it now.
00:07:13
Speaker
You're not going to answer this question because somebody else is.
00:07:16
Speaker
And you sort of did something that's a little unsettled to me ahead of this podcast.
00:07:21
Speaker
But maybe we should just let people listen to what you created by, again, using AI, which is an AI-generated podcast.
00:07:30
Speaker
Yes, this is a fantastic and free tool created by Google, which allows you to upload a document or even a series of documents and on the fly generate a fully AI generated podcast about the topic.
00:07:45
Speaker
Okay, so let's hear a quick clip.
00:07:47
Speaker
Think about it.
00:07:48
Speaker
You could use this AI to quickly analyze all the Fed speeches, all the reports, and highlight the most relevant points for you.
00:07:58
Speaker
I see where you're going with this.
00:08:00
Speaker
Right.
00:08:00
Speaker
That would be super helpful for anyone trying to stay ahead of the curve.
00:08:03
Speaker
Exactly.
00:08:04
Speaker
On monetary policy.
00:08:05
Speaker
Exactly.
00:08:05
Speaker
Imagine having this AI analyze every Fed speech in real time.
00:08:10
Speaker
Wow.
00:08:10
Speaker
Flagging potential shifts in policy before they even make headlines.
00:08:14
Speaker
Right.
00:08:15
Speaker
That kind of advantage.
00:08:16
Speaker
Yeah.
00:08:17
Speaker
Could revolutionize how we approach investing.
00:08:19
Speaker
Wow.
00:08:21
Speaker
I've listened to it already once before, but it's still striking as to how good it is.
00:08:26
Speaker
And also how enthusiastic they sound.
00:08:29
Speaker
Enthusiastic, clear, picks up on this point about real-time analysis of Fed statements.
00:08:37
Speaker
So very powerful when you think there's a whole army of people listening to the Fed.
00:08:40
Speaker
that potentially using an AI tool, you could do it quickly, you know, fast results that people could trade on.
00:08:48
Speaker
And in fact, I think there's another piece of work that you've done that takes that advantage.
00:08:52
Speaker
But before we get to that, is this the end of me and you as podcasters?
00:08:58
Speaker
That is a great question, because I think often when people see things like this technology with these AI-generated podcasts,
00:09:05
Speaker
An obvious question is, what's the point of that?
00:09:08
Speaker
Is the idea that eventually these podcasts that are generated by AI get good enough that you don't need humans doing it anymore?
00:09:16
Speaker
Maybe that's possible, but really I think the bigger change here is there is some minimum threshold below which there's a number of listeners below which you wouldn't bother doing a podcast.
00:09:29
Speaker
We'd be very upset if we only got one listener to this podcast.
00:09:32
Speaker
That would be seen as a massive failure.
00:09:34
Speaker
Whereas, you know, with an AI-generated podcast, it allows you to generate a podcast that literally only one person in the world cares about.
00:09:41
Speaker
You can upload a document that's only really of relevance to you.
00:09:44
Speaker
You're the only person who wants to hear that podcast.
00:09:47
Speaker
And that is fine.
00:09:48
Speaker
And this, in the same way that when, you know, Amazon was able to have this incredibly long tail of books that were available for people to
00:09:57
Speaker
to buy, that completely changed the structure and the economics of the book selling market.
00:10:03
Speaker
And there's a good chance that developments like this will completely change the economics and the setup of the podcasting market.
00:10:10
Speaker
Because it's literally a question of loading up the content and saying, produce a podcast.
AI in Financial Markets and Venture Capital Outlook
00:10:15
Speaker
Yes, exactly.
00:10:16
Speaker
And at the moment, it's always the same two speakers that are generated by AI, and it only does it in English, but clearly all of that is going to change.
00:10:25
Speaker
So let's come back to following the Fed.
00:10:27
Speaker
So we can see it's a powerful tool.
00:10:30
Speaker
There are some issues regarding consistency, but in terms of the speed of analysis, your other piece of work seems to demonstrate that it can be used for trading strategies?
00:10:40
Speaker
Yes, definitely.
00:10:41
Speaker
So we did some analysis with our FX team, and they were looking at can they take the output generated by this AI tool and use it to help with investing in either the rates or the FX markets.
00:10:55
Speaker
And what they found is that, you know, there are signs that if you aggregate over a short-term window the signals from this tool on recent speeches, then it does a good job of tracking the aggregate hawkishness and dovishness of the Fed.
00:11:11
Speaker
And in some instances, that has led important changes in rates.
00:11:17
Speaker
However, there are many situations where
00:11:21
Speaker
The Fed and the market might disagree on the likely direction in which rates are going to go, and sometimes when that happens the Fed turns out to be right, and sometimes the market turns out to be right.
00:11:33
Speaker
So in and of itself that doesn't feel like a natural use case for a trading strategy.
00:11:38
Speaker
You need more than that if you want to use this at low frequency.
00:11:42
Speaker
what is reliable is looking at short-term price moves after the speech happens.
00:11:48
Speaker
So what they found was that looking at very short-term moves over the minutes following a speech in the FX upmarket, then if you follow the direction in which the AI tool said, whether it was hawkish or dovish, then that led to a reliable price move in the FX market.
00:12:07
Speaker
And I think speed
00:12:09
Speaker
is really of the essence there.
00:12:10
Speaker
These tools can process these speeches much quicker than a human, and that can be helpful from a trading perspective.
00:12:17
Speaker
Well, we'll see where that takes us, but I wanted to just come back to another piece of work.
00:12:23
Speaker
I mean, this is the regular survey that we do, the Funding the Future survey.
00:12:27
Speaker
I guess not surprisingly, in the latest edition that came out just before Christmas, AI is still a big factor there.
00:12:34
Speaker
There's a lot of enthusiasm.
00:12:35
Speaker
But the survey generally is,
00:12:37
Speaker
is unusual in a way that it's all the respondents, be it private equity and venture capital respondents or institutional investors, which we also survey, are all enthusiastic about the market.
00:12:47
Speaker
Yes, the overall sentiment in that survey in this most recent one seemed very positive.
00:12:54
Speaker
I believe it was 83% of respondents saw an increase in venture capital and PE activity.
00:13:00
Speaker
over the next year.
00:13:02
Speaker
So overall it was very positive.
00:13:04
Speaker
One of the big drivers in here is clearly AI.
00:13:07
Speaker
The most commonly cited tailwind given by survey respondents was the rapidly changing technological abilities, which is clearly related to AI.
00:13:19
Speaker
And AI was by far and away the most commonly cited sector in terms of likelihood of an exit and the areas which are most likely to see continued investment in coming quarters.
00:13:32
Speaker
What's interesting about AI is, in a way, one has to think about it as a little bit as a sort of cross-sectional type of influence, because one of the favorite sectors in terms of response for respondents was health care.
00:13:45
Speaker
But I guess AI is also influencing that because it's helping to be more effective in terms of research.
00:13:50
Speaker
Yes, AI is a critically important area in health care.
00:13:56
Speaker
partly for the drug discovery process, but also there are many other use cases where people are creating startups that apply AI technologies to other areas within healthcare.
00:14:07
Speaker
So, for instance, automated note-taking systems that can enable doctors to spend less time writing up notes and more time seeing patients.
00:14:16
Speaker
And there are automated generative AI tools that can help with putting the documentation together for the clinical trials that you're running.
00:14:24
Speaker
And there are thousands of use cases of AI within the healthcare sector too.
00:14:28
Speaker
It's an important sector there as well.
00:14:29
Speaker
Sometimes we ask a one-off question and we ask them the respondents what they thought about our nine themes.
00:14:34
Speaker
Which were the themes out of the nine that sort of came to the top?
00:14:37
Speaker
Well, the one that really stood out was disruptive technology.
00:14:40
Speaker
Not surprisingly.
00:14:41
Speaker
Exactly.
00:14:41
Speaker
I mean, the survey is called Funding the Future Survey, so that's a natural focus for them.
00:14:46
Speaker
Other of our key themes which were regularly cited were the energy transition and digital finance.
00:14:54
Speaker
Every time we've had a discussion on AI on this podcast, I've always been surprised by the speed of development.
00:15:00
Speaker
Do you think that momentum is maintained or does it start to plateau?
00:15:05
Speaker
I mean, there is clearly a lot of hype.
00:15:07
Speaker
That's unambiguous.
00:15:08
Speaker
But there's a lot of real meat there as well.
00:15:11
Speaker
And so I think even if the capabilities of AI improved no further today, even if the models that we have today, that turns out we can't build models better than that.
00:15:23
Speaker
I don't believe that's going to be the outcome, but let's, for the sake of argument, say that that were to happen.
00:15:28
Speaker
I believe there's still five to ten years of real productivity benefits that come as people work out how to really embed these new technologies into business practices and make the efficiencies that will result.
00:15:43
Speaker
So I'm very bullish on the impact of AI.
00:15:46
Speaker
I think the technology has accelerated much further ahead of the ability for people to use it.
00:15:52
Speaker
And do you think maybe to finish on, is this a ubiquitous benefit for society or some people are going to have an advantage that they really have for a while?
00:16:02
Speaker
So I think as we discussed in our earlier work about AI, the good, the bad and the ugly, there are definitely ugly sides to AI.
00:16:11
Speaker
And one of the ugliest is probably the distributional aspect.
00:16:14
Speaker
It would
00:16:15
Speaker
AI is leading to rapid change.
00:16:17
Speaker
It's impossible for the change to be beneficial to everybody.
00:16:20
Speaker
There will be some winners and there will be some losers.
00:16:22
Speaker
And the more rapid the change and the harder it then is for society to adjust to those, the more disruptive that could be.
00:16:30
Speaker
The U.S. is clearly the area that's dominating at the moment.
00:16:35
Speaker
They do seem to have a clear advantage there, not just in Europe.
Open-Source AI and Global Disparities
00:16:39
Speaker
having the tech but also when you look at the companies that are reporting that they're practically using it a lot of those are concentrated in the US as well so it's not just on the tech side it's also on the implementation side that they're doing well but there is a massive open source AI movement it's not just the models that are
00:17:03
Speaker
offered as APIs by the big tech companies, there are a lot of very powerful sort of open source AI models, not just from the US, some coming from say China as well.
00:17:14
Speaker
And so I think the technology is there for everybody to use.
00:17:19
Speaker
So I think it could be very beneficial.
00:17:21
Speaker
Brave new world.
00:17:22
Speaker
Indeed.
Conclusion and Upcoming Events
00:17:23
Speaker
Well, Mark, I'm sure we'll have you back on the podcast.
00:17:27
Speaker
I hope it's me interviewing you.
00:17:29
Speaker
Me too.
00:17:29
Speaker
I hope we're not both replaced by avatars.
00:17:31
Speaker
But for now, thank you very much.
00:17:32
Speaker
Thanks for having me back on.
00:17:38
Speaker
If you'd like to catch up with the latest views from the team here at Global Research, then check out our new 2025 HSBC House Views report.
00:17:46
Speaker
This multi-asset report outlines our thinking about global economics, currencies, fixed income, emerging markets, multi-asset strategy, equity strategy, global trade, sustainability, and commodities for the year ahead.
00:18:01
Speaker
And with the world bracing for the impact of proposed changes to U.S. policy, our chief global economist, Janet Henry, will be taking your questions at our next live insights.
00:18:11
Speaker
The event takes place on the 14th of January at 9.30 UK time.
00:18:15
Speaker
Anyone can join.
00:18:16
Speaker
You don't have to be an HSBC client.
00:18:19
Speaker
Full details on how to sign up are on our LinkedIn page.
00:18:22
Speaker
Just search hashtag HSBC research or email us at askresearch at hsbc.com.
00:18:33
Speaker
So that's all from us today.
00:18:34
Speaker
Don't forget to follow the macro brief wherever you get your podcasts.
00:18:38
Speaker
Thanks for listening and we'll be back next week.
00:19:01
Speaker
Thank you for joining us at HSBC Global Viewpoint.
00:19:05
Speaker
We hope you enjoyed the discussion.
00:19:07
Speaker
Make sure you're subscribed to stay up to date with new episodes.