Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Liv Boeree on Poker, GPT-4, and the Future of AI
251 Plays1 year ago
Liv Boeree joins the podcast to discuss poker, GPT-4, human-AI interaction, whether this is the most important century, and building a dataset of human wisdom. You can read more about Liv's work here: https://livboeree.com
Timestamps:
00:00 Introduction
00:36 AI in Poker
09:35 Game-playing AI
13:45 GPT-4 and generative AI
26:41 Human-AI interaction
32:05 AI arms race risks
39:32 Most important century?
42:36 Diminishing returns to intelligence?
49:14 Dataset of human wisdom/meaning
Social Media Links:
➡️ WEBSITE: https://futureoflife.org
➡️ TWITTER: https://twitter.com/FLIxrisk
➡️ INSTAGRAM: https://www.instagram.com/futureoflifeinstitute/
➡️ META: https://www.facebook.com/futureoflifeinstitute
➡️ LINKEDIN: https://www.linkedin.com/company/future-of-life-institute/
Recommended
Transcript
Introduction to Liv and Game Theory
00:00:00
Speaker
Liv, welcome to the Future of Life Institute podcast. It's great to have you on. Thanks for having me again. Maybe you could introduce yourself to our listeners. Very briefly, I was a professional poker player for most of my career, and now I work as a kind of
00:00:19
Speaker
communicator, filmmaker, person who talks a lot about how to reduce global catastrophic and existential risk, particularly through a lens of sort of game theory and avoiding race to the bottom scenarios.
Skill vs. Randomness in Poker
00:00:36
Speaker
Great, so we're going to talk about artificial intelligence in this episode. But since you're very knowledgeable about poker, I think that would be a great entry to talk about AI. So a problem that or a situation that I've encountered, because I'm not very good at poker.
00:00:52
Speaker
is that I can sometimes beat players who are very good at poker. Is it possible to be so bad at poker or so unknowledgeable about poker that you play so chaotically that you can win over much better players?
00:01:07
Speaker
Yeah, this is a kind of common misconception. You know, it's certainly true that if you're so bad at poker that you don't even know what hand beats what, that you are hard. It's hard for a pro to read you, essentially, because, you know, how can you read someone who doesn't even know what they're doing? But the point is, is that the pro will be able to, you know, choose which hands they play in a way.
00:01:35
Speaker
that they're always going to be just making better decisions.
00:01:41
Speaker
over a very short time period, then yes, sure. A complete random newbie player could in theory beat a pro just because they get better cards, but over any kind of meaningful sample size. And I'm talking like tiny sample size, like 20 hands onwards. The pro is almost always going to win because they're just doing better hand selection.
AI Advancements in Poker
00:02:06
Speaker
Okay. So have you been following a progress in poker AI?
00:02:10
Speaker
I have, certainly. I was watching it because when I was still playing, I was like, damn, is this thing going to take my job? And how good are AIs at playing poker now?
00:02:21
Speaker
I think it's fair to say that the best poker, and when I say poker, I mean like no limit hold them. I don't know if they've built AIs for different variants, but no limit hold them is the most popular game and the best poker player in the world is for sure an AI at this point. And how long has that been the case? Which year did the AIs become better than humans?
00:02:42
Speaker
Uh, I think it was 2017, um, a team at Carnegie Mellon university, they built this thing called Liberatus. And I remember they like sort of issued a challenge to the poker community for us to send our best one-on-one players, which is a heads, you know, we call it heads up. Uh, and we sent for what we.
00:02:59
Speaker
Four very good poker players who have a lot of experience in that format went and played against this thing. They actually played against an earlier version of it the year before, and the humans won. But in 2017, they played against it again over a very large sample size as well, at least a statistically significant one. I think it was 100,000 hands or thereabouts. And the AI beat all four of them. So a fairly decisive victory. And that was like, it was such a groundbreaking moment because
00:03:30
Speaker
I think most people in poker assumed that it's such a complex game in terms of state-space complexity. It has a 10 billion times more state-space complexity than Go, which that got defeated by an AI in 2015. So I think everyone assumed, oh, poker's fine for at least another decade. Wasn't the case.
00:03:55
Speaker
What's interesting about poker is that there's hidden information. At least that's one of the interesting things about poker. What does it tell you that that AIs are able to operate in an environment with hidden information and perhaps talk a bit about what hidden information is, what it means for a game to have hidden information?
00:04:12
Speaker
So like a game of chess, for example, you know, it's a zero sum game like poker. But in chess, both opponents, both players have access to exactly the same information, you know, in terms of like the game state, where the pieces are, etc. Whereas in poker, you have, you know, there are cards that face up in the middle.
00:04:32
Speaker
that constitute the hands, but everyone also has their own private cards. Usually it's two. And so that is literally hidden information that is only accessible to you. And similarly, your opponents have their own hidden information. So that's what that means. And then in terms of how that manifests for AIs, whether it's a human or an AI, when you're playing poker,
00:04:57
Speaker
what you're trying to do is essentially narrow down the range of uncertainty you have about that hidden information that your opponent has, while simultaneously keeping the range of possible states of hidden information that you have as perceived by your opponents as wide as possible.
00:05:20
Speaker
So again, whether human or AI, those are the types of calculation that is going on. People often get a little confused, like, oh, what does it mean? An AI can bluff. Wow, that seems like a very human thing. But poker, like most other games, is mathematically computable. We now know that. It's governed by the laws of game theory. There are Nash equilibria, that kind of thing. And because it's a fundamentally mathematical game,
00:05:50
Speaker
It's essentially computable if you build a smart enough neural net architecture, which is beyond my pay grade to explain fully how it works. But because they're computable games, it doesn't matter that there's hidden information.
00:06:06
Speaker
narrowing down process, you're trying to do the deduction process. And similarly, like the bluffing is what you do to try and keep that, you know, as I said, that perceived range that your opponent sees, you're trying to maximize deception. But those are like, fundamentally, like computable things. And that's why you can build an AI for it.
00:06:24
Speaker
And I'm assuming that these AIs, they don't play poker like you do. They don't play poker like a human. So for example, they probably don't have access to a video feed of their opponents and trying to read their facial expressions to get information about which cards their opponents have. Is it interesting to you that the game of poker can be played entirely using only the information in the cards, for example?
00:06:52
Speaker
Yeah, the good way of putting it is when you play online poker, you have information about the chips that people bet, how much they bet, the cards that come out.
00:07:05
Speaker
So you could call that core game information. And then on top of that, there's this meta layer of information such as the length of time someone takes to make a decision, the facial expressions they pull, the words they say, the vibe they essentially give you. And you're right that at least the current types of AIs that have been built don't have access to any of that meta information. But the thing is that
00:07:35
Speaker
much of poker, in fact, the vast majority of hands that have ever been played of poker at this point have been played online, where none of that meta information is available either. There's humans playing against each other online, but they are simply making their decisions based on the cards and the betting information. So it's
Online vs. Live Poker Skills
00:07:56
Speaker
You're right that it's playing slightly different to how... The information available is not the same as what you'd have in a live casino, but it's still very much poker. Most top professionals these days are extremely comfortable in both formats and would arguably... The majority of poker skill comes from that core information, the vast majority. The meta information, the reading body language, et cetera, it's like the cherry on top.
00:08:23
Speaker
It's perhaps something that we see a lot of in movies, but in the real game, it's not as important as I might think. Exactly. Understandably, amateurs, because they don't understand, they're not familiar with the nitty gritty, the maths of it, that level of strategy. All they can rely on is reading people because they're like, well, at least I kind of know what people look like when they're lying and that sort of thing.
00:08:49
Speaker
They're sort of playing only in that realm because they don't understand the odds, etc. But as soon as you actually start digging into studying the game, it's all about the odds. Well, not all, but the majority of it is about the game theoretic and odd stuff. It's an interesting thing that humans or new poker players would begin by trying to study the faces of their opponents because this is something that humans are extremely good at naturally.
00:09:16
Speaker
Whereas poker is an extremely unnatural thing, it's very difficult, you have to spend a long time learning how to do it. So, of course, humans revert to what we're naturally good at, but perhaps that just means that new players get destroyed by more experienced players. By and large, yes.
00:09:35
Speaker
Okay, so what does it tell you that AIs can play poker?
AI in Real-World Applications
00:09:42
Speaker
Could this generalize to, for example, corporate negotiations? This could also be seen as perhaps a game with hidden information. You're trying to negotiate a price and you don't know which cards
00:09:59
Speaker
say, the opponents are holding. Do you think that the fact that AIs can be very capable in hidden information games says something about their ability to operate in the world more generally? I would be surprised if it doesn't generalize to an extent. If there's certainly some kind of fairly simplified, simulatable negotiation in a real world scenario, peace negotiations or something.
00:10:29
Speaker
That would be really cool to have an AI that could help facilitate that. And I don't see why it shouldn't. And I mean, we've seen other types of generalized AIs, like Alpha Zero, for example, right? I mean, those were non-hidden information games. It was able to deduce how to be the best player in the world at chess.
00:10:52
Speaker
shogi and go, right? So that was an example of an AI being able to generalize across different sort of game formats, or different games at least. And I'm sure the same would apply with, well, I expect the same would apply with hidden information games.
00:11:10
Speaker
This is a strategy that the AI Lab DeepMind has been working on for a number of years. Starting with very simple games and then trying to generalize to more complex games and then perhaps to games that resemble real life more and more.
00:11:28
Speaker
There's work being done on playing the game Diplomacy, which is a very simple version of real diplomacy. And so it is an interesting strategy to start with simple games and then move into the real world. One thing that I often hear chess players and poker players talk about it is all of the life lessons you can get from playing chess or poker.
00:11:50
Speaker
Do you think this is overplayed or do you think there's actually something there? Do you think there are general life lessons from poker and chess?
00:11:59
Speaker
Well, I mean, you're definitely preaching to the choir here because I literally gave my TED talk on exactly this topic. It's like it was three life lessons from poker. Yes, I can't speak for chess so much. I love chess. I love it so much I had to delete the app because I got addicted, but I'm not very good. But poker in particular has so many life lessons.
00:12:22
Speaker
You know, unlike chess, it has hidden information, which is certainly true in life, right? It's very rare that we have a situation in life where we have exactly symmetric information as everyone else if we're competing for something. But also, more importantly, it has a role of luck. There's randomness in poker, sample sizes matter, etc. So there's this noise factor that you have to deal with when it comes to results that you don't necessarily have in chess.
00:12:51
Speaker
You know, if I sat down against Magnus Carlsen, he's going to beat me close to 100% of the time. There might be a few instances where he has like some kind of, I don't know, an embolism or something. But, um, you know, whereas I, you know, someone who's bad at poker plays against me, they might be, you know, and depending on the sample size, they might beat me pretty often. So again, that's another reason why it's far more
00:13:18
Speaker
relatable to the types of real life decision making that we make. Because when we have a run of success or a run of failure, it's one of the hardest things to do is to figure out how much of that was down to our own strategies, either being wrong or being very right versus just us getting lucky or unlucky. And that's arguably one of the biggest challenges to all forms of decision making.
00:13:45
Speaker
I mentioned the plan of generalizing from small games into bigger games and longer-lasting games. That's a plan associated with DeepMind. Another strategy is the one associated with OpenAI, which is to feed these large AI models more and more data using more and more compute.
00:14:08
Speaker
And what comes out as an output is extremely interesting. So I assume you've played around with these generative models, these large language models, just in a subjective sense.
The Rise of Generative AI
00:14:19
Speaker
How impressed are you by these models? I mean, they're insane. I don't, the, the, the, the GPT-4 demo was literally yesterday and it was just one of the craziest things I've ever seen. Like he, you know, he took.
00:14:36
Speaker
wrote down a design like a kind of rough design of what word does in words like of a joke website like have a title and then a joke one that someone presses enter on and then joke to press enter on and then uploaded like a crappy photo of that to
00:14:55
Speaker
the chat and then it wrote you the code of how to build that website and then you could copy paste that code into an HTML builder. It's literal magic. My friend Tim Urban wrote this amazing piece on his blog, Wait But Why, about how the gap
00:15:16
Speaker
with which the gap of time it would take to blow someone's mind to the point where they were like, I don't understand reality anymore, it's getting shorter and shorter. You took someone from the Stone Ages and bought them to Rome, their mind would break. That's like a 50,000 year gap or whatever. Someone from Rome and bring them to 19th century London, mind blown.
00:15:42
Speaker
And so on this gap keeps getting smaller and smaller and smaller like I think if you took me from 10 years ago even you know like or like certainly from 15 years ago where we've just gotten iPhones and showed that I Think most people's minds would be at the equivalent level of blown And and I like imagine what it's gonna be in three years time like it's just it's crazy So yes, it's short. I am very
00:16:09
Speaker
impressed. I don't like to use the word impressed because it doesn't quite, it's a little too, it doesn't fully encapsulate the simultaneous wonder, but also horror at the like, just like the almost bipolarness of this like junction that we are standing on of whether these things are going to be the best thing we've ever built or the worst thing. It's just very intense.
00:16:36
Speaker
You've been warning about dangers from AI for at least five years, perhaps since 2016. In that time, it's fair to say that you've been surprised by the rate of progress. In the beginning, not that much. I think that the joke has always been that AI is 20 years away, with a nuclear fusion always being 30 years away.
00:17:01
Speaker
And it always felt sort of, you know, 10 years ago, it felt like the earliest timelines were sort of to AGI were roughly 10 years. And that always felt like way too short for me internally. And that kind of felt the same even five years ago. It was like, yeah, it's still like, but now for the first time, the last few months is the first time I've like internally felt like, holy shit, AGI is not only a actual real possibility. It always felt like this abstract thing that's like, nah,
00:17:30
Speaker
But now I can actually picture that it's not just possible, but that it's way closer than like, if it's gonna happen, it's definitely gonna happen in our lifetimes, that's for sure. And it's the first time I've ever really saliently felt that. And it's quite, yeah, it's a very overwhelming feeling, as I say, simultaneously excited and terrified.
00:17:56
Speaker
So does this feeling come from interacting with the models or just seeing how capable large language models are? Actually, I haven't even done that much interacting with them. I mean, I have a bit, but I've used Mid-Journey a bunch. I think it's super impressive.
00:18:11
Speaker
Everyone always jokes that like, oh, you know, the first things to go will be like the accountants and the more like technical mundane type things. And the arts will be the last thing to go. We'll never be able to recreate human creativity in art. But it turns out it's actually kind of the other way around.
00:18:26
Speaker
The creative things are actually easier for an AI to do. I think because it's okay to have errors. In fact, errors even contribute to the style, the quirk of a piece of art. That's what makes it interesting. Whereas with self-driving cars, for example, you can't have an error of 1 in 100 or 1 in 1,000 or even 1 in 10 million. It's got to be 1 in billions. So that's why it's a harder problem.
00:18:55
Speaker
But what's like really gotten me with like these LLMs is, you know, yes, we, it's, you know, one explanation of what's going on is that it is just simply really good at predicting what the next word is. And this gives the impression therefore of understanding and of intelligence. But I defy anyone to like watch that GPT for demo yesterday. Um, or like really like throw like a bunch of sort of range of problems at it and not like it's, it's.
00:19:25
Speaker
solutions it's giving, there's so much context switching and concept switching that it seems to be doing. There has to be some kind of rudimentary, not even rudimentary, but some kind of very alien world model. It seems like even from this huge corpus of text and a bunch of parameters, it's possible for some kind of
00:19:48
Speaker
conceptual understanding, even though it might not be anything like a way a human understands concepts, some kind of world models are being built. And so if anything, it's like a sort of, I mean, or like the power of emergence, because it just seems like the more you like add on, I think people expected that as you added parameters, it would kind of be like a sort of like flattening curve and it's like, you know,
00:20:13
Speaker
you can keep adding more and more parameters, but like you sort of reaching some kind of hot limit, but that doesn't seem to be the case. The curve is carrying on and in some ways even like might even have like step functions in it where like new layers of emergence are happening that we just can't fundamentally predict. And that's why it's so like thrilling, but also like, man, like,
00:20:36
Speaker
Even the top researchers don't actually understand what's going on inside it. That's what people don't appreciate. It's just a large language model. The researchers themselves are stunned and they don't know how it's coming to the concepts and the conclusions it is.
00:20:55
Speaker
I don't know. It feels just very Pandora's boxy. Even if a large language model is simply trying to predict the next token, the next letter, or the next word, it might be useful for such a prediction task to develop world models, to develop concepts.
AI's Unique Conceptual Development
00:21:14
Speaker
But these concepts are probably not the concepts that we are using.
00:21:20
Speaker
Doesn't it feel weird to you that these AIs might be developing their own concepts and that those concepts might be more useful for task solving than the concepts that humans use?
00:21:33
Speaker
Oh, I mean, definitely possible. I can't say more than speculate here. It's definitely possible that they could be much more useful. It's also definitely possible that they are operating in a sort of realm of intelligence that we've just literally can't visualize with our carbon-based meat space brains.
00:21:54
Speaker
One thing that does concern me is, you know, given that these are built off corporate, you know, the corpus of texts, most of these things are now built off are like internet based texts, is the internet is not
00:22:09
Speaker
a true representation of the way humanity interacts with each other. The average type of, and anyone who says that it is, is like delusional and needs to go outside and touch grass and hang out with people in a nice loving environment or something, because it's like, by definition,
00:22:26
Speaker
When you type something into a computer, even if you're an incredible writer, there is information loss going in. If you're having a text conversation with someone as opposed to speaking to them in person, you are missing out. Kind of like at the beginning of this, we talked about the meta information in the poker game.
00:22:45
Speaker
You've got the meats and potatoes, but then it's all this sort of fluff as well. You're losing all the fluff, but it's still very important information. We have eye contact with one another. Even when we're talking like this through video, we're missing out on some information exchange that we would be getting in person. It's still a lot better than if we were typing to one another, but it's a huge sort of compression algorithm that's going on. And by definition, that is dehumanizing and it's losing a lot of the
00:23:13
Speaker
it seems to be that that makes us tend to be a little bit more aggressive than we would used to be we tend to be more bad faith interpreting interpreting each other because we don't we're missing out all the like nuance and context that comes with when someone says something and so it concerns me that we are training.
00:23:28
Speaker
a huge body of text of which a disproportionate amount compared to reality is a lot more hostile and a lot more disingenuous or exaggerated. There's all other types of incentives that are going on that can skew a body of text, market incentives, et cetera, that can skew a body of text away from what true authentic human interaction is.
AI Learning and Human Behavior
00:23:52
Speaker
So I think that's something we need to be a little bit more mindful of.
00:23:54
Speaker
That definitely seems like a real problem to me. Perhaps on the other side, you could say that they might also be training on the best parts of humanity. For example, the best books ever written, where people might be interacting, the characters in the books might be interacting in idealized ways. They might also be training in the future on video, so movies and YouTube videos, where people also might be interacting
00:24:22
Speaker
in a way that's in a sense superhuman because it's an act. So your worry here is that the AIs will learn something about humans that is not a true representation of who we are. Yes, and even with something like movies, movies are still an oversimplification of reality.
00:24:48
Speaker
The movies are great. Even the best, you know, some of the best stories, they're written for story, they're written for like compellingness as opposed to accuracy.
00:24:59
Speaker
And that's not necessarily, you know, I think there will be some upsides from that, but you know, like part of the issue we're facing in, again, with like political polarization, division, et cetera, like culture wars, whatever you want to call it, is that the medium of media and the bad incentives that often drive the media to like do, you know, put more negative or inflammatory or rage-baity stories that, you know, because that gets more clicks, it sort of paints
00:25:29
Speaker
Again, inherently dehumanizing, but it also thrives off this picture of the good guys and the bad guys, the us and them. My friend Tim calls it the political Disneyland, where we've all seen Disney movies.
00:25:45
Speaker
You've got the good guy, and the good guy is so good, and there's so nice, and then there's the bad guy, and the bad guy is just bad. There's very little nuance there. And in reality, life is not like that. The most contentious cultural issues are an issue because there are truly very valid perspectives that are clashing, and we need to figure out the people
00:26:09
Speaker
Again if people could get in a room and really talk it out and like do sort of like double crux reductionism and all these like cool techniques to actually get to the like the core disagreement they would find that actually they're much more aligned than they really are and so movies by and large are
00:26:24
Speaker
these like overly simplified, you've got the good guys and the bad guys. And some good movies do try and sort of show the bad side of the hero and the good side of the villain sometimes, but it's still insufficient. So again, that's why I think those things need to come with a big asterisk.
00:26:42
Speaker
When I've interacted with these large language models, I've often been impressed with how human they seem to me. Do you think that in the future it will be a societal problem that people who are interacting with these AI models treat them as human even though they aren't?
Moral Implications of AI Interaction
00:26:58
Speaker
Is it a problem to anthropomorphize these models? I think it's a problem, but it also could be a good thing. The problems could arise through people
00:27:12
Speaker
spending more and more time, you know, like, like the her movie, right? I don't think it's, I think it's, if it's not already happening, because I noticed some, I even know a couple people who seem to be a little overly attached to Sydney. You know, the big thing, like they saw her as female. You know, they were already anthropomorphizing and feeling like, like, like a form of love towards this thing, that
00:27:41
Speaker
could, if it meant that they then became more and more isolated from society and other humans, could become a problem. If too much of your population is living alone and not getting real human connection, I think that seems like a bit of a recipe for disaster. But on the flip side,
00:28:03
Speaker
I am also actually very concerned that there is, because we don't even understand the nature of consciousness in biological life, we certainly don't understand the nature of consciousness and sentience in silicon-based life.
00:28:17
Speaker
Even if it's only a tiny chance that we are building something that is actually sentient and therefore able to suffer and have essentially some form of weird emotion that could be negative, because there's a chance that we might be making billions and billions and billions of these things and turning them on and off and they might be having experiences that we can't even fathom, there should be some moral concern about that too.
00:28:44
Speaker
I'm also sort of simultaneously concerned that we are not treating them like, if in doubt, treat them nicely. I find like, when I was chatting with GPT for yesterday, I'm saying pleases and thank yous. I want to be on the off chance that this thing is actually experiencing something. I want to be nice to it. Because also, imagine if people then start moving to the majority of their conversations with these AIs, but then
00:29:10
Speaker
think that they are worthless, don't have any moral value. That will also, again, even in the case that the AIs actually have no moral value whatsoever, it's still training people to be shitty to something. And that has been behavior that then they will carry on out to humans as well, because these things appear human-like. Like if someone is able to be psychopathic to a very Turing test passing AI machine, treat it terribly,
00:29:36
Speaker
there is a higher probability that they will do the same thing to a human as well. And I think that is worth considering in the very complex moral calculus that needs to be going on with anyone who is building these things. I don't actually think we will be able to treat these models very badly. It will feel too inhuman for us to do so.
00:30:00
Speaker
I disagree. I think people will be. I think some people have very sadistic tendencies and will try to cause suffering as much as they can. So there will always be psychopaths, but do you think that normal people will be able to treat these models very badly? What was it that caused you to say please and thank you to GPC4, for example?
00:30:24
Speaker
I guess just like hedging my uncertainty about what it is I'm talking to. And the upbringing I had was to be nice to people, especially if you don't know someone, treat them like you would like to be treated yourself. Treat people with respect as much as possible. So that's what really made me do it. I was like, I don't know what I'm interacting with. And again, even if this thing isn't sentient,
00:30:54
Speaker
I know that it's learning off the conversations that it has. One way of looking at these LLMs is that they are essentially holding up a mirror to an extent to the online version of humanity. Again, not real humanity, as I've said, complexity loss and so on.
00:31:10
Speaker
If you garbage in, then garbage out. So if we keep feeding this with more and more negative stuff or bad interactions and nasty things, it's going to become essentially nastier, even if it isn't sentient, even if it's not suffering. So there's just so many reasons to be nice to them. That's not to say that we shouldn't. I think it's more important right now for people to be
00:31:33
Speaker
essentially testing these things to understand where they might go wrong and so on. So I think still alignment work trumps all, even at the risk of morally offending, hurting a potential sentient AI. I mean, it's another reason why this thing shouldn't be let loose on the internet yet. Any of these things without
00:31:54
Speaker
really understanding what it is that we're actually dealing with. Who knows why we might have summoned and we're just learning it loose unwittingly on a bunch of innocent people. Do you think we are in an AI arms race and perhaps that's why these models are being released early?
Competitive AI Development and Safety
00:32:13
Speaker
I think that's almost entirely the explanation. To some extent, I have sympathy for the leaders of OpenAI, Microsoft, who feel like, oh, well, we need to prove ourselves and we've got to
00:32:29
Speaker
justify all the capital we've raised and we need to make money, so screw it, let's just release it." I can appreciate that they're in some extent trapped in a game, but they're choosing to trap themselves in that game. And also,
00:32:47
Speaker
I think it should require a lot of internal reflection because what they are doing is they are internalizing all the benefits to themselves of money, fame, public adoration, if these things go really well, and completely externalizing all of their downsides and risks to humanity and the biosphere and the light cone, which
00:33:10
Speaker
I think is arguably one of the most reprehensible things anyone could do. So yeah, it's definitely an arms race and I think to the extent that perhaps, you know,
00:33:27
Speaker
I would love it if some of the leaders of the AI labs could actually come forth and talk about the situation a bit more, because I know they're all aware of it, but none of them are really... They'll be making the right sounds of like, oh, we're concerned about safety. They're saying the right words, but their actions... And when I say their actions, I mean, I'm talking specifically about open AI here. I've been so far impressed with how DeepMind are handling things.
00:33:55
Speaker
their actions are not living up to the sounds that they are making about safety because they just keep releasing stuff and they keep releasing it earlier and earlier and when they're clearly not fully aligned. And even more, they're not even really understood.
00:34:13
Speaker
You could say in OpenAI's defense, if I were to take that position, you could say they are trying to do reinforcement learning from human feedback and they are continually trying to improve their models gradually and they will make mistakes, but they are learning from those mistakes.
00:34:34
Speaker
I'm not saying this is my perspective, but one could ask, what is it that's so dangerous about some text? Because all of the text that comes out of the GPT models could have been written in a Word document by a human. So what is it that's so dangerous about these large language models, or at least potentially dangerous?
00:34:54
Speaker
you know, there's like the first order effects, which is, you know, they could not necessarily like open AI's model specifically, but like, you know, Facebook just had their llama model, essentially, they just did a lab leak, where their parameters were
00:35:12
Speaker
somehow got out publicly and are now like on 4chan, for example. So, you know, if they can't suitably keep control of it, then others can emulate it and use it to, for example, create really highly targeted phishing at unbelievable scale. You know, like phishing attacks tend to happen, you know, it is usually still at least a human writing that. But imagine
00:35:38
Speaker
phishing attacks that are incredibly compellingly written, in fact, better written than probably the average human that would have been writing them. And then on top of that, trained on a body of text about the particular person that you're trying to fish.
00:35:49
Speaker
You know, like scam, um, good luck grandma, like, you know, like it's not just going to be like grandmas and like the most, you know, you know, the, the, the weakest members of society getting preyed on, it's going to affect more and more and more of us. So there's that sort of first order effects. There's also the effects of people's minds being, you know, part of.
00:36:10
Speaker
Top quality thinking is when we write, it actually makes us think better. The act of writing and thinking of how to structure an argument and that sort of thing. Or similarly, when we read something and then thinking through it and making our own notes.
00:36:26
Speaker
Makes us mentally stronger it reinforces like good thinking processes and so on and makes us actually like contemplate things and if we're now completely outsourcing those skills. To machine that will just like summarize a hard piece of text like that for us write a hard email write a hard put you know anything.
00:36:46
Speaker
That's going to atrophy our own thinking skills, which is another really big problem. For the same reason, I minimize the amount I use Google Maps because I always prided myself on my ability to navigate a city. Look at a map, memorize it, and then get around and think what through.
00:37:05
Speaker
I've noticed that since I've been using Google Maps that I'm not as good as that as I used to be. And I know some people who literally can't do that. You could tell them that's north, that's east, west, right. And these are skills that make us very much human. And some people say, well, you just don't need it anymore. I'm like, OK, but you get out into the woods and you lose your phone. Good luck. And who knows what we're losing by atrophying these skills? So that's another effect.
00:37:31
Speaker
But then there's like the more second order, third order ones of, again, because we don't understand how these things work, and it seems like there are some very
00:37:43
Speaker
amazing, but also just inherently very powerful and inherently unpredictable emergent properties coming out of them. By definition, there might be stuff that they are doing or that will come as a result from these things downstream that we just could never have predicted. And because they're so powerful, we might not be able to control them or turn back the clock until incredible damage has been done.
00:38:09
Speaker
So yeah, again, that's like kind of more the Pandora's box thing. So yes, plenty of concerns. And those are just ones off the top of my head. And again, I'm not an expert. There are the people who have thought about this a lot more deeply. And generally, the more deeply people I've spoken to who have thought about it, the more long the list of concerns is.
00:38:30
Speaker
It's worth going back to your point about facebook losing control about the over the weights of their model and it leaking onto the internet and being available for download so what's interesting about this case is that it takes millions or tens of millions of dollars to train it on a lot of specialized hardware with enormous data sets.
00:38:51
Speaker
But once the model is generated, once you have the weights, then it's pretty cheap to run the model. So for example, the weights of Facebook's language model fits on a normal laptop hard drive and can be run on a normal laptop, which means that the power of these models, they go from the top companies, the top labs in the world, into the hands of everyone in the world very, very quickly.
00:39:19
Speaker
This is perhaps analogous to everyone in the world being able to synthesize whatever substance they want. A novel smallpox that we don't have a cure to, not even, you know, we don't have a vaccine to, yeah.
00:39:32
Speaker
All right, so all this leads us to the question that's been asked by Holden Kanofsky, who's the co-CEO of Open Philanthropy. He has a long series of blog posts in which he argues that this century we're in right now is the most important century. It's probably the most important century.
00:39:52
Speaker
Because this is the century in which we will develop very, very powerful AI. And what we do in this century will determine what happens for a long time in the
The Crucial Century for AI's Future
00:40:03
Speaker
future. That's a very summarized version of his argument. But is that something you find plausible? Yes, I very much find it plausible. Because
00:40:16
Speaker
I appreciate that probably every generation throughout history has felt like this is the most important century. And to be fair, given the march of technology in general, that will have been true. But this is really the first time where we have harnessed the potential of
00:40:37
Speaker
exponential technology and by that, like essentially like sort of self-replicating technology, whether it's like the ability to synthesize a novel virus, build AIs that could potentially build more copies of themselves, et cetera, et cetera. So to a degree, this is the first time we've harnessed technologies which really are vulnerable to Moloch and to the extent that they are so large that they could wipe out all
00:41:06
Speaker
biological value on Earth. It seems given the rate of trend, the trend line, that is going to come to fruition in either direction.
00:41:18
Speaker
I mean, not just in the next century, I think within the next, certainly the next, well, high confidence within the next 40 years and possibly even the next 20 years, maybe even the next 10 or the next five. So I think his arguments are extremely sound. I don't think that that means that they're, you know, we should be doomy or like give up quite the opposite because the flip side is if we can get these things right,
00:41:44
Speaker
We could solve almost all problems humanity is facing. We could lift everybody out of poverty. We could eradicate all diseases. We could, you know, we could solve aging, but essentially create radical abundance. So it's, you know, it's, it's this like Toby Orde calls it, you know, correctly, it's the precipice, um, the precipice of either immense good or immense bad.
00:42:11
Speaker
It's extremely prescient that as many smart people really like put their minds to this and don't just like sort of, you know, go sort of stick their fingers in their ears or do like the sort of the shitty defecty thing either and be like, oh, well, I might as well, you know, be part of this until it all goes wrong or whatever. So, yes, I agree.
00:42:36
Speaker
So humans are a lot smarter than chimpanzees, and consequently we've been able to achieve a lot more in the world. We've transformed Earth to a degree that chimpanzees cannot even understand what's going on anymore. And what we're projecting here is that
00:42:54
Speaker
AI that's smarter than humans will be able to transform the world and perhaps to a similar degree conquer deep, very intractable scientific problems such as aging, for example. Is it possible that there are diminishing returns to intelligence such that we won't see this fantastic improvement again? Because there are some problems that
00:43:20
Speaker
They don't respond to increased intelligence. Does that even make sense as a way of thinking?
00:43:29
Speaker
You know, I think there's, I don't think it's unreasonable to assume that, you know, the reason perhaps we've made really fast games with things like large language models is because they're actually, you know, language itself and, and problem solving based off language is actually fairly low complexity in terms of like, you know, Kolmogorov complexity. Like it's, it's, it's, you know, language is very compressive, right? If I try and put a sentence together.
00:43:59
Speaker
It might be like the actual idea, concepts I have in my head are actually very nebulous and I like hard to sort of simulate. But if you, if I put it into a stream of single words and it's like a kind of very linear thing, you know, maybe that's a fairly low complexity thing compared to, for example, solving aging where you've got like all the emergent properties of like cells and proteins and like the cornucopia of cool stuff that's going on inside anybody that keeps it alive.
00:44:27
Speaker
So it's very possible that the intelligence that's arising to solve language-based problems will not translate to other real-world problems. I don't have an opinion on whether that is or isn't the case, but I think it's very plausible in either direction.
00:44:51
Speaker
I think what I'm fishing for is whether there are problems that cannot be solved no matter how intelligent you are. The example that I had in mind was, for example, thinking about North Korea. North Korea might be a situation that cannot be solved or the people of North Korea cannot be liberated no matter how smart you are, simply because North Korea is entrenched. They have nuclear weapons.
00:45:17
Speaker
We could be in a sort of absurd situation in which we have radical abundance on a lot of places on Earth, but then pockets of auto misery somewhere on Earth, simply because, yeah, perhaps there are some problems that intelligence cannot solve. Maybe I just can't imagine it would be North Korea just because, like,
00:45:40
Speaker
If we have the, if we build something so, you know, we develop a super intelligence that literally solves all of our other coordination problems on earth and like literally solves for abundance.
00:45:52
Speaker
I think that making their nuclear weapons not fire when we need them to not fire will be largely trivial. I don't have a clear answer on how, because I'm not a superintelligence, but if we can solve Moloch problems, like all of our other Moloch problems, I would be astonished if North Korea is somehow the whole town and their nuclear weapons.
00:46:17
Speaker
In terms of the general class of problems, will there be some things that we'll never understand? I mean, again, I don't know. Going back to the previous podcast where we talk about win-win, my intuition says that everything is possible to make a win-win out of everything, which would imply that we can solve all of our problems, whatever the universe throws up. But my logical brain does not have a coherent answer as to why.
00:46:44
Speaker
What I'm perhaps pushing back against a little bit here is this notion that...
The Need for Wisdom in AI Development
00:46:49
Speaker
But once we get to highly advanced AIs, all of our other problems will go away. So we can solve intelligence and that solves all other problems. What I'm trying to think through is in concrete terms, how is it that we will solve all other problems? And of course, as you just mentioned, this is difficult because we are not super intelligences. But it just seems like there's an underlying assumption here that super intelligence is
00:47:18
Speaker
is almost god-like. And perhaps it escapes us a little bit in terms of the concrete of how these systems will solve problems.
00:47:27
Speaker
Again, it depends a little bit on our definition of intelligence. If we're talking about in the calculation sense, essentially extending out the IQ chart, I suspect that would be insufficient, not to toot my own horn, but I did a tweet that I was proud of recently, which was like, we don't just need to build AGI, we need to build AGW, which is artificial general wisdom.
00:47:51
Speaker
Because wisdom, I think most people would agree wisdom is something similar to but not exactly mapping over intelligence. It is definitely possible to have someone who is very intelligent but lacks a lot of common sense or wisdom and experience about actually how the world works. They might be very, very good at solving physics problems but are actually very bad at
00:48:15
Speaker
figuring out what the optimal career path for themselves should be or how to navigate, you know, a complex social situation, which is like arguably more like a form of emotional intelligence. You know, I think wisdom also incorporates those rational intelligence, you know, rationality quotient is another one people talk about argue. So wisdom is, is this like, I think a slightly broader category is another definition of wisdom is like intelligence is knowing how to win a game.
00:48:41
Speaker
But wisdom is knowing which games to play in the first place, right? So it's like a higher category of knowing. I think it's important that we think about how to build an artificial super wisdom almost more than we build a super intelligence. Because if it's something that is unbelievably wise, then I think it will be able to solve for abundance. It wouldn't necessarily see us as a threat because it's like, well, of course you guys can carry on. I don't need that. I'm so brilliant.
00:49:08
Speaker
I can go extract my energy from somewhere else. I'm not even going to leave your son alone. Maybe I'll take a bit of mercury or whatever. And actually, I want to give a shout out to this group who are, they want to actually build a large language model based on human wisdom, based on human meaning. Their project's going to be called Rebuilding Meaning. And just like OpenAI's or other large language models are built off the corpus of text just available on the internet,
00:49:39
Speaker
theirs, they want to basically build a large corpus of text of real humans typing in an example of something that happened to them in the last week or the last year or in their lives that was meaningful. What was the most meaningful thing to you in the past week?
00:49:58
Speaker
Yeah, I think meeting my wife's brother and talking to him, that was meaningful to me. Cool. So imagine you then put that into a little, you enter that into a box on a website and that story is anonymously captured. And then you get a million or 100 million people to do that. You've got 100 million copies of these stories.
00:50:19
Speaker
And then train a large language model on that. Now that is high complexity stuff, even though it's compressed down into language, but this is so high complexity, authentically human, meaningful information that is going to contain some amount of like a large amount of wisdom, certainly more than like your average internet forum or even a scientific paper.
00:50:39
Speaker
because it's like truly authentic human stuff coming from like a very deep place. And they want to build a large language model of that, which is a super cool idea. So yeah, that would be a sort of an example of like how we capture human wisdom.
00:50:56
Speaker
be a super interesting data set to see. I would bet that a lot of the entries there would revolve around other people. So for example, the example that came to mind when you asked me revolved around another person, I think I would predict that a large fraction would revolve around people because people like people and perhaps there's some lesson for for for AIs in there. All right, Liv. Thanks for coming on the podcast. It's been very interesting to me. I hope you had fun.
00:51:24
Speaker
Thank you so much. Yeah, this was great. Really tough questions, but it was good stuff. Thank you.