Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#31: Jacob Browning: Unmasking the Fake Minds of Large Language Models
0 Plays2 seconds ago
Have you ever wondered if AI models actually understand the words they generate, or if they are just really good at faking it?
On this episode of The AITEC Podcast, Roberto García and Sam Bennett are joined by philosopher Jacob Browning (Baruch College, CUNY) to unpack his article, Intentionality All-Stars Redux: Do language models know what they are talking about?
Using a clever baseball diamond metaphor and drawing on the philosophy of Immanuel Kant, Jacob explains why Large Language Models lack the "intentionality" required for genuine comprehension. We cover:
- First Base (Formal Competence): Why LLMs struggle with basic logic and negation, revealing the absence of an underlying logical engine.
- Second Base (Rationality): Why true understanding requires purposive behavior, and how LLMs hilariously fail at "intuitive physics" (like trying to inflate a couch to get it onto a roof).
- Shortstop (Objectivity and World Models): Why genuine understanding requires grasping an objective, mind-independent world that determines whether sentences are true or false. This position explores how LLMs lack a coherent "world model," causing them to fail at tasks that require intuitive physics and planning for counterfactual situations (like predicting where a billiard ball will go or playing simple video games).
- Third Base (The Unified Self): Why making a claim requires a persistent self that takes responsibility for its beliefs—something a next-token predictor simply cannot do.
Whether you're exploring the intersection of AI, technology, and ethics, or just trying to figure out if your chatbot actually knows what it's saying, this conversation will give you the philosophical toolkit to see through the illusion.
Recommended
Transcript
Introduction to Guests and Episode
00:00:16
Speaker
All right. Hi, everyone. Welcome back to the ATEC podcast. I'm Sam Bennett. And today, along with ah Roberto Garcia, we are interviewing the philosopher Jacob Browning. Jacob is a professor at Brute College, Cooney in New York City.
00:00:28
Speaker
His work focuses on the history of philosophy, the philosophy of mind and the philosophy of technology. So, Jacob, thanks so much for coming on. Thank you guys for having me. It's a pleasure. Absolutely. Yeah. So basically we're going to be discussing themes from your recent article, Intentionality All Stars Redux.
00:00:46
Speaker
Do language models know what they are
Do Language Models Understand?
00:00:49
Speaker
talking about? So before we kind of dive into the article, could you just tell us a little bit about yourself, like you know where you grew up and maybe like how you first got into philosophy?
00:01:00
Speaker
Sure. Yeah. ah So I am originally from New Mexico and grew up there, then ah went to college in L.A. and then came to grad school in New York. And I've been here in New York ever since. um I got into philosophy.
Jacob Browning's Philosophical Journey
00:01:18
Speaker
i was a religion major back in college. And, ah you know, so I was kind of on the... the the you know edge their philosophy. But I was taking great books courses and I read ah Kant's ah Groundwork of the Metaphysics and then ah Nietzsche's ah Genealogy of Morals and Kierkegaard's Fear and Trembling. And I was like, you know, there's a discipline out here that I think I might actually ah be better fitted for. And so i kind of was near the end of my degree anyway, i finished that, but then i transitioned, read a ton of philosophy for a few years, and then came to the new school and was really infested in Kant and Hegel, which
00:02:07
Speaker
was kind of the new school thing. ah Lots of time there with the Bernsteins studying Kant and Hegel. So yeah, that was kind of what got me into this. and I was really ah just still a big fan of great books because I just love ah reading the history of these things.
00:02:23
Speaker
Awesome. Yeah. And we'll we'll definitely ah be talking a little bit of Kant in this conversation, I think. So that's cool.
Understanding Beyond Fluent Sentences
00:02:29
Speaker
um So your work you know is investigating language models, and you're kind of asking this question of whether they know what they are talking about.
00:02:40
Speaker
So maybe we can just kind of start with that question. um you know it seems like It seems like to know what you're talking about, it's more than just producing fluent sentences or saying things that sound right.
00:02:55
Speaker
it's not just like talking in a convincing way. um But yeah, I guess, i don't know, can you kind of like talk a little bit about what does it mean to know what you're talking about? And how did you get into this question?
Language Models and Intuitive Physics
00:03:09
Speaker
So, I mean, early on ah when I was back working in, ah I was with Jan LeCun in his lab and Jan has always been really super focused on ah kind of world model questions of like, how do you get the intelligence of, say, a cat was his thing. um And I spent a little time, up ah you know, in Boston and the question there was, how do you get the intelligence of a three-year-old?
00:03:34
Speaker
And these kinds of questions are like, embodied intelligence and intuitive physics, your ability to move about the world and predict changes and things like that. And so their questions were very much grounded how do we have an understanding of the physical world, the social world, and so on.
00:03:53
Speaker
um And so when language models came out, there was this really superficial level to them where they often said really compelling things, but when you probe them for their intuitive physics understanding, are they going to understand what happens, you know, when you ah kick a table? Are they going to understand that the things on the table will fall?
00:04:13
Speaker
They were really messy on that. They've remained really messy on these questions. And so you saw this kind of gap between the competence they display with language and the underlying comprehension about kind of the way the world works, the way things happen that you would expect to find in any of kind of ah kind of you know, mammal ah that's sufficiently sophisticated, the capacity to predict how ah objects are going to react and so on. And that's just been something that neural networks have struggled with. And i kind of wanted to bring out like, look, why is it that they're so competent and they do such a good job and they're able to pass these tests on the one hand, and they also say really stupid things, um kind of
00:05:01
Speaker
regularly how is it this kind of ah tension coming about and so i wanted to kind of answer that by saying look there's a deeper level um that we sometimes associate with understanding that goes beyond mere confidence with saying the right thing good you maybe can really quickly uh on that whole idea of like um intuitive physics so that's like the idea like um you know all the Obviously, not all of us have much of a grasp, let's say, on Newtonian mechanics and all that or whatever. i don't. But anyway, i do have the sense, though, that like objects, you know they're persisting over time. you know When I leave a room,
00:05:40
Speaker
the table continues, you know, um, solid objects are not like passing through each other. You know, I can't get this cup to go through this chair. Uh, things are like when they fall, they're generally falling downward. And, um, and then I guess it's like, is it with LL with large language models? Um, you know, you could probably ask it a lot of sophisticated questions about Newton and relativity theory and,
00:06:07
Speaker
ah you know, they would blow most people out of the water, but could you just maybe like expand on like, what are are the, some of the things that they, they sometimes like goof up when it comes to, yeah, I'll give my favorite example of this. Cause it was one from a paper back in 2022 that kind of floats around. And I think I discuss it in, uh, the intentionality all-stars piece, uh, involves a couch and they say, how do you get a couch to the roof?
00:06:36
Speaker
Uh, without using the stairs or a pulley. And it's actually kind of a hard problem. I don't know. I'm kind of like, well, what would you do for that? um But there's obvious things you can't do. And the language model often suggests things that are just straight out forbidden, like inflating the couch so it floats to the roof. And you go, that's not how...
00:06:58
Speaker
No, and the other one that i always love is lower from a truck bed. And what I like about lowering from a truck bed is there's like obvious sense, like if you can't raise it, you have to lower it, but there's no way that the whole problem of getting the truck to the roof is even more complicated. So you kind of this illusion that they're fully on board with the physics of the world and they could easily pass a physics test And then you kind of give them a bespoke problem that actually requires some thinking through. And they often do things that are real superficial.
00:07:34
Speaker
If I wanted to get something to float or to get something to be raised, I might inflate it. You go, okay, if I wanted to get something and I can't raise it, I might lower it. Okay. So there's this kind of intuitive logic to what they're doing at the level of language that doesn't work with the actual physics of the world.
00:07:54
Speaker
I guess I want to add and another layer of complexity here, though, because even though it does seem like the large language models, ah if you if you ask the right questions, you can kind of see their failures. But for the most part, we are where they're so seductive and they sort sort of lull us into thinking, you know, ah in regular usages, into thinking that they kind of know what they're talking about.
00:08:21
Speaker
ah Maybe you can tell us about this whole idea that we're sort of sitting ducks for, ah you know, falling for large language models.
Attributing Emotions to AI
00:08:32
Speaker
Yeah, so... um I forgot the name of the study, but way back ah in the 1940s, they did some very basic studies about, they just put little tiny graphics on a screen. One was a circle, one was a triangle, and they moved in kind of irregular ways that
00:08:53
Speaker
you know, where they bumped into each other and kind of went around each other and stuff, they didn't make like physics moves. It wasn't like straight lines. It was kind of more animated moves. And when children, four-year-olds looked at this, they ascribed intentional descriptions. They said, oh, it believes that it's blocking its path or it wants to get around it or the other one's being mean and it's blocking it. You know, they they immediately ascribe even to these geometric shapes,
00:09:23
Speaker
ah kind of the language we would expect of an intelligent agent. And so this has kind of come to be this assumption of the kind of baseline intentional stance that Dennett sometimes talk about, that just we encounter something that behaves in complicated ways. It's really very natural to reach for intentional ah descriptions and say it has beliefs, desires, emotions, you know, it's bull kind of doing things to block other people or assist other people and so on.
00:09:56
Speaker
That is really hard. It's, you know, ah sometimes, again, another type of intuitive. We sometimes call that intuitive psychology. It shows up very young, not quite in my six-month-old, but, you know, give her another year, you'll start to see them, ah you know, picking up on my kind of theory of mind. They'll ascribe attention intentions to me, they'll try and figure out what I'm trying to do, they'll start to predict ah my behavior in terms of what they think I'm trying to do and what I want.
00:10:30
Speaker
um so Very early developing, very natural to describe ascribe it to in ah non-human um and even non-biological objects. And so you get a machine that talks, boy, we are, it's absolutely overkill. Like we're already sitting ducks for this kind of behavior. And then it's, so these are so intelligent in so many ways.
00:10:57
Speaker
that we're just sitting ducks. Yes, absolutely. And you know it's been kind of a tragic watching people succumb to this because very early on, we had the Blake Lemoine case where he thought they were conscious and called a lawyer and tried to get file a habeas corpus petition.
00:11:15
Speaker
And now we have people with AI psychosis. It's you know it's just very hard not to ascribe a mind to these ah machines and to assume they know what they're talking about, assume they have agency and an identity and ah the whole lot of it.
00:11:33
Speaker
So just kind of a basic hardwired aspect about us and hard to avoid.
Evaluating AI Comprehension
00:11:38
Speaker
Yeah, I wanted to like highlight for the listeners how what sort of a perilous situation we're in, right? Because on the one hand, it's so natural, as you just explained, to describe to them minds.
00:11:53
Speaker
um But when we have frameworks like the one you're going to present today for kind of you know testing that, inquiring that, So yeah, there really is um some some glaring problems there um to yeah that you you make you make obvious, but they're not you know ah they're they're intuitive once you present the framework, I guess what I'm saying. I guess I'm already lauding your paper.
00:12:17
Speaker
Well, yeah, maybe we can kind of jump, kind of try to describe like your your overall positions. at this point, just so they yeah the the listener can kind of get a sense of the big picture view here.
00:12:29
Speaker
so Sure. um It seems like you know your thought is, no, large language models do not understand what they're talking about. and The reason they don't understand what they're talking about is that they lack the conditions required for something called intentionality. They lack the conditions for intentionality.
00:12:49
Speaker
And you actually in your in your work, you've kind of talked about different conditions. There's multiple conditions that something would need to meet in order to count as having intentionality. So so um maybe we can start with intentionality. you know like what How would you define that? And then why is that so crucial for you know something having genuine understanding?
00:13:15
Speaker
Sure. um So kind of a big picture thing. ah Kant is famous for the problem of intentionality. And his you know he asks in the first edition Paralogisms, ah how is it that our thoughts and experiences can be about the world? And this aboutness question, how is it that we have all these you know sensory experiences and data that happened to us, and we know they're not just our own subjective experience. They're actually about something independent of us ah that you know persists and can falsify our beliefs and the whole lot of it.
00:13:51
Speaker
And he says, in order for you to have this, there's a bunch of conditions that need to be met for intentionality, for our thoughts to actually be about the world. um And some of them are what he calls some pure concepts, space, time, objects, causality.
00:14:10
Speaker
um Some of them are formal conditions. He says you kind of have to have a mind unwritten by logic. ah Some of them are rational principles about how the world all hangs together. And the last one is a unified self.
00:14:26
Speaker
um And so the paper is basically saying, look, this is a big problem. we have this question of how do we know if these models are talking about the world or are just using words?
00:14:39
Speaker
And what we want is some way of deciding are the words about our world or are they about something else? and Or about nothing? Are they just you know purely parroting things they've heard as you know the Emily Bedders stochastic parrots?
00:14:57
Speaker
And Kant's conditions provide a mechanism for saying, well, look, here's a way of deciding, do these things actually know what they're talking about? are they Do they have true beliefs? do they have you know Are they making claims? Are they making assertions?
00:15:15
Speaker
um kind of Are they actually using language as full-blooded language? Or on the other hand, you know is it just play of words?
00:15:26
Speaker
The other thing though is the paper is supposed to help us realize what people are already doing. um The conditions Kant lays out are also things that we just care about kind of independently and Kant provides a framework for saying like, here's why Noam Chomsky is complaining about this thing. Here's why Piantadosi and Hill are saying that yes, these things actually do have understanding.
00:15:49
Speaker
Here's why Maywald and Ivanover saying, no, no, it's not going to do it. So I'm trying to also provide a framework for you to understand why different people are reacting different ways to this thing.
00:16:00
Speaker
So quick follow up on that. um So, okay, so to, so you're kind of inspired by Kant here. Kant is saying, you know, for a system to have intentionality for, in other words, for it to have like, don't representations that are about the world, um,
00:16:21
Speaker
four conditions would need to be met. And you know so you just ran through that. you know There's some pure concepts. There's some more like formal things about logical consistency, inferential structure, that sort of stuff. And then there's some rational principles. There's unified self.
00:16:36
Speaker
so So one question I just wanted to really quickly ask is, would it be possible in Kant's mind or in your mind for something to like satisfy a couple of those or one of those and therefore have some intentionality, but not full-blown intentionality? So I just want to just ask, do you think, is it possible for something to have like degrees of intentionality to satisfy some of them, but not others of them? Or how do you look at that that kind of question?
Can AI Have Partial Intentionality?
00:17:03
Speaker
So, yeah, there's, um you know, and ah the paper is, is you know, ah kind of a reference to John Hoagland's intentionality all-stars. And in other papers, he sometimes talks about AirSat's intentionality. And what he means by AirSat's intentionality is kind of this derivative notion where, you know,
00:17:25
Speaker
there's you know systems that have a kind of derivative intentionality or they have something a lot like intentionality, but in some ways deficient. um and you know One way of thinking about this, we often with understanding, and I endorse this as well, I think if you want to say these language models have understanding and a really loose sense of like being competent with language, I'm fine with that. I don i think i think I've endorsed that in places. you These are very competent with language.
00:17:52
Speaker
The question is, you know Are they actually understanding what they're talking about in this deeper comprehension sense or ah even a deeper sense than that, which is kind of what Kant's after?
00:18:04
Speaker
um But, you know, when we study, say, animals, you study in, you know, the crows and they do this cool trick, like they'll drop ah pebbles in water to bring up the to displace some of the water and it raises some food to it.
00:18:20
Speaker
And there's an obvious competence in terms of getting the food out of the tube using the rocks. But then scientists come along go, well, they do they know what they're doing? Or is this just a, you know, dumb behavior? And so they come up with, like, tricks to see if, can you use displacement in other scenarios? And some crows can, some crows can't. And in that case, you're asking about kind of the underlying reasons these things happen.
00:18:46
Speaker
And some crows can't. And that tells you, you can be competent without the comprehension. You can have the comprehension about this problem, but still fail on other causal issues.
00:18:58
Speaker
So I think there's kind of a lot of dimensions here that are separable in the sense that we have lots of um little representations that, you know, you can have an evolved representation to do this task or an evolved representation on the cognitive map.
00:19:17
Speaker
But what Kant's after is a kind of rational mindedness where you actually have kind of a comprehensive understanding of the world. It's not just piecemeal. It's not just ah different competencies that you've, say, evolved or have dis innate dispositions towards.
00:19:36
Speaker
It's a real capacity to make sense of your world that he thinks is uniquely human. So derivative, yeah, I think lots of animals can have kind of a derivative one that's not going to have the logic aspect, not going to have all the rational aspects, are going to have some, you know, gaps in their intuitive physics and so on. But the full sense of it that we would expect for somebody who uses language effectively, as effectively as these language models, no. If you're going to use language as effectively as a human model,
00:20:10
Speaker
you should probably have the whole gamut there.
Baseball Metaphor for AI Intentionality
00:20:14
Speaker
All right, good. So we have ah we've made that distinction between, I guess, maybe understanding in scare quotes or whatever, and and the the kind of Kantian sense of understanding. ah but This is all so ah you know it's so ah abstract, right? I think we should start talking about baseball.
00:20:31
Speaker
Okay. that That makes it easier, i think. So maybe, i guess you you do borrow this from Hoglund, and so you can tell us about the baseball metaphor that he used and how you're using it or adapting it.
00:20:47
Speaker
And yeah, just kind of walk us through some of the bases. Yeah. So ah the baseball metaphor was just a way of ah lining people up in terms of kind of On the one hand, ah you know, some degree their conservatism, the right side of the field was a kind of more traditional cognitive science. ah In the middle was a little bit looser, ah but still part of the traditional form of cognitive science.
00:21:16
Speaker
um And then as you went left, it got to be a bit good deal more pragmatist on Hoagland's conception. um I've kind of... moved away from that slightly, but my first base still is the traditional cognitive science.
00:21:31
Speaker
Second base is still broadly speaking something like Dennett's intentional stance and the sellers and kind of that tradition. um And then third base, shortstop is an additional position um about objectivity that I thought was really missing in the original Hoagland paper.
00:21:51
Speaker
And then the shortstop position actually is still somewhat the pragmatist position, um and it's going to talk about self. So first base will talk about traditional cognitive science and formal competency, the stuff that Chomsky and others are worried about.
00:22:06
Speaker
Second base, kind of um some of Dennett's concerns with rationality um and the fact that a rational mind needs to have some purposive behavior.
00:22:18
Speaker
shortstop with objectivity and uh third base then with the self okay uh so chomsky is on first right so uh let's uh i You know, it just sucks because I'm not an avid baseball watcher. So ah so I should say right now, where we just ah bunted and we're moving toward ah ah first base.
00:22:43
Speaker
ah And so ah that's where we talk about formal competence. And so, i don't know, maybe you can tell me if this is ah an example of formal competence that might please everyone, right? So um obviously, la audition ah can tell you all about a disjunctive syllogism or whatever.
00:23:02
Speaker
But even like ah maybe hunter-gatherers, right? They chase some prey into a cave that has like ah a Y formation and they're right there at the ah the you know ah the crossroads.
00:23:14
Speaker
And one of them goes left, the other one stays right at the crossroads and they come back. No one's there. ah The prey isn't there. So they infer, oh, it's it's going to be over here. So they get ready. And so even hunter-gatherers seem to engage in this kind of ah reasoning. That would be a disjunctive syllogism, I was i would say. So it's just an example of some you know basic ah practices of humans that you know highlight our capacity for formal competence.
00:23:43
Speaker
Yes. ah I mean, there's lots of examples of this and, you know, um a lot of our ability to reason depends on this. You know, if you engage in hypothesis testing, which is what you were just talking about, where, you know, you hypothesize the animal went left or right and it didn't go left, therefore it went right. There's kind of this underlying logic to it. That doesn't mean that there has to be an underlying um logical engine.
00:24:10
Speaker
ah But you do have to go through something like um the same kinds of processes, even if they use some other mechanism to get there. um And for the Chomsky crowd, this was always, they always just said, well, no, there is a logical engine at the bottom of human understanding. And The reason they thought so is because we, especially with language, engage in lots of really complicated um kind of operations.
00:24:43
Speaker
I mean, you know, you see this in mathematics, but mathematics is undoubtedly learned in like the we really drill it in. And if you don't drill it in, ah humans don't don't pick up much in the way of mathematical sophistication.
00:24:57
Speaker
ah But language, we all pick up on language. And we do things in language that, logically speaking, are actually really complicated. um you know And a simple one is ah the red box was on the blue cube or whatever.
00:25:12
Speaker
In this case, logically underneath, you have to have f of x, g of y, r, x, y, as all the logical operations capturing which one is the blue box, or red box, which one is the blue cube, and which one's on top of the other.
00:25:30
Speaker
And for Chomsky, Chomsky said, that's like a normal sentence for us. That is a normal thing we do. um It's a normal part of our vision, that our vision does things like that, where it recognizes these kind of relations. And we do it really intuitively. we do it really quickly. We're able to vary it up.
00:25:47
Speaker
um and so... a lot of their argument just came down to, look, the only thing that we know does this kind of thing rapidly, effectively, um and and is capable of making inferences based on this, logical inferences based on this, is something like a computer.
00:26:06
Speaker
So let's just assume at the bottom um you have a language of thought and it's not like, you know, just a written language, it's a programming language, but that programming language is the mechanism you use to, you know, reason through problems. And so if you're trying to figure out how humans use language, You need a logical engine at the base of it. And that's the first base position. And, you know, as soon as ChatGPT came out, I think it was Noam Chomsky and some of his friends wrote an n New York Times article and they said, this doesn't tell us anything about language.
Formal Competence in AI Understanding
00:26:41
Speaker
You need something that reliably does things like recursion, like he said, she said, or, um you know, the embedded operations in mathematics. And he says, don't see that here.
00:26:52
Speaker
This is not happening. This is a dumb system. It's just processing statistics. It doesn't know. ah The
00:27:01
Speaker
superficial form doesn't really reveal any underlying competence with logic. And as such, you know, it's really going to struggle when you start probing it.
00:27:12
Speaker
And so that was the argument of the first base position. Just a quick question here. Isn't that where the where the term stochastic parrot came from in one of those exchanges from one of Chomsky's buddies or something like that? The stochastic parrots came out of an Emily Bender and Margaret Mitchell paper, um,
00:27:33
Speaker
when they were back working at Google, I think, when I think Mario Mitchell was working at Google. um And their argument was actually just, you know, these things, they had kind of a ah something closer to a Searle argument that um really...
00:27:51
Speaker
These things can't have communicative intentions. And so, yes, they're saying things, but they don't want to accomplish anything with that in your mind. So I don't say, you know, if I say the cat is on the mat, it's because i want you to think about the cat being on the mat. If I say it's a little cold in here, it's because i want you to close the window. And they say they don't have any of that, so they don't mean anything.
00:28:13
Speaker
They're just using words. they're just stochastic parents. They use words, but they don't want to accomplish anything in their hearer. There's no... performative dimension of it where they want to accomplish, you know, they want to make assertions in the game of giving reasons or whatever.
00:28:30
Speaker
So they just thought it was all empty play. um That's my memory. And then they went on to talk about how environmentally damaging these things were in 2021. So it was a little prescient as a paper.
00:28:42
Speaker
um But the Stochastic Parrots thing ah has been kind of a lightning rod ever since, is I think safe to say. But just to clarify for the the listeners, you know, when it comes to traditional cognitive science, position um like you said, you know, they're kind of thinking, hey, the mind must be basically a computer because the only a mechanism we are aware of, which is capable of generating formal competence is computational as computers. So I guess the mind, but that's, that's what must be what the mind is doing. Cause it's like, that's the only thing we know that's capable of generating this kind of formal competency. Whereas it correct me if I'm wrong, but I was thinking, you know, in your paper, you weren't necessarily like committed to that view. What you're kind of saying is like,
00:29:31
Speaker
hey, ah regardless of whether, you know, mind is literally computer or something like that, understanding requires formal competence. So if LLMs, you know, if language models are going to really understand what they're saying, at the very least, they need formal competence. And there are various indications that these language models lack formal competence. Is that like a fair description of your view?
00:30:00
Speaker
Yeah, absolutely. I mean, um so, you know, and all we can punt for it for now, but I do mention the second base position. The empiricist has always argued that we can gain formal competency by learning language. So you don't need language to learn language, which is the Chomsky argument. You need to have a language of thought in order to learn natural language.
00:30:22
Speaker
And the settlers line, by contrast, we can you can learn natural language and in the process you gain a language of thought. um So yeah, I think it's, I think it's just an open question. um But the first visit base position is really good because there is a large body of researchers who just want to study this question. And so you have Ellie Pavlik up at Brown has just done tons of work on variable binding, ah which is, if I say, you know, the blue cube on, or blue,
00:30:54
Speaker
what did I say? It's red box on the blue cube. Uh, you know, is the system going to bind the right property to the right object? Is it going to get the, the red goes to box and blue goes to cube and which ones on top of which, and she's just been really focused on this. Um, and I know Jake Russon at Fordham, uh, worked on this as well.
00:31:18
Speaker
Um, you know, totally obsessed with this question because it's at the root of compositionality. If you are able to put together a bunch of words together in a sentence, you need to make sure you're able to put them in the right order and the right structure. um As the old line goes, there's all the difference in the world between, you know, dog bites man and man bites dog. But if you, early language models couldn't recognize it because the statistics, man bites dog is statistically very, very rare. And so if you gave them ah a story that includes a line about man bites dog,
00:31:57
Speaker
They would almost always invert it because it was so statistically unlikely. And variable binding is really about saying, don't worry about the statistics, just bind the right property.
00:32:07
Speaker
And that's kind of what computers excel at. If you tell the computer, this belongs to this, your computer is going to do an excellent job of that. Language models struggled with it, and, you know, Ellie Pavlik really worked like crazy to show this.
00:32:23
Speaker
And there's other issues, too, that you really we see people working on. Negation's a huge one. um You know, syntactically, or kind of at the level of grammar, there's or statistics, let's say, the difference between Donald Trump is...
00:32:40
Speaker
alive and Donald Trump is not alive is in very minor difference. It's just one bit, ah you know, in in a sentence. But the difference in our world is massive. And so you want the language model, if it's going to actually recognize language,
00:32:58
Speaker
Negation, it you want it to realize these sentences are who describing two entirely different worlds. And we've also had a lot of studies on negation showing, no this is really one of their weakest points. They really just cannot figure out negation consistently. And so if you're ever trying to mess with it, ah negation's just really, really hard for them. So the formal competence is kind of a good place to start because it just kind of rules out certain strong claims about these systems competency if you know what the right at the start that there's also these huge gaps in what we would expect from them. And, you know, the negation one's a big one. um
00:33:42
Speaker
Levenstein and Haramon have recently written some papers about this, and they just said, look, somebody said you can tell the what's true for these models because if you encode true beliefs, it'll recode them as much, much, much, much more statistically likely. And we went in there and tried to mess with, add some negation to them, and it really the system didn't recognize any difference, really. um so Yeah, that's that's a pretty good indication they're not really ah having a logical engine underneath. They did not pick that up during their training.
00:34:16
Speaker
Can I summarize what you just said as... ah Whenever there is a test or often when there's a test of formal competence, recognizing ah barr things like variable binding and negation, the LLMs default almost always to statistics, right? So, and that's why there is no engine there. there ah That's the reason why we believe there's no logical engine beneath it.
00:34:44
Speaker
Right. And, uh, You know, there was always this argument, Fodor and Pashillin in 1988 argued, hey look, maybe a neural network could implement a digital computer. Like if you trained it right, it would implement it. And I've seen some people say, oh, yeah, well, they're implementing it. And this is an indication, no, they did not acquire formal competence through language comprehension.
00:35:09
Speaker
So you still have this problem here. and um We can soften that in various ways, and ill I'll try and help that position out some. But the formal competency view is very strongly like, it didn't it didn't get what you were hoping for out of this. It's not the same thing as a logical engine.
00:35:27
Speaker
um But, you know, at the start, we don't know that humans ah have a logical engine. We don't know um that that's how we process language. So the rebuttal to this is always, well, you kind of, this is the traditional cognitive science view, but not everyone's a traditional cognitive scientist. They were much stronger 40 years ago, but now that we have language models showing us the potential for a connectionist approaches to the mind, people feel kind of differently about these things. So, um yeah, First Pace is kind of its own position in these debates.
00:36:02
Speaker
So I don't you know want us to stay on First Pace forever, but I just have one more question about it. you know So just kind of stepping back for a second. um How would you respond to something like this? Someone might be thinking, um you know this formal competence stuff is super interesting. it's you know they It really seems to reveal a super important dimension of the human being.
00:36:27
Speaker
yeah like There are various skills we have related variable binding and compositionality, which are absolutely fundamental. and you know, if if someone doesn't have that, like they're not going to be able to live a normal like human life kind of thing. But going back to the intentionality thing, someone might be thinking, but wait a second, you know, like my dog, you know, doesn't have formal competence, but I also kind of want to say my dog has, i don't know, thoughts about
00:36:59
Speaker
me or has thoughts about a tree or like represents the tree to itself or represents me to itself. So it seems like there's some kind of intentionality in my dog.
00:37:12
Speaker
I mean, this is kind of getting back to what stuff, you know, you already talked a little bit about with like airs, that's
AI Rationality and Logical Challenges
00:37:17
Speaker
intentionality and stuff. But at any rate, ah you just want to address that kind of worry of like, okay, yeah, this definitely seems like crucial to being human formal competence. But on the hand, is it really like necessary for just like intentionality in every situation So yes, so there is this kind of one of the questions. So the problem of intentionality kind of split in two after Kant. One aspect was Kant's question of like, how do you get a representation of the persistent objective world?
00:37:48
Speaker
And then we kind of ask a sub question, kind of a more basic functional level question of like, um how is it that this rat's cognitive map is about this maze rather than that maze?
00:38:00
Speaker
And, and The second problem of intentionality you know is the kind of thing that you adopt teleosemantics for and you're kind of usually able to say, like well, look, they evolved this innate disposition to um you know draw maps of of their environment as they go through it and so on.
00:38:20
Speaker
And great question. I think it's a perfectly good question. but If you want to kind of ascribe rationality to, like my dog, well, then you kind of have to add a little juice to it because it's not enough that my dog can um do kind of normal dispositions. You need it to have some broader competency. And...
00:38:47
Speaker
ah Honestly, that's kind of the bite of second base. So should I just go into? Yeah, yeah, absolutely. Second base, yeah.
00:38:57
Speaker
All right. So um second base in the Hogan paper is associated with Daniel Dennett. And let me just start there. So Dennett was part of the interpretive this tradition. And the interpretist said, look, I don't really care if there's a computer in your head or whatever. I don't actually care how that part of it works.
00:39:17
Speaker
I want to know if it makes sense to treat you as a rational-minded agent. Do you have beliefs and desires? And so if I'm watching my dog, you know um pursue a squirrel up a tree, does it make sense to say ah he believes the squirrel is going to go left and that's why he's trying to cut them off, but then you know circle back to the tree and that's why he's trying to cut them off. um And so this is going to be a case of my dog desires to capture the squirrel. It believes it's going to juke them. So it's making this movement that isn't directly obvious ah just following it, but it's because it predicts the scroll going to do this thing ahead.
00:40:02
Speaker
um And then it's big thing was like, look, as long as it's behaving rationally, um and the interpreter of this position was, um as long as it's behaving rationally, and then beliefs and desires are going to be worth ascribing to it, even if it doesn't have the underlying engine of beliefs and desires. And so if I want to describe my dog that way, great, do it.
00:40:25
Speaker
um But then it then also, step back for a second, he said but look, If you're trying to use this in animal cognition research, you kind of have to be a killjoy.
00:40:35
Speaker
um The romantic is just going to look at my dog and say, oh, it's a rational agent. And then you do something to kind of trick your dog and your dog suddenly just, you know, absolutely flails. And um going back to the the ah good the crow, because this is the example that comes to mind, they did studies where the crows If you guys ever seen this, they'll like bend a piece of wire.
00:41:02
Speaker
So it creates a hook and then they use it to get food out of a tube. And so they were like, does it actually understand hooks? And so they gave it all these other scenarios where a hook might be useful and you can make hooks out of different things or they even gave them a hook.
00:41:19
Speaker
And a lot of the crows just did nothing with it. Or in fact, I think none of them did. And it was kind of like, oh They don't understand hooks. That's actually really disappointing. We thought they would rationally figure out this new tool, but apparently they had kind of gotten this trunk down.
00:41:37
Speaker
And we see that a ton with animals. And this is what Hoagland was worried about with their sax intentionality. We have all of these cases where they have this representation that is so impressive in context.
00:41:53
Speaker
and then you just change the context. It no longer looks like a rational behavior. Another example, just because this one came to mind, I was writing about this the other day, um desert ants. ah They find their way back to their nest by remembering the exact number of steps they took and the exact direction of the steps. And to get back to their nest, they simply reverse direction and take the exact opposite path, the exact number of steps. So some mean scientist added stilts to them when they got all the way to the distance from it.
00:42:26
Speaker
And naturally the ant overshot its nest by the exact ah number, of you know, the exact distance predicted by the size of the stilts based on the number of steps they took.
00:42:37
Speaker
Same issue, totally rational behavior if you're a romantic, you're a killjoy, totally not rational behavior. So this is at the heart of the second phase. you want to describe mindedness, they need to be rational. And rationality for Dennett just means they have to be purposive. They have to be trying to do something and they have to be leveraging true beliefs.
00:43:00
Speaker
Well, they have to be leveraging beliefs to achieve their goals in kind of more optimal way. You know, the chess machine doesn't have to play perfect chess, but it better be trying to win in a recognizable way for you to describe ah rationality to it.
00:43:17
Speaker
And for a lot of people like Pintadosian Hill, and Ellen Pavlik wrote an article where she talks about this too, is that look, neural networks are probably not going to have, or they're just not going to have a logical engine built in, but if you teach them enough language,
00:43:34
Speaker
maybe a logical oil engine will develop and as a result of this, they'll behave like they have beliefs and desires in the way traditional cognitive science talks about and maybe they'll do for variable binding and they'll do all these things, but they don't need to have the underlying thing.
00:43:54
Speaker
um And so you can have a rational agent that behaves in perfectly human way or in a perfectly you know rational way without requiring a specific type of underlining machinery.
00:44:09
Speaker
And this sort goes goes back to Sellers who said, when you teach a child's language, you're not just teaching them the words for the world, you're building a logical world for them. You're telling them about the logical structure of the world. And so you're basically opening their eyes to the rationality of the world around them.
00:44:27
Speaker
um And rationality in this case, the fact that when things happen, they happen for reasons. It takes some time to know what the reasons are, but if you know the ball falls off the desk, the child will at least know something caused it to fall, and they can start doing experiments or whatever kids do. So that's kind of the heart of it.
00:44:49
Speaker
So putting aside the fact that now I've lost all respect for crows and ants, um the question here then is if we you know train um large language models with enough data, so much so that you know they they're learning language and they can actually grow ah you know rationality,
00:45:10
Speaker
ah now that Now the question is, well, when we test them, what actually happens? So ah what what, I guess, ah you know, um what kind of ah tests have ah been run in this ah domain and why might we think that language models don't have this rational behavior?
00:45:30
Speaker
so um There's two kinds of ways you can test this. The first is the first base way. You can just go, all right, look, you say that if um you train them well enough, then they're going to, let's say, like, they have a bunch of true beliefs and they desire this thing, then they should infer, you know, like,
00:45:54
Speaker
I want to go to the store, so I you know need to go to my car. My car needs gas, so I need to go to a gas station. They should be able to make the proper um kind of inferences at the logical level. And so if there's problems in how they're logically reasoning through steps, that's going to be a problem.
00:46:12
Speaker
The other side of that same example, though, is the background knowledge thing. Are they actually forming true beliefs about the world ah that are going to allow them to engage in planning through?
00:46:26
Speaker
um And so there's kind of two sides to it. One is how robust is their logical reasoning capacity gained just from the statistics of language? And the other one is, how robust is their conceptual grasp of the world, their kind of conceptual scheme?
00:46:44
Speaker
um And the conceptual scheme should tell them, you know, dogs are mammals and, um you know, animals. what isn't ah hammers or, you know, artifacts and these various categories, but they should also teach them things like um the steps you'd have to go through if you needed to fix a car or the steps you'd have to go through if you wanted to, um you know,
00:47:13
Speaker
repair a flat tire. And so you're hoping that it really ends up with what is historically been called common sense. um And if you get both aspects, then the machine has it. The machine has gained a background understanding of the world and it's able to ah engage kind of using the logical machinery. And this was always kind of the hope of the sellers, the Brandoms, you know, the McDowells and others where you they go, hey, if you learn language,
00:47:42
Speaker
You'll be able to engage in conversation. Conversation involves lots of arguing with each other. You may have beliefs that are true and you argue somebody and you have, you know, your belief is falsified. so you have to make new changes to your belief and you'll be this rational person engaging in these kind of things.
00:48:01
Speaker
ah You know, the giving and oh what is the phrase? The space of reasons and the giving ah and ah demanding justifications. All right so the tests of these things.
00:48:14
Speaker
ah Formal competency tests. Another set of tests is just how well they understand the world, just their common sense comprehension. um On that stuff, sometimes they do well if it's kind of the kind of comprehension that just requires them to explain concepts.
00:48:32
Speaker
But using concepts has proven ah more sticky. Planning has proven ah more sticky. And those are kind of the issues that come up at the shortstop level.
00:48:43
Speaker
um And then there's the final one, just kind of like, are your beliefs consistent? um Are you able to update your beliefs in a rational way? These are areas where language models really are just, there's basically no hope for um in the way they're currently designed.
00:49:01
Speaker
ah Because like I said, negation is something they struggle with. They also don't really have ah memory in the way we consider memory, where you store information as true sentences. These are all the things I believe. you know However we store it, we definitely have a set of things but that we're committed to.
00:49:24
Speaker
They don't have anything like that. That's not how they store information. So um there's issues about how they're dealing with you know, their beliefs that just, it's not consistent. It's not going to be a series of true beliefs um that are being held and they're going to be committed to and that you can predict their behavior.
00:49:47
Speaker
And they're not going to be able to leverage their true beliefs because they don't hold a consistent set of beliefs in order to accomplish lots of different goals. So that's kind of where those problems arise.
00:49:58
Speaker
show up. But again, you know you can kind of modify these systems to achieve better and worse ends on this. It's just, this is the stuff where second base people are really you know getting pushed on.
00:50:10
Speaker
If you want to say that you can acquire these capacities, why are they still struggling with these formal competencies? Why do they have this common sense deficit? And why can't they tell the difference between true and false? Those are the kind of the questions that continually struggle or flummox second base.
AI and Intuitive Physics Failures
00:50:28
Speaker
When it comes to yeah the the whole like true false issue, does that connect to shortstop? So the shortstop position is my understanding is something like you know to have intentionality, to have a mind, to understand, to genuinely um genuinely understand what you're saying.
00:50:45
Speaker
um a language model would require a kind of objectivity. like it it it It needs to grasp that it's words are about an objective mind independent world language independent world, at least.
00:51:01
Speaker
Yeah. Can you kind of tell us a little bit about this position, maybe how it relates to the second base and that sort of thing? Sure. So there's two aspects to that. One is the mind independent world. It's going to determine which of your sentences is true and false.
00:51:18
Speaker
um And that's just kind of a big problem. um And it's one that these systems just are not really equipped for right now. Like as long as things like the negation issue and um kind of the the way ah they store their beliefs in ah their weights,
00:51:36
Speaker
where it's not in terms of things it's committed to and things it does not believe, um you're just going to continue hit problems with its true belief issues. um This is a stuff Herman and Levenstein have been really working on.
00:51:51
Speaker
um The other part of it though is yeah the objectivity question of world models. And the world model debate is like you know how is their intuitive physics? Are they going to make the right predictions? Are they going to make the right inferences about how things are going to happen?
00:52:10
Speaker
um Can they ah plan for counterfactual situations? And that's really at the heart of the shortstop position. um World models are the idea that um there is a specific ah way the world is, rules, things in and out, and And we've been studying this at the shortstop position using a lot of different approaches.
00:52:36
Speaker
Maywall and Ivanova, especially Anna Ivanova, has been doing, um she's created kind of a huge playground with all these different puzzles to test how well they understand.
00:52:47
Speaker
that objects exist over time and gravity and the difference between fluids and other types of, um you know, solids and and gases and so on, and just kind of probing things. And it's a simple thing to probe because you just ask human human children even, like, what's going to happen here?
00:53:06
Speaker
um Usually, visually, it's it's much easier with children. They often can't figure it out in a word problem, but you show them a video and they do really well with it.
00:53:17
Speaker
Language models, sometimes you do do the word problem because that's what they're actually better at, the video flummoxes them. So different tests, but um underlying it. great tests.
00:53:28
Speaker
Language models have really struggled with it. There's been some kind of in-principle stuff. A fellow GPT was designed to figure out, look, maybe these things can encode objects and space and time and all of, or at least space and some of these other things.
00:53:46
Speaker
And the results have really been mixed. It's not clear that they're able to disentangle things like objects and space in the way we would hope for. Um, instead of recognizing objects can appear anywhere in the world and space is the same everywhere, um they seem to really struggle with concepts like that. They instead, ah in the Othello piece, in order to understand where a piece was, it would memorize all of the other ah pieces around it And so instead of learning like
00:54:22
Speaker
you know, F4 is black, which is an Othello thing, you would do, if piece X is here and piece Y is here and piece Z is here, then this is black. And it was like, that's a really convoluted way of doing it.
00:54:37
Speaker
And as a result of that, if you tweak it at all, it's going to just absolutely fall apart. You don't want a rude Goldberg machine for your world model. You want a really simple, like,
00:54:49
Speaker
this location is this object, and this object can also be here, can also be here, can also be here. So this has been something that we've really kind of seen as a weak point for language models. They can pick up a lot of common sense knowledge in the sense that, you know, Americans drive on the right side of the road, but the common sense knowledge embodied in our intuitive physics has proven really hard.
00:55:13
Speaker
um And a couple of quick examples from intuitive physics, just so people know kind of what we are testing for in these kinds of puzzles. um One example is you'll see, ah you'll watch a video of billiard balls bouncing around and you're watching it and the instructor just says, pauses it and says, where is this ball going to end up?
00:55:36
Speaker
Children are very good at this. They they can figure it out. you know um we have We ascribe a noisy Newtonian model to children where, yeah, they're not going to get it exactly right. They'll sometimes think the ball is going to go in the pocket and it won't.
00:55:52
Speaker
for the most part, everybody's really good at this. Language models really struggle with these puzzles. They struggle with puzzles ah that you would give somebody, you know, like,
00:56:02
Speaker
um those stupid iPhone puzzles where you have to use levers and pulleys to accomplish different tasks, or like Incredible Machine, where you do build Rube Goldberg machines. Those the kind of things kids are pretty good at. I loved Incredible Machines as a kid.
00:56:18
Speaker
ah Language models still struggle with it. And then one final thing that we do a lot to test this is just have them play video games. And that's ah Julian Togelius's lab at NYU does this a ton.
00:56:30
Speaker
And Man, man you don't you never hear somebody as disappointed as he is with language models. Why can't these damn things play a game? like You have all of this processing power, and every time we give them even a simple game, even a language game, they just fall apart. And so, yeah, these are all cases where you're like,
00:56:51
Speaker
They should have this competence. um They should understand how to plan ahead and you know predict how things are going to work in space and time. And also we toss into this intuitive psychology, they should be able to predict what other agents are going to do. Intuitive biology, they should be able to understand that if you don't water a plant, it's going to die. um you know These kind of things are things they really struggle with.
00:57:18
Speaker
And that's a sign that language isn't quite sufficient to bring this about, at least not without a lot of scale, more scale than we have currently. I just wanted to say one thing to make sure listeners ah kind of ah differentiate between the different kinds of AI. But I think a lot of people just sort of um have a a homogenous graph. Like AI is this one homogenous thing, right? So you're saying large language models can't play video games.
00:57:47
Speaker
But someone might say, oh, but I thought, you know, the people... over at OpenAI i um or AlphaZero, right, it's just playing games. So that that's not a large language model, right? So that's that's a deep a learning model, but not a large language model.
00:58:02
Speaker
Right. And it's worth flagging. um So AlphaGo, Alpha Zero was a later version of ah AlphaGo, and they gave the exact same model and they trained to play chess and played great chess. They trained to play Go and it played great Go, backgammon, great backgammon.
00:58:20
Speaker
And they trained versions of it later to play StarCraft and sort of played StarCraft. um But the same model could only do one.
00:58:30
Speaker
You can't get the same model to play Go and chess, Go and backgammon. You couldn't even get the same model to play... ah i don't know. Do you guys play StarCraft? I'm going to assume your audience plays StarCraft, despite that they almost certainly don't. ah In StarCraft, you can't play... what the Alpha Star could play as the humans, but it couldn't play as the Zerg. So it can only play as one species on one map. So these are not general models. And that was always supposed to be the promise of language models. You're just like, hey, you're an artificial general intelligence. You can do generally intelligent things, play games, and they don't. And
00:59:10
Speaker
um You can machine to play a game, but right now you can't get the same machine to play lots of games. That's just something that we've really, really struggled with. um So it's kind of a sign that the generality of their intelligence isn't quite what you were hoping for. And that's part of what the shortstop position is kind of helping us with.
00:59:31
Speaker
So I feel bad to abruptly shift to the third base, but I just kind of want to be sensitive
Does AI Need a Stable Self for Intentionality?
00:59:36
Speaker
about our time. So um if we've if we could go travel to third base, then maybe I could just try to describe briefly how I understand third base. And then Jake, you tell me like, you know, if if kind of like I'm getting an idea. So The third base position is kind of like, well, to have intentionality, at least one thing you're going to need is a sort of stable subject of experience maybe, but at least a sort of subject or person over time who has
01:00:08
Speaker
beliefs and representations that belong to it. like you're going to have a belief, well, you need an agent who has the belief, and that agent needs to have a persistent identity, and there needs to be memory that links the past and the present. So I don't know. I guess worry would be maybe large language models don't have Yeah, they don't have a self, and and that's like a really big problem. um So anyway, yeah, that's just my attempt to like introduce the view. like How would you describe it?
01:00:43
Speaker
So yeah, I think that's right. So the Kantian idea is look, you can't have a persistent objective world where, you know, things have persistence unless there's something persistent about your mind, ah that you have to have some stable subject. Otherwise, you can't really understand the persistence of the world. You can't recognize that it's persistent.
01:01:05
Speaker
um but you can have like a really more basic version of this. If you have an animal, like a rat, even if it's experiences, even if it doesn't have kind of robust, rational cognitive model, if you have kind of more of a rile sense of these things, you can just be like, well, you know, as long as it has kind of ah persistent capacity to learn from its experiences and it has the same body throughout its life,
01:01:34
Speaker
that's going to count as a persistent self. And so you don't necessarily ah need to talk about what's happening internally beyond that. What has to be happening internally is that there has to be some, um,
01:01:50
Speaker
The things that you've acquired in the past have to still be retained in the present. So most language models are going to have that in the sense that they've been trained and they remain um the same over the course of their training. So you can kind of get a loose version of that if you want to identify it just with the language model, for example.
01:02:10
Speaker
or the rat's body. But there has to be something that's the same over time on this conception, um even if it doesn't have an agent that recognizes it's the same over time.
01:02:24
Speaker
That's kind of a separate a question. um So the separate question then becomes like, well, do they have the capacity to recognize that they're the same agent over time?
01:02:37
Speaker
That's something, you know, the rat, I don't know if rats can do that. Maybe they wake up every day, a new rat. Good for them. um Buddhist rats. So totally possible. well Humans, we don't have that privilege. oh You are required to wake up and honor the contracts you made yesterday.
01:02:57
Speaker
You are required to wake up and, you know, if you wake up with a ring finger ah or a ring on your finger, you you better honor the ring. You wake up and there's a kid crying at you. better go, that's probably responsibility. And so part of what Kant was after was not only do you have this persistent body,
01:03:16
Speaker
and these persistent kind of, you know, habits and memories, but you've also in the beliefs, in the judgments you've made in the past form beliefs about how you think the world is. You have decided the germ theory of disease is true and therefore you vaccinate your kids and, um you know, you the ah Einstein got it right and so you will enjoy some science fiction and other stuff you're going to find stupid.
01:03:43
Speaker
Whatever. And he also thought, of course, that the commitments you make bind you in terms of the actions you make, that if you vowed to be faithful to your wife, well, you are committed to that and you should ah live up to that.
01:03:57
Speaker
We expect that from language users because when language users are talking to us, they're often making ah assertions. And we expect that if you say that this is true, that you're going to continue to believe and live like that is true. That is an expectation we have. If you promise me that you're going to do something, um then I, you know,
01:04:22
Speaker
I'm expecting you to do something. I'm going to rely on it. And so I expect you to um abide by it. And that's kind of where language models have really struggled here. um They often purport to have this kind of um rational unity where they're like, oh, yeah, I remember what you were talking about. Or, oh, yeah, um you wanted me to do this thing and I did it. And they're just talking.
01:04:46
Speaker
And they're irresponsible in the strong sense of not doing what they say they're doing, and in some cases being incapable of doing. If you say, hey, you know, language model, promise me that you're going to remind me of this thing in a week, um unless the system makes the right kind of note ah in it, which depends on kind how you design these systems, and a lot of systems won't make the node at all, um you know it's not going to be able to keep its promise. And even if it does, it's mostly because of some extra tinkering the programmer did, not because of anything the language model did. It's because the programmer was
01:05:25
Speaker
ah designed a sub-program or whatever. So that's kind of where the Kantian comes in. He says, look You have a persistent world, um you need to have a persistent person underneath it, someone who takes responsibility for their beliefs and lives up to them and ah can be relied on. And if you don't have that, you're really just dealing with something that's speaking a lot and has a competency with language, but it doesn't know what it's talking about.
01:05:56
Speaker
um And a simple you know example of this is oftentimes if you ask the wrong language model to ah explain an article that you're reading to you.
01:06:07
Speaker
um Some language models actually don't have PDF capacities, um so it won't be able to read what article you're talking about, and it will say, oh, of course I read the article, and it'll make something up. you're like I don't, that's not helpful. I don't want that. But, you know, the language model is just trying to be helpful and they'll tell you they have capacities for PDF reading ah that they don't have. And they have the capacity to search the internet, even if they're not connected to it. And they'll do stupid things like that.
01:06:37
Speaker
So irresponsible agents and Kant would say, yeah, that's that's not a full rational mind. You can't be rational unless you're responsible. Okay, so maybe ah one last question here to wrap things up. We just went through four criteria.
01:06:53
Speaker
ah So it's um having the logic engine, ah rationality, objectivity, or something like a world model, and a sense of self.
01:07:04
Speaker
ah So i I guess I think I know the answer to this question
Future Potential of AI Intentionality
01:07:08
Speaker
here. But Do you think large language models will ever get there to have all four? um and And I guess the deeper question, maybe the one you can spend more time on, if some kind of AI were to have all these, would would that count as having a mind and as understanding?
01:07:28
Speaker
So ah with language models, I'm pretty bearish. um I think this technology, we keep adding scale to it, we keep adding tweaks to it, but we're we've really just seen um limits to its gains in formal competency and kind of, you know,
01:07:50
Speaker
rationality and and kind of purposiveness in the gains and common sense and objectivity. And we certainly don't see any indications or, you know, achieving self, which is probably good for the the paranoid people who are terrified these things are going to wake up tomorrow. um But you know like lots of people are working on lots of things and I'm excited about it. i have never I've always assumed that you know cognitive science ah in the broader sense of um you know minds are machines in some loose sense.
01:08:24
Speaker
I thought that's probably right. And probably some kind of machine, some kind of AI will be able to have a mind. So I've always been kind of hopeful on this and i remain so.
01:08:35
Speaker
I think language models are an incredible technology in a lot of ways. I have my beefs with them, but um I think they're a wonderful proof of principle of what AI can do. And even if that is not a full comprehension. We've learned a lot in the process of studying their failures. So think it's been worthwhile to study them.