Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
AI and Cognition with Daniel Sternberg
318 Plays7 months ago
What can neuroscience teach us about AI, and vice versa, what can AI teach us about our own intelligence?
Joe and Rolf talk to returning guest Daniel Sternberg about advances in AI over the past year. Topics include using the methods of cognitive psychology to understand AI; representation in artificial intelligence; what current large language models (LLMs) are good at and not good at; sentience in AI; the future of humanity; and other important stuff.
Natural and Artificial Intelligence: A brief introduction to the interplay between AI and neuroscience research (Macpherson, et al., 2021)
Can AI language models replace human participants? (Dillion, et al., 2023)
Language models show human-like content effects on reasoning (Dasgupta et al., 2022)
Recommended
Transcript
Introduction and Guest Appearance
00:00:07
Speaker
Welcome to Cognation. I'm your host, Rolf Nelson. And I'm Joe Hardy. On this episode, we have a returning champion.
Advancements in AI: A Year in Review
00:00:17
Speaker
Dr. Daniel Sternberg is with us once again. We were just reminiscing that the last time he was on the show was about a year ago. Actually, a lot has happened in AI since then.
AI and Neuroscience: A Symbiotic Relationship
00:00:26
Speaker
So we wanted to bring Daniel back to talk a little bit more about artificial intelligence and particularly the relationship between artificial intelligence and neuroscience.
00:00:36
Speaker
What is neuroscience teaching us that can be helpful as we develop artificial intelligence and artificial intelligence systems? And then vice versa, what influence is artificial intelligence having on the research and science that goes into understanding how the brain works?
Dr. Sternberg's Background and Role
00:00:55
Speaker
So, uh, Dr. Sternberg is the head of data at Notion and has been working in, uh, data science for a number of years now since he received his PhD at Stanford and cognitive psychology. And, uh, yeah, so Rolf and Daniel, I have had the chance to work together in the past at Lumosity for a time and, uh, have stayed in touch and yeah, great to have you back on the show. Yeah. Thanks for having me back. It's great to be back.
00:01:26
Speaker
Absolutely.
Why AI and Neuroscience Matter
00:01:27
Speaker
Yeah. So starting off with the sort of, you know, the topic of, you know, what can, uh, what is the sort of the mutual influence of neuroscience and AI? It might be useful to start by thinking about like, why is that even an interesting topic to talk about? Like why, why is that something that we would think about?
The Evolution of Neural Networks
00:01:50
Speaker
Yeah, I think like if we go back, uh, to actually our last conversation a year ago,
00:01:56
Speaker
Um, we talked quite a bit actually about the kind of lot like rewinding, you know, 50, 60 years, uh, to when neural networks, the kind of basis for the, all of the models that we're seeing out in the world today, large language models, et cetera, their route going further back, um, really go back to some things people were trying in the fifties, um, forties and some, actually in the forties in some cases.
00:02:19
Speaker
And so this interplay in relationship has actually existed for a long time, and it's had ups and downs.
Is AI Surpassing Human Intelligence?
00:02:28
Speaker
But it's interesting to see now. I'm curious if you go to the talk to AI folks, I think some people would say that we're kind of like trying to surpass the brain. We're trying to surpass human intelligence. It's kind of like take the best of what machines are really good at. This can write code really well, and then it can run code.
00:02:49
Speaker
for example, and the best of what looks more like human-like intelligence we're going to talk a lot more about today. And so I'm curious how you all think about this too as we go through this conversation. Because I'm not sure that everyone on the AI side of the world is just pushing the technology forward and on the neuroscience side of the world necessarily have the same perspective on what the goals are either. So we might end up talking about that a little bit too.
00:03:17
Speaker
No, absolutely. I mean, when I was in when I graduated college and was looking to go to graduate school in psychology and or neuroscience, my stated goal at the time was I wanted to develop a system that would
00:03:35
Speaker
basically model the brain.
Perspectives in AI: Computer Science vs. Human Cognition
00:03:37
Speaker
In other words, you could have a brain that was in silica. So with the sort of mutual benefit of having a really smart computer and then also having a way to study fundamentally ourselves as human beings, like to use the artificial brain as a way to understand better our own experience and our own brains.
00:03:59
Speaker
So that was like my perspective coming from a psychology background and interested in human beings. But to your point, someone who's coming at it from the perspective of wanting to solve a specific problem in computer science doesn't fundamentally care that there's any relationship between the brain and how their computer system works.
00:04:25
Speaker
So I think there's a few different ways to take that. I think we can explore these different
AI Models and Human Brain Functions
00:04:30
Speaker
elements. But one of the things to consider is, in the current state of the art, how much are these new systems that are really exciting, like the large language models, some of these image recognition models, how similar or different are they to what we understand about how the brain works right now? Yeah, and this is where we have
00:04:53
Speaker
So we were talking before about how we have three cognitive psychologists here, and it's not typical. Three cognitive psychologists get together and talk about this kind of stuff. But I think we probably all think about this as using methods from cognitive psychology to kind of probe the mind of an artificial intelligence. And I know that's one thing we'll be getting today. So so, you know, in the
00:05:19
Speaker
It's not straightforward or necessarily easy to figure out what's going on in the mind and the methods that have been developed over a long time. I think it's really interesting that we're to the stage where we can now treat a model in a way that's like a human mind and we can begin to unravel. To what extent does it have the same kinds of biases, shortcuts, kind of heuristics that regular people do?
00:05:47
Speaker
does this all fall out from the data that's being collected for these models? So yeah, so that's certainly one thing that I'm really interested in.
Learning Mechanisms in AI and Brain
00:05:55
Speaker
Yeah, I mean, thinking about, say, for example, taking specifically the large language model. I think it's useful to think about both from the behavioral perspective of how similar or different are these models to how they behave, but then also underlying
00:06:15
Speaker
way that the computations are happening, how similar is that to something that's happening in the brain versus something that's very different from what's happening in the brain? So there are aspects of these models that I think if you zoom out a little bit, you can think of them as being similar in certain ways to the brain. So for example, the way that they learn from input
00:06:44
Speaker
in a statistical manner matches on some level the way that groups of neurons and neuroplasticity happens, the synaptic plasticity, et cetera. But if you drill down into the details, the models will take shortcuts that brains can't take.
00:07:09
Speaker
In an article I was reading recently, you're talking quite a bit about how, as I learned in grad school, backpropagation is a core part of how these models work. Backpropagating error signals, so they learn from how close their output matches to expected output in some way. For example, predicting the next word or something like that, or the next token in the cases of the current generation of LLMs, which are
00:07:36
Speaker
you know, chunk either like a letter or chunks of letters. But
00:07:42
Speaker
But the way that they do that by backpropagating that error through many, many, many layers of a system is not really exactly how a brain works. And so what you see is in more computational neuroscience, neuroscientists for actually decades now have worked on different types of models that can do something like error propagation in a way that is more biologically plausible.
00:08:10
Speaker
than the feedback signal flowing through the systems in the way that an error signal is in a model. And I think there's just a question sometimes to me of also, to what extent is
00:08:22
Speaker
At what level do we want to model a
The Role of Error Correction in AI and Brain
00:08:25
Speaker
brain? Do we want to model a brain by literally recreating all of the connections between neurons in a brain, making sure that we model all of the biological processes going on, et cetera? Well, obviously, someone doing AI is not interested in solving those problems necessarily unless in some way it enables them to do something they couldn't otherwise do.
00:08:45
Speaker
But nonetheless, I think a similarity between the models and how a brain works, in my opinion, is that you are at some level taking basic input in some modality and trying to use it in some error correcting way, which we know the brain does do.
00:09:09
Speaker
The brain does have systems that involve error correction to enable learning, whether that be cortical, whether that be reward processes, etc. These all exist in brains and at some level where there are analogous problems to be solved from input to output and prediction in AI systems. But the exact details of how they go about doing that vary.
00:09:37
Speaker
and vary from the brain. Yeah. But it is interesting how much in neuroscience, in the nervous system broadly, but in the brain in particular, feedback is really important. So that was one of the things that I found surprising learning about the visual system is if you look at the signals going from the retina to the thalamus,
00:10:01
Speaker
Uh, you know, most of that signaling is actually happening in a feedback direction. Like the vast majority of that signal is actually not coming from the eye into the brain, but actually from the brain back into the eye, which is surprising. Uh, you know, it just, it shows how important that role of feedback is and, and neural processing as well.
00:10:22
Speaker
Although I think the kind of feedback that you're talking about with back propagation would be a little different than that sort of corrective feedback. That's a question, yeah. I don't really know how different or similar it is. Well, maybe I may have just a really rudimentary understanding of it, but I'm thinking of just something simple like a classifier where you're trying to determine if something's a tank or a car.
00:10:49
Speaker
So, you know, with that back propagation signal, you're sort of telling the ground truth whether that picture actually does represent a car or a tank. So you have accessible to the system something that humans don't necessarily have access to, which is sort of what the ground truth is, right?
Understanding AI Representations
00:11:09
Speaker
So that, I mean, that's what I think of as sort of a difficulty with that kind of back propagation.
00:11:15
Speaker
I mean, there are a number of attempts to create something that kind of works like backpropagation, but is more gnarly plausible. And supposedly, while there is no evidence that backprop
00:11:34
Speaker
itself is possible in a brain. There are different types of approximations that there is some evidence may exist in the brain, though I'm not an expert in mapping that back to brain processes specifically. I think a different direction we could go here is to
00:11:54
Speaker
Is the mechanism the same as the mechanism in the brain? Separately, there's also are the representations that models get similar to representations that we observe in the brain. Even if the exact nature of the algorithm to get there looks different. And that's been something that's even before this current wave of large language models and
00:12:17
Speaker
vision models, et cetera, of this current generation, there's been studies for actually decades kind of showing how you can through some sort of statistical learning model end up building representations that look similar to representations in visual cortex, for example.
00:12:35
Speaker
So there's a bunch of research from the late 90s, early 2000s looking at just basically learning the statistics of natural scenes and how you actually end up building out representations that are similar to the representations in primary visual cortex, etc.
00:12:55
Speaker
And just noting the idea of the point being many statistical learning algorithms with certain characteristics will end up with representations and structuring the world in a way layer by layer that looks similar to what you actually see in a brain.
00:13:14
Speaker
Um, which, which speaks a little bit, I think to the, um, one of the things I find interesting about thinking, looking at these models and, um, you know, similar to, I think things we might talk about later in this conversation around language models as well, where there's sort of just a, you know,
00:13:31
Speaker
put the exact biological plausibility of the mechanism aside for a moment, you can learn a lot about the statistics of input that humans likely receive over the course of development just from taking a model that's really good at pulling out statistical information and assuming that the brain has various ways of doing that as well, though obviously the input that we get looks a little bit different than just being trained on the text from the internet or something like that.
00:14:00
Speaker
Yeah, no, absolutely. And I mean, I think that speaks to the point of, you know, as we think about similarities and differences in artificial versus we say, call natural intelligence or human intelligence. One of the key things that drives them in the same direction is they're operating on the same world. But, you know, the
00:14:23
Speaker
For the most part, when we are working with artificial intelligence systems, we're interested in solving human scale problems. We're interested in doing things that we find interesting and useful. That's just because we're human beings and we care about things that human beings care about. So we make the artificial intelligence care about things that human beings care about. So we train them on inputs that are
00:14:47
Speaker
the kinds of inputs that human beings would use as well, like things that are visual inputs to a human eye, things that are auditory inputs to a human ear, words, you know, these kinds of things that are human scale inputs from the world that we all live in. So it, you know, just if you think about homologous, you know,
00:15:12
Speaker
structures or, or frameworks within these systems, it would make sense that there would be some similarities because they're operating on similar statistics in the natural environment. And I think in, in the large language model, this sort of speaks to, you know, the, the next layer up is in a large language model, for example, which has been trained on the corpus of say, all of the internet, it has the statistics of the internet and it's sort of worldview, if you will.
00:15:41
Speaker
And it tells you a lot about what's out there in the internet as we talk about biases and things like that. But what I find an interesting question here is, to what extent does, based on the way that the large language model work, does it actually have a representation or a worldview that's somehow embedded in its structure?
Do AI Models Have a Worldview?
00:16:05
Speaker
Because a lot of the feedback systems that the way that the brain works is that it has sort of like a
00:16:10
Speaker
canonical view of what should be there, makes predictions about what should be there, and then is working off an error signal of how different the thing in the world is from that. It's got kind of like a worldview or an understanding of the world embedded in it in that sense. And to what extent does the large language model, for example, have a worldview embedded in it? I think I feel like I might be repeating myself from a year ago, but I think it has
00:16:41
Speaker
more of one than we might have thought. I find myself kind of in the middle of this debate, which I find myself in. I think it is surprising the extent to which just training on text can allow you to build up
00:17:05
Speaker
more complex representations. And I think some of the things that the models are able to do, yes, it is quote unquote, fancy autocomplete. That's the kind of skeptical take on the whole thing. But to get as fancy as it is, I think there are some representations that you need to build up over time. However, I don't think it's enough. And I don't know the extent to which it's
00:17:31
Speaker
the multimodal models that are now being trained in a lot of cases. So a lot of the really recent models that we've seen come out post GPT-4 have been actually multimodal models or some version of them is multimodal, which means that they have input like video input with text attached to it. They have image, audio,
00:17:54
Speaker
All like many modalities simultaneously being trained but Which may help some but they're not like a robot that's walking around in the world and getting input from a bunch of sensors but
00:18:08
Speaker
But I think it's been surprising that it picks up on more things than we would think of as being complex and requiring some level of world knowledge than we might have thought was possible with just these models alone. On the other hand, I do think the
00:18:27
Speaker
Folks who argue that, yes, these models seem quite powerful, but there are additional systems and processes that we have not built yet. We haven't really created separate systems for planning, for example.
00:18:45
Speaker
like planning an action. And so like even planning an action are still dependent on just like iterative, kind of like a recursive loop within these models, like asking these models to like do something, take an action, prompt themselves to do something else, et cetera. But it's all still just happening with this one large language model that is trying to pick highly probable next outputs.
Limits of AI in Mimicking Human Intelligence
00:19:11
Speaker
So I do think like,
00:19:13
Speaker
I don't know that we're going to hit a wall, but this is very much personal and not where I work or anything like that. I personally think that there are other
00:19:24
Speaker
There are other components of a full artificial intelligence system that we will need in order to really achieve what we'd look at as human-like intelligence in a number of cases. And so it'll be interesting to see as we especially start to work more on things like more autonomous agent-like behavior in these models is the LLM plus prompting plus a couple of other tools it has access to enough.
00:19:52
Speaker
I wanted to drill into the idea of representations because I think that's a really interesting, like you say, if you think of it as just a glorified autocomplete, it's hard to imagine that there is a real representation inside an LOM. But then again, it's not really easy or obvious to see how we have representations instantiated in our brains too.
00:20:19
Speaker
There's, you know, you can, neuroscientists don't, you know, you can't just find a representation in there by cracking open the skull and looking. How do you go about looking? I mean, again, in cognitive psychology, we have ways of sort of indirectly testing, you know, and you have some, some intuitions about this too. So I guess that comes with just interacting with the model. But how do you, how do you sort of figure out the parameters of what a representation might be for an AI?
00:20:49
Speaker
Yeah. I mean, in an AI, how do we look at representations? I mean, historically, you would look at statistics of the, provide an input, look at the representations in internal layers of that model, and look at them statistically, essentially. You're looking at the weights on different
00:21:14
Speaker
We look at the weights. You're also just looking at if I give you a particular set of inputs, what is the activation in one of those hidden layers look like at different points along if you're having something that's like a recurrent. Well, back in the day, this was me aging myself. They don't use recurrent neural networks anymore. They use these transformer models now, which don't have to have recurrence and have all these nice properties.
00:21:40
Speaker
Um, back when, uh, I was doing like neural networks in the dark ages. Um, we, uh, the dark ages of the, you know, uh, early 2000s in my case, um, you know, we was like the winter, you know? Yeah, exactly. Exactly. It was all about, um, this is a real aside, so you feel free to throw it out, but it was all about, uh, you know, uh, when I was in grad school, it was all about like Bayesian non-parametric, uh, models and the Bayesians were really hot in the mid to late 2000s.
00:22:09
Speaker
And then in ML, in the ML world, it was like, I don't know, different types of models, which I won't go into right now. But, but back then, like, you know, you could do things like, you know, I would, if I were having a model that was trying to, you know, predict the next word, I would just look at in this like hidden layer or context layer, what was that representation like? So, you know, there's a model from the, this is a very simple model from the
00:22:37
Speaker
early 90s called the simpler current network, which is Jeff Elman's model from Jeff Elman, who's a cognitive scientist at UCSD. It was a really simple model. You basically had a feedforward neural network with three normal input layer, hidden layer output, and all you would do is essentially feedback the prior hidden layer activation as another set of inputs so that it had some prior context from that.
00:23:07
Speaker
The problem with it is it loses context really rapidly. That is not enough memory, essentially, to do a good job. Many models since that time have built up way better ways of holding on to context over long periods of time, and Transformers are kind of the latest iteration on that. But yeah, you would look at literally at different points in a sentence what was the representation in that.
00:23:28
Speaker
in that hidden layer to see what did it know about the sentence. You could do hidden layer activation comparisons, you could do principal components analysis, you could do all kinds of statistical analyses to try to build up a story about how they work. There's much fancier ones now. Right. On a smaller model, it would seem like you could get a handle on things, but when you've got 175 billion parameters or something,
00:23:56
Speaker
Is it something you can make any sense out of under the hood? I have not actually tried to do this recently. I would say what I see people doing more often, there are some fancy statistical techniques, and I've seen some very cool visualizations and stuff like that, and I forget exactly how they're built. But the other thing I've seen more commonly with these types of models is try to get at some idea of what the representations might be
00:24:27
Speaker
by looking at what predictions they make, for example. In a similar vein, so I'll go maybe down a rabbit hole for a moment.
Enhancing AI with RAG and Semantic Retrieval
00:24:40
Speaker
One of the big, maybe in the last year, one of the big things that's been happening in AI world in terms of applications of these models has been this thing called RAG. You know what RAG is?
00:24:54
Speaker
So RAG is retrieval augmented generation. So essentially, a lot of product features, we have one at Notion, which is like our Q&A feature, which we launched late last year in beta, is you take a large language model and then you give it access to a way to retrieve content, like basically do a search of content from somewhere.
00:25:22
Speaker
Or another product I really like called Perplexity, which is a LLM-driven web search product that's for knowledge. It's really, really cool. It works really well. And you're giving these models a way to search and then take some content back. The basic idea, I won't give away any secret sauce, but the basic idea, you can look this up anywhere, RAG, with the acronym R-A-G.
00:25:51
Speaker
You basically say, hey, model, you ask it a question. It constructs a query itself. It searches some search system, pulls some end pieces of relevant content, and then says, using this content, answer the question, or provide an answer response to this.
00:26:12
Speaker
Okay. I think I know what you're talking about now. I think I've heard of this. Yeah. And so commonly, one of the common ways that that search happens is through some type of semantic retrieval. And so add in the idea now that actually what people are using is what's called like a vector database where I have taken all of this content, chunked it up in some way, and then I have
00:26:36
Speaker
Fed those, I have an embeddings model. Basically, it's a large language model where you're giving me some internal layer of that model. So I provide a big chunk of text, and it provides me back just a vector, a bunch of numbers from an internal layer of that model. Or they may do something fancier than that and combine multiple layers in some way.
00:27:00
Speaker
But essentially, and then when you are doing a search, you actually retrieve that way. So you construct a new query, and then you find the chunk, the text that is the closest match in this vector space to that query. And so there, again, you can start to see how these vectors are representations from the model.
00:27:21
Speaker
And again, I don't know how easy it is to visualize all of this or the ways people are doing that these days. I haven't been as close to that because I'm not deep in that part of the research. But you can certainly start to see what does it find similar, for example, based on some set of texts. And so that tells you something about what the model has learned about the world.
00:27:47
Speaker
Yeah, I mean, in many ways, that kind of, you know, representation, looking at those representations, based on the response to a certain stimulus input, is a lot like mapping a receptive field, you know, in the, in a brain, in certain ways, like, what is the
00:28:06
Speaker
What are things responding to? And what does that tell you about the role that that particular part of the brain is playing, for example? And in this way, you could imagine probing the neural network in much the way that neuroscientists probes the brain.
00:28:21
Speaker
I think an interesting thing too is in a lot of these cases where, to mention the receptive field of neurons, I mean, this may be an aside, but we have a much easier time understanding the kind of representational properties of non-human mammal brains than our own brains because we can stick electrodes in them.
00:28:40
Speaker
So, I would actually say, reading through some of the articles we were talking about in advance of this conversation, it was intriguing to me thinking about it as like, well, actually, in most of these cases, what we're really talking about is, oh, this recurrent neural network, this aspect of it has similar properties to some primate neurons that we have. Non-human primate, yeah. Yeah, some non-human primate, exactly.
00:29:05
Speaker
Well, I mean, that gets to a good question, which is, you know, you mentioned earlier that, you know, what we're really kind of getting excited about and interested in is, you know, that a lot of times these networks are becoming as good as or better than humans at things.
Defining Intelligence in AI and Humans
00:29:24
Speaker
And I think that this to me is sort of like prompts the question of like, when is AGI coming?
00:29:34
Speaker
And then of course you have to answer the question first, which is what is AGI? What is artificial general intelligence? And I think that I actually know the answer now, which is this improvement over the last year. Tell me. So it's, it's actually very, very simple, which is that, I mean, so first you have to define intelligence and I certainly know the answer to that now, which is as I've been saying for a long time, but now I'm certain of it. It's just.
00:30:04
Speaker
how people think, how human beings think, period, end of story. We say that someone is intelligent to the extent that they think like us or that they solve problems that we find interesting better than us. We would call them more intelligent than us if they're solving questions that we particularly find interesting and important in ways that we may not be able to. Critically, as human beings, we fundamentally understand
00:30:32
Speaker
that someone is not intelligent or not intelligent, there's a range of intelligence, and that there's some sort of some sense that there's some multi dimensionality to intelligence but nevertheless you could say that someone is more or less intelligent.
00:30:48
Speaker
The machines are going to, so if you think that about artificial general intelligence, you will have artificial general intelligence when the machine can solve most or all of the problems that humans can do at least as well as human beings. And so the point is actually, I think we have artificial general intelligence already. It's just not especially intelligent. So if you think about actually artificial general intelligence should also exist on a continuum.
00:31:18
Speaker
All kinds of different systems are solving problems better than humans already. And they're able to solve them flexibly somewhat. In other words, they're not just solving one tiny problem. It's not like a calculator that's just solving long division, but can solve a variety of problems somewhat flexibly, like large language models certainly fit into that category. So there is some level of generality and there's some level of intelligence
00:31:44
Speaker
I think we already have artificial general intelligence. It's just not as intelligent as we are across the full breadth of things that we're interested in in SALT. Interesting. It's also interesting to think about this in the context of we also as humans and in the history of psychology, we decided that certain
00:32:10
Speaker
capabilities of humans kind of fell under what counts as intelligence versus not. And one could argue that
00:32:20
Speaker
even decades ago with like symbolic systems entirely, like there were, you know, it wasn't human like you were talking about before, like the things they were really good at were things actually we found really hard. But like, you know, you wanna do, you know, factoring of extremely large numbers or things like that, right? They could do them really fast, really efficiently, much better than a human could ever do. And that was, you know, decades and decades ago, like literally the first computers probably could do things that a human couldn't do very easily. That's why we built them.
00:32:47
Speaker
but um and why they why they caught on but you know even if we look at what humans can do and what machines can do you know in some sense i think you know 10 years ago you would have said there was like a famous like you know xkcd the like comic there's a famous example of like there's i think what it was was like uh
00:33:07
Speaker
I can pull it up. But the joke has to do with GPS is really easy for a computer to do. If I want to geotag the location of this photo, it's really easy. But if you want to tell me whether there's a bird in that photo, it's going to take a research team in 10 years or something like that. I'm butchering the joke, I'm sure. And now, actually, we basically solved that problem. We can find a bird in a photo really easily.
00:33:31
Speaker
Um, and like anyone can grab some API off the shelf and it'll do it for them and it's not expensive either.
00:33:37
Speaker
But so that's something that it turns out was really hard for computers for a bit. And then we just figured out a better algorithm. And the algorithms got better. And then it was actually not that hard for computers to do it with the right amount of input and all of those sorts of things. And those are things that maybe fall under, well, I would need that to be able to show AGI, to be able to show intelligence. Well, I need to be able to recognize objects and things like that.
00:34:02
Speaker
Because if I want to do an intelligence test or something like that, these are some of the basic skills you would need to have. On the other hand, the robotics side is still pretty early. And actually, when I was talking to someone the other day about this who's been following how these new AI models are affecting robotics and driving it forward, and it was basically like, actually, they've solved a lot of the basic planning and reasoning. But the actual action stuff with
00:34:30
Speaker
sense and feedback loops with sensory and motor processes is still really hard. But we don't think about that- Even just staying on balance. Even just being able to take steps forward and staying balanced is really surprisingly challenging. It's strange to think that folding clothes would be something that takes a lot of intelligence. I would buy that thing in two seconds if they were one of them. That's good.
00:34:53
Speaker
But yeah, that's really hard still. I don't know if that's going to just fall under the, we don't have the right training data to the right degree yet, plus we don't have the right exact algorithms and we'll just find them and then solve the problem. But it's interesting to me because we could get to AGI and never have that. Even though, again, if we really want to say, does it do the things that humans can do? Well, those are some of the most basic things. My kids learned to do that long before they learned to do all of these other things.
00:35:23
Speaker
yet the machines can't do it. My two-year-old can pick up a cup better than a robot can. Yeah, and that's all that stuff that just seemed like it would be really straightforward and not difficult at all. You don't pay people a lot to do those tasks. You pay people a lot to figure out financial kinds of things, and it turns out that stuff's pretty straightforward. Yeah, and that's a really interesting point. I think the idea of there's the disruption of these models on
00:35:53
Speaker
society and the economy is actually like coming for like certain kind of mid-level knowledge work first. Yeah. Yeah. It's like if you had made a bet 20 years ago, who's going to be out of a job first from AI and robotics, like a mail carrier or a lawyer? You probably would not have said the mail carrier. You probably would have said the mail carrier is going to be out first and the lawyer is going to have a job for a long time, but actually that certainly looks the opposite now.
00:36:20
Speaker
Yeah, it's really interesting. I mean, I would say like my view on like, what are, what, as someone like kind of working with the products that are built on top of these models and what are, what are the capabilities right now that like people are anywhere near actually shipping or are shipping in the world. It feels a lot like, it's like,
AI's Impact on Software Development
00:36:39
Speaker
equivalent to hiring a contractor or an intern. I would say better than an intern actually because it can get there faster than an intern. But it is a little bit like a contractor. I need to spell out for you exactly what I want you to do and the exact criteria and then have you go and do it. You can't expect it to think outside the box and come up with the solution itself quite yet. You need to specify things in a very clear way and it's like I'm handing it off to you and I'm going to
00:37:05
Speaker
check on it when you come back to me with your results next week or whatever. Yeah. I mean, this is where like if I, you know, today, if I had to guess what the future of say software development was going to look like in five to 10 years, it would be that everyone's job is going to be a lot more like a product manager and a lot less like a developer than they were 10 years ago or certainly 15, 20 years ago.
00:37:27
Speaker
And it's like you have to be able to specify what you're trying to achieve very clearly, and it needs to be the right goal. But the actual development of the software is becoming increasingly easier.
00:37:37
Speaker
Yeah. It's interesting to see how even in just in the last year, I feel like the norms in software development have shifted. At least where I work, I hear many more people saying they are using constantly and honestly becoming somewhat dependent on co-pilots for writing software.
00:37:59
Speaker
It's become quite, I think, common at startups for engineers to have adopted these pretty heavily. Not everyone, it's like there's a gradation of people who are and aren't using them more heavily. But to the point that I've seen getting feedback for other tools we have at the company where they're asking these other vendors we work with, hey, I'm used to working in this way, but when I use your tool, you don't have a co-pilot style auto-complete and I'm really very used to having that now.
00:38:27
Speaker
Is there any way you're going to have that in the very near future? It's definitely already shifting things quite a bit. Here's an aside too. What do you think about kids coming up and learning coding? It's in flux now because it used to be like, okay, coding is going to guarantee you a good well-paying job. Now, not so much. It's unclear. There are a lot of people who go through that and don't necessarily find a great job doing it.
00:38:56
Speaker
Maybe there's something useful about still learning the logic of it. What's your take on that?
00:39:02
Speaker
I mean, there's a short, medium, long term. If I were a teenager, I would probably still say there's value in learning that way of thinking. And that, again, while you don't need to produce all of the code yourself anymore, at the end of the day, the company I work at is between 600 and 700 people now, and it's like 200 engineers or so.
00:39:29
Speaker
And with 200 engineers working across the code bases, whether you're having AI do a bunch of it for you or not, you're going to have to think about all of the side effects that it has on other parts of the code base system, et cetera. And it can probably help you do that, but you certainly can't trust it to think through all of those implications for you. You'd always need a human at the top level, at least.
00:39:55
Speaker
at least. And so I think if you're in college, you're going to be a computer science major or something like that, your career is going to look a lot different than you probably thought it was going to. You might have thought initially it was going to, or we would have thought 10 years ago. But I think that's relatively safe in the near term. I do think the question of I have young children and what
00:40:16
Speaker
what actually is valuable for them to learn in that area right now is really hard to say. My basic assumption is being good at math and computational thinking is still a valuable thing. And the people who are building the systems also need those things. So you can always end up actually working on these tools themselves. When I was in college, computer science was still part of engineering.
00:40:46
Speaker
Fundamentally. I don't know if that still is the case. Seems like it's less and less like that. Right. And that's more like there was a period, but I think it's going to have been a fine. We'll look back on the historical and say, this is like a finite period of time where like learning Java was like a career building, you know, something that would really guarantee you to have a good job as in your career.
Evolving Engineering Education for AI
00:41:05
Speaker
I think knowing Java or knowing C or knowing C++ is no longer a primary job qualification. You have to understand what you're building. You have to think about systems.
00:41:21
Speaker
It's back to learning engineering as a broader subject matter. Particularly as you think about building the systems we're talking about in the future, which are going to involve more robotics. They're going to involve more effector systems, sensor systems, and how those interact. So you're going to need to know a lot more
00:41:39
Speaker
I think just thinking systems wise from an engineering perspective is going to be super valuable as well as like the more product management perspective of like understanding on a human level, what is the job that's being done by this technology and what's the customer hiring it to do? For sure. I mean, one of the things that we run into now is, um, you know, for example, on, on the AI team, um, that's part of my group, um, we kind of have two sets of engineers on it. Um, we have kind of more like.
00:42:08
Speaker
More what you would think of as kind of similar to your like machine learning engineers from, you know, that we've had, I've hired that type of role for a number of years now. But though it looks a little bit different in terms of like what they actually end up doing because
00:42:22
Speaker
We're not building foundation models or sometimes fine-tuning existing models, mostly using APIs, things like that. But still a lot of how do you think about measuring success of the model, evaluation frameworks, those sorts of things for the specific tasks you're trying to enable the model to do. But then we have this other role, which is an AI product engineer role. So it's like a more, in some sense, it's like being a regular product engineer. But it's something I think that
00:42:51
Speaker
This is the one where I'm like, I don't know that this is a role long term that is differentiated from the rest of software engineering. I think most people agree with this. I think everybody is kind of ramping up on how to
00:43:05
Speaker
do software development for AI-driven features. And it looks a little bit different. You don't need to be a machine learning expert. You don't need to go deep on the math or anything like that. But you're doing software development of features that involve non-deterministic systems. And that's very different, I think, than what a lot of product engineers are used to. And so getting used to how you prototype and build for that
00:43:37
Speaker
is just different. And I'm sure they also use a lot of these co-pilot tools and things like that along the way, but it's also just a different flavor of engineering. But it's very trainable, I think. For what it's worth, for us, we found actually younger, more junior, honestly, and engineers in a lot of cases are better suited to it or get there faster. Because, I don't know, maybe they're growing up with these technologies coming up as they're in college and
00:44:05
Speaker
And also just like they're not as kind of set in their ways about how software engineering works. It seems like, I mean, in the workforce, people are going to be forced to be more and more human. Yes. Right. Like you're forced to not do all of those rote things because that's somebody else can always do those things.
00:44:24
Speaker
Is that going to be fun or is that just going to be harder and harder? Well, this is again an aside that you can feel free to use or not use, but I think about this in the context of my kids actually a lot.
00:44:39
Speaker
I am who I am, so I still care about, oh, they need to learn a good level of STEM curriculum and all of those sorts of things, because these are things I care about personally. Though again, I don't know which parts of it are going to be most valuable or not. But the other thing is we've leaned in educational decisions for at least our older one toward, let's make sure they know how to resolve conflict with other humans.
00:45:02
Speaker
They know how to work collaboratively because these are all things that probably we will need to do, whether it's fighting our AI overlords, just kidding, or just to operate in society. We still need humans to make the decisions for the human society. Yeah, exactly. Speak another language.
00:45:22
Speaker
All different ways to get along, right? Different ways for human beings to get along. Cause fundamentally that's, as we think about the problems that we're solving as people with technology and the problems that we're not solving with technology. Uh, I mean, certainly we can do things that will help us grow food more efficiently. Hopefully we can solve things that help us get rid of some of the waste that we've created, you know, with technology, all that kind of stuff. But fundamentally they're not, at least technology is not helping us resolve our conflicts at all.
Human-Centric AI Deployment
00:45:51
Speaker
or get along any better. It may give us tactics, right? You probably get some tactics for a given situation, but knowing how to stack rank the importance of different problems we need to solve, et cetera. You need human values and judgments to make those decisions because that has to be what we care about, not what just a machine rationally thinks of it. That's my take. I'm sure there are people out there who disagree with me. I think the machines can figure it all out.
00:46:18
Speaker
No, I, I a hundred percent agree with you. And I think it all, it, to me, it all is of a piece with the definition of intelligence, which is like fundamentally we're asking these machines to solve human problems at a human scale. Cause that's just what we care about. And ultimately what's going to be most important for us as human being is figuring out what is important and how can we, you know, surface that, how can we center that in all of our decision-making when it comes to technology and, and, and other, you know,
00:46:47
Speaker
all the different ways in which we're engaging in development and commerce in our society. So one of the things I wanted to make sure to get to talk about today because you had sent us an interesting paper on this on using AI as participants in research studies.
AI in Research and Bias Considerations
00:47:10
Speaker
And this is a really interesting idea.
00:47:13
Speaker
Okay, obviously in research, you know, it's expensive and time consuming to use participants. And I think anyone who's ever run a research study would be happy if there was a perfect AI that they could just do everything on. Be quick, almost instantaneous. You come up with neat results. You could use a million subjects if you wanted to.
00:47:31
Speaker
But obviously, I don't know if we're quite there yet, but there's some positivity about the idea about this happening. There have been some studies that have already used AI agents and replicated some kinds of psychology findings. I found this fascinating that there was a study that
00:47:55
Speaker
in a sense replicated the Milgram experiments. So these experiments from the 50s of obedience to authority where after a certain point people will continue to press the button and give you electric shocks for a long period, but after a certain point they won't. And they found that
00:48:17
Speaker
that AI would, I guess under prompts, still behave in a similar sort of way. So I would love to get your take on where we are. What does it mean when we're probing in a large language model on something like this? And what can we get out of it? Yeah, it's interesting. So going back to something Joe said earlier,
00:48:41
Speaker
Wouldn't it be great if we just had like a brain and silica that we could like, well, if I know it's a one-to-one representation.
00:48:49
Speaker
of the brain that I can just provide at the same inputs and get the outputs and trust that they're like the right outputs. Of course, that has to be a specific brain. It can't be. Every brain is a little bit different. But I got a distribution of replications of people's brains and I could do this. That would be ideal because I would just know that, well, I know I've done this one-to-one mappings and it works just like their brain does. So I can just use it in place of them.
00:49:12
Speaker
and I can spin it up on a server somewhere. But in the absence of that, we kind of have to know, we can test that a
00:49:29
Speaker
A large language model, for example, shows some similar features in terms of the way it, no, I'll be careful about saying it reasons, but it does reasoning tasks. It responds to reasoning tasks that is similar to the way that humans, that we have data from experiments that humans respond, and some of the biases we have, et cetera.
00:49:53
Speaker
It would make sense that they have some of those biases because they're trained on the output of humans. I have two different thoughts on this. I think it makes sense that there are biases that are similar to human biases given the training. So it feels like there are tasks in which we should be able to use it.
00:50:16
Speaker
we should be able to use these models. On the one hand, studies to show that these models are showing similar biases and tasks like this paper we were looking at around context effects and reasoning, which you can go into more in a second. That tells me that there are some similarities in some way. I don't know that it's similarity in the mechanisms, but similarity at least in terms of the inputs and outputs.
00:50:45
Speaker
Would I be willing to extrapolate a novel finding without ever testing it on a human? I don't know why I would be willing to do that. I think it would be very cool for generating quick, specific predictions about human reasoning. So like given the historical
00:51:09
Speaker
Given what we know, for example, about human reasoning, I might be able to test some novel ideas, a bunch of them, on an LLM, pick the ones that show some clear trend, and then go and test those on a human afterwards. So I might use it as a tool for
00:51:33
Speaker
quickly deciding hypotheses to pursue. But I would never, like, are we going to be willing to publish papers in psychology journals that are like, well, the model showed this, so humans show this. It's like, no, I can't extrapolate that. That's right. Because what you're going for, what you care about is how humans perform, not necessarily how the model performs. I mean, if what you're doing is really just trying to figure out what the mind of an AI is like,
00:51:58
Speaker
I mean, I think that we're like taking it one step forward. If you had multiple brains and you had a whole bunch of them, and then so these are autonomous agents now, so these are like, you know, like modeling a society. Now you could do some really interesting experiments that are just impossible to do ahead of time that might give you some
00:52:20
Speaker
help in making decisions, for example, with policy. So we were talking about conflict resolution. Imagine you modeled two warring factions and the behaviors of those individuals. I guess it's sort of like an extension of, let's say, the AI game theoretic approach. So how would they respond to this type of
00:52:43
Speaker
intervention towards peacemaking or a different approach. How would that play out? Might be some interesting ways to generate hypotheses there that could be difficult to test. It's always like a one shot in the world of how you deal with these policy issues. Yeah, I like that idea of using it for testing hypotheses or sort of as preliminary analysis, but not necessarily something you're making conclusions on.
00:53:13
Speaker
Yeah, I mean, in the paper that we were reading on using LMs in place of human participants, did they actually, I'm trying to remember if they directly advocated for the idea of like just literally running studies where you never have a human. I think they waffle, I think there was some appropriate waffling and sort of toning it down. It wasn't like, these are now humans that you can just report these results in your, you can do 30 experiments a day and publish them, right?
00:53:41
Speaker
Yeah. I wonder, maybe there's an intermediate version here where I'm thinking back to my time when I was actually writing papers. I don't really write papers anymore. And you sometimes would see this thing of you would run a behavioral experiment, you had some hypothesis, you would run some set of studies to try to fence in what the potential interpretation
00:54:08
Speaker
correct interpretation was, and then at the end you might have like, and we like built some model and it made these predictions and in a future paper, in a future study, you know, we're going to test these or something like that. Would we be comfortable with the idea of like you run an experiment, and then you, on humans, you get some outputs, you replicate those outputs in an AI model, make some novel predictions and then like come back later. So you have this like final section of the paper that looks at some of those predictions. Yeah, that would be, yeah, yeah, yeah.
00:54:36
Speaker
So I have a bonus question for you guys that came up this morning as I was on a walk with a friend of mine. We're talking about AI and intelligence and the definition of intelligence center. He had a question that was a little different take on some of this than I've heard before, which is,
Self-Awareness and AGI: A Debate
00:54:58
Speaker
Would for you to consider something to be like an AGI, like a real true artificial general intelligence, would it need to have some kind of self-awareness or self-consciousness? Certainly it feels like you would need to at least have it have what we're talking about is like some sort of worldview or representation of the world.
00:55:23
Speaker
And I think also probably some sort of intention about what it was trying to accomplish. Now, maybe you could say that that intention could be programmed in, um, you know, with sort of the rules of robotics or whatever of robot, you know, but like it, I wasn't sure what the answer was to that. Like, would it, would it be necessarily, there'd have to be some kind of sense that this thing, uh, this AI knew what it was doing and why.
00:55:53
Speaker
I think this goes back to what Daniel was saying earlier, talking about sort of kind of modules or capabilities that an artificial intelligence might have and, you know, stuff like attention and memory. And I think one, one thing that you could talk about there is just a sense of self. I mean, we have, you know, we have a specific system in our brains that let us know what our cell, what, what's a part of ourselves and what isn't. And
00:56:22
Speaker
You know, I guess we've talked about that before in with respect to psychedelics, where you can you can experience no self. So, you know, both experiences are possible, but our brain causes us to experience a self and and, you know, experience the contrast between ourself and other things. And that may be in. Is that essential or absolutely necessary in in artificial and
00:56:48
Speaker
in a generalized artificial intelligence, I could see it as being part of the package. Well, this is, it kind of gets to the core of the question we started with, which is like, what is the relationship between neuroscience and AI, right? Like, this is an area where you might say, look at a human being that's evolved over all these millions of years. And it seems to be that this sense of self is like a core, core element of what it's like to be a human being. And it might be important for some reason, like,
00:57:17
Speaker
You know, like nature has expended a tremendous amount of energy or whatever it is, there's a tremendous amount of energy associated with the sense of self. Maybe it's important.
00:57:28
Speaker
Yeah. Yeah. Without getting too deep into the philosophy, because there are like, you have to have different people have different philosophical stances in general on what do we mean when we say self-awareness? Yeah, certainly. We talked about that on the show quite a bit. And Ralph and I have different views on that. Oh, interesting. OK. OK. I'll try not to out mind right now. But I don't want to know that I disagree with either of you. But I think some version of self-awareness
00:57:59
Speaker
and intention, I think it's useful to the extent that it's necessary. It's necessary for them all to work. I think it's a separate question to me, just in the definitions I would use, of AGI only requires that if that was required to achieve behaviors that we would say correspond to AGI. I'm adding myself as a slight behaviorist right now, which I kind of am, and all has happened. But I think
00:58:26
Speaker
to the extent that you need a representation of self or intention in order to behave in ways that we would think of as being intelligent, then maybe that is necessary. I think there's very minimal versions of that that people do with LLMs today. The ones that you get to use directly in consumer products tend to have been prompted with a bunch of things about what they should do, how they should act,
00:58:58
Speaker
I'm a helpful agent who's trying to help you do XYZ, so this should inform the way that you act, essentially. And they're also obviously trained over time to give different types of outputs and be steered away from certain types of outputs that we think of as harmful, et cetera. So there's both explicit and more implicit ways of training that.
00:59:21
Speaker
I don't think that corresponds to world knowledge or an actual sense of self-awareness necessarily, but we do find those are useful for getting certain types of outputs we think of as being human-like. To the extent it's required, I think it's important. I wouldn't be one to say,
00:59:38
Speaker
Uh, that some, we need to know the model has some quote unquote true self-awareness to qualify as AGI because first off, I don't know how the hell you would ever measure that. And if it's not measurable, then what are you supposed to do? Uh, what I like about it though, I agree. And that's why, but that's why I liked the particular phrasing or like, so it was like, uh, just like more of like a, uh, where, where he was coming from. Right. Like it's like, actually.
01:00:08
Speaker
It tells you something about what you think intelligence is, of how you answer that question, right? It's irrespective of what you think, like what you think about awareness is. Yeah. Which is a harder question, right? So it kind of gives you a little bit of a lever into thinking about like, you know, what do we mean by artificial general intelligence?
01:00:32
Speaker
Would it be like, if it was just a machine that just only did exactly what we asked it to do all the time, would that count? Because actually it seems like intelligence to us in some way implies, you know, working in the environment in such a way that benefits the agent. It is like there's some benefit to the self that if someone acts in ways that just like,
01:01:02
Speaker
are seems like always counterproductive to that person's interest. You don't consider that person to be especially intelligent, for example.
01:01:09
Speaker
again, I'll put my behavior's hat back on. You need to give me a way that I would like actually empirically measure that. First, at a minimum for me to be able to say that it is required. So I need to know like, we need to come up with, and I'm sure a psychologist is going to do this at some point, like, how do I measures of intention and in large language models or in AI systems?
01:01:34
Speaker
I'm sure probably someone has already worked on this. You need that. Then I think to your point, you need to be able to say that, have some way of showing that this is required to perform in certain ways that we consider to be intelligent for the model to perform in certain ways that we think is intelligent. That seems plausible. The flip side version of this is
01:02:04
Speaker
I also think humans overestimate what percentage of the actions we take are operating at that highest level of self-awareness. Literally, maybe I'm outing myself and how I act in the world. I always worry that I'm doing that. It's like everyone else is actually really thinking through every action they take very carefully explicitly and through language and internal monologue the whole time. But I think
01:02:34
Speaker
There's, I'm thinking as I'm speaking right now, but also like my speech is slightly ahead of in my thinking in some ways, unless I'm being like very careful in giving a speech or something like that. And so I do think
01:02:50
Speaker
I do think we overestimate that about ourselves in some ways as well. And a lot of what's surprising about the performance of some of these models and makes people like Noam Chomsky say in the New York Times that they can't be doing what humans do, I think rests on a bunch of non-empirical assumptions about how humans work and how most of how most human action happens. And I do think there's
01:03:15
Speaker
people have very different opinions. You mentioned the two of you have different opinions about some of this. People have very different intuitions about how we operate in the world, and especially around things like degree of self-awareness, et cetera. I really felt this when I was in grad school too, that my advisor has my one view of the world and
01:03:37
Speaker
There was entirely opposing views. And I was like, these people are convinced they are operating in the world in very different ways. And that's also quite interesting to me. But I think a lot of that is not empirical. I think a lot of it is very hard to wrap your head around, get your grasp on just the empirical data. And I don't know.
Legal and Societal Implications of AGI
01:04:00
Speaker
I don't know if I want to try to... If we have to go there with building AI models, I think it's going to make our lives a lot harder.
01:04:07
Speaker
Well, it'll be good, I mean, this is the thing where, you know, in this lawsuit between Sam Altman, well, it's between Elon Musk and OpenAI, but, you know, where Elon is saying, look, you guys already have artificial general intelligence. Yeah. Why is he saying that? Because, well, actually, you made the decision that when you reach artificial general intelligence, you have to make sure everyone has access to it.
01:04:37
Speaker
So you're not gonna be able to make as much money. So who has the incentive to say artificial general intelligence is this really big thing versus who has, who's motivated to say that this artificial intelligence, artificial general intelligence is actually a subset of the things that we think of.
01:04:57
Speaker
And now we have a judge who's going to have to make some decision on whether we've had AGI or not. So there are stakes. There are actual stakes. Yeah. And there's going to increasingly, I mean, you can imagine all the different stakes that become, of course, then you have the famous things from all the novels and literature associated with robots and androids and whether they have rights and so on and so forth.
01:05:26
Speaker
But yeah, there's a lot of there are a lot of real practical policy implications of the stuff that we're talking about. Yeah, I just think I just think like if for intelligence, I would be concerned if we got to a place where like from a legal standpoint, et cetera.
01:05:43
Speaker
I could see for rights to your point, this actually feels much more important, although again, it's going to be such a difficult thing to make an empirical decision about. But I think for those, I can see really strongly why we intuitively would care a lot about self-awareness and intention when thinking about things like the rights of AI systems or robots or androids or things like that in the future.
01:06:12
Speaker
for like, is it intelligence?
01:06:17
Speaker
It feels self-serving to whoever is on the side of the debate that would benefit from this to be able to say, well, it's not because it does these things differently than a person does, even though it does. Imagine we have a model that is able to replicate humans in a huge variety of ways that I would happily hire to be my intern and that replaced hundreds of thousands of millions of jobs, et cetera.
01:06:45
Speaker
And you're like, well, it doesn't count as AGI because it, you know, it's not, we don't have evidence that it's self-aware. Um, and I'd be like, well, but like it's having all of the implications on society that I would have expected, um, from hitting AGI. So like, why does it matter how it works? It just matters that it works. Yeah, absolutely. No, I think I, I, I'm very sympathetic to your sort of operationalist, uh, sort of approach to this, you know, like.
01:07:11
Speaker
And that's completely constant with my take on what intelligence is, which is just like, it's a vibe. It's a vibe. It's a vibe. It's like what you think it is, literally. Yeah, no, for sure. And that's like the people who are on the skeptical side saying we are being fooled by these models based on their fancy outputs and that they're able to do these sorts of things.
01:07:33
Speaker
And I think to your point, well, if it's a vibe, then they're giving the vibe really well. And yes, people are like, you can probe them in different ways and show some of their limitations. And people are easy to fool to some extent with these sorts of things. So they endow more intention and all of those sorts of things behind the actions than maybe is reasonable. But you could make the same argument that if I didn't believe I had intention and self-awareness, I know I do.
01:08:02
Speaker
So I look at you, Joe, and you look similar enough to me that I assume you do too. And that's kind of all I have to go on, right? Like, I don't have any other evidence that you have those sorts of things. So on some level, I get to the like, well, why is it any different with one of these models?
01:08:18
Speaker
If it acts in a way that is similar to having intention, how am I supposed to know? Well, this goes back to some of the episodes that we've had on the show about defining consciousness. This is where integrated information theory gets into all this trouble because it claims that it's representing consciousness when there's no way to know for sure.
01:08:41
Speaker
or neuronal global workspace theory. So, I mean, if we use those as a description of how, or neuronal global workspace theory, which we've talked about, if we use that as a sort of description of how intelligence works in humans that
01:08:57
Speaker
Most things are unconscious, but some things are conscious and broadcast to the whole system and sort of generally flexibly available. If we think of that as a way to talk about how awareness or consciousness works in humans, then we can say, well, no, chat GP doesn't have awareness in that sense because it doesn't work that way.
01:09:17
Speaker
Yeah, exactly. We're still left with some question about are these accurate models? And can this really tell us about the stuff we care about? But we do know, I mean, the architecture is different enough so that it doesn't operate in that same way.
Embodied Cognition and AI
01:09:32
Speaker
Right. I mean, I think another example of that of how similar and different AI systems are to humans and how it makes it feel more and more human as you add some of these capabilities
01:09:43
Speaker
One is actually inspired by that book right behind you, Rolf, The Extended Mind by Annie Murphy Paul. We had her on the show last year or the year before. Super influential in my thinking since that time that we've had that conversation. She talks about embodied cognition, the idea that your cognitive status is not just in your head. It's actually in the environment. Your thinking is happening in concert with your interaction with the environment.
01:10:14
Speaker
uh, through your senses and through your actions and that the interplay of the environment and how it responds to your, uh, actions, you know, whether that environment is just the, you know, uh, your
01:10:27
Speaker
room that you're in inside or if it's out in the woods or it's around other people. Importantly, the idea of cognition extends beyond the body. And certainly it extends beyond the brain, that's for sure. It's easy to see how cognition extends into the periphery of the nervous system. And that's where as AIs start taking on more capabilities in terms of sensors and effectors, it's gonna feel more and more like
01:10:56
Speaker
they're human in that way, as their ability to extend their cognition into the world and have effect in the world. Yeah, I mean, a very simple example of that that comes up. So this is less in the context of like, this is still embodied, I think, and it certainly fits with the ideas and extended mind. But if I'm writing something,
01:11:21
Speaker
Um, there's an interplay between, uh, like if I'm writing a paper, there's an interplay between, uh, what I'm thinking in my head as I'm writing, I'm going back and looking at what I wrote. Um, which like, I probably could not remember exactly what I wrote, but I can see it on the page and I can, uh, I can revise that sentence and I'll probably, you know, I might do a better job revising the sentence if I have the exact sentence from three paragraphs earlier in front of me and I can like look at it.
01:11:46
Speaker
And I actually think we're starting to do things in terms of features with models like this that start to look somewhat similar to that.
01:11:58
Speaker
I think one of the ways I'm excited about using these models in tools like the ones that I work on are places where I can get a model to restructure an output for me really well. So I can pull from a few different documents and I can generate a table that's built off of that that's pulling in all of that information. And again, it's not like the model has memorized all that information. It has a way of retrieving it, and then it's
01:12:27
Speaker
It's like it's opened the page and it can see it now. And then it's going to go and take that and I'm going to ask it to restructure it in some way. And it can kind of take all of those actions. But in order to do that, it, like humans, is getting access to stuff that's in its version of the world, which is some set of documents.
01:12:48
Speaker
maybe the internet and maybe some other things that can look up. And so there's a similarity there that I think is really important. And I think to your point, Joe, one might argue that things that we've seen in human intelligence like the Flynn effect, et cetera, some of that has come from like we have created technologies that make it easier for people to be smarter in various ways over time.
Intelligence and Technological Advancements
01:13:14
Speaker
And like a lot of what is like increasing human intelligence has come out of
01:13:19
Speaker
has come out of things like, I would assume the industrial revolution, I don't know if anyone's looked at this, probably someone has some historical, some historian of these things. Like did the human intelligence increase after the industrial revolution or as part of the industrial revolution? The printing press changes what people can do in the world and the access to these sorts of tools over time, the ubiquitous access to computers while maybe it's made our society worse in some ways and smartphones and things like that.
01:13:48
Speaker
There must be ways, unless it's enabled human intelligence to increase over time or to democratize access to certain ways of working in the world over time. But similarly, I think we're giving these models more access to more and more tools and capabilities that extend beyond the model itself. Absolutely.
01:14:13
Speaker
Yeah. And just, just a callback there. The Flint effect is referring to the observation that scores on intelligence testing since they started in the early 1900s increased steadily across the developed world. Um, until very recently. Yeah, that's not my understanding is it's no longer, uh, it's stabilized. If anything may be coming down a little bit. So.
01:14:35
Speaker
at least by the measures of our own psychologist-derived intelligence tests, people were getting smarter for quite a while. Probably because of the things that they're able to learn, although there's other alternative explanations, for example, improved nutrition, et cetera. There are definitely other explanations. I can imagine it going two ways, too. Having access to Google makes you smarter in a way, but in another way, when you're without Google, you're a little less smart.
01:15:05
Speaker
Well, I mean, my brain has been completely broken by the internet. So all the external, like all the external stuff you use, like it kind of makes you smarter while you're using it. But if you're tested without it. Yeah. This is again, he goes back to this like whole question around definitions of intelligence because an intelligence test has been like refined over a long period of time to measure the specific set of abilities that correlate with each other in certain ways, et cetera. But like what actually matters for like.
01:15:33
Speaker
succeeding in the world looks very different over time. Though I don't know that there's been any findings that the performance on intelligence tests, I'm going to be careful to not go too deep into my beliefs about IQ and intelligence because I could also get quite a- It's another episode. It's another episode. Yeah, it's another episode. Yeah, you don't want me to be on that one.
01:15:58
Speaker
I have no idea if we're going to see those correlations of those tests shift or whether they change the tests to continue to correlate better over time. Because I do think what it takes to succeed is changing over time. And I don't know if that's in ways that are in concert with the way intelligence has historically been measured or not.
01:16:20
Speaker
Well, I think that, you know, uh, it's a good place to sort of start wrapping it up here. We have a last question and I like to ask people, which is, you know, what are you, what is coming up that you're really excited about? Like, maybe we can keep it in the context of this conversation of AI. Like, what do you, what do you see coming down the pike with AI that you're like, you just think is exciting that might be happening soon. Yeah.
Future Directions and Innovations in AI
01:16:41
Speaker
Um, I think.
01:16:44
Speaker
So a couple of things I think are going on. So I'm really curious. This could be an exciting slash disappointed one. I got to be careful what I say. But I'm I am really curious to see how much more we can get out of these these types of models that are basically the state of the art right now. I do think there is a take that has some validity right now, though I could be very wrong about this. So get it on tape.
01:17:14
Speaker
that that we are actually hitting some limits in terms of just are closing in on some limits in terms of just like how much more capable the raw like
01:17:29
Speaker
transformer models are at solving these types of problems. I don't think we're at a complete asymptote or something like that, but I don't know that we're seeing in recent models that have come out, I'll be curious to see if OpenAI releases one this year that is far more capable, that we're getting the same degree of gains we were getting generation to generation over the last few years.
01:17:55
Speaker
I do think there's tons more, it's really early still in terms of how do we use these models combined with other capabilities to solve more complex problems. So I do think the current models have a lot more legs to solve more types of problems and we're in a really early period there.
01:18:15
Speaker
But just in terms of the raw power of the models, I'm less convinced now that that's just some magical path to superhuman intelligence on its own just through improvements to those. We'll see. Could be very wrong. But what I'm excited about, I think, is that it is really, really, really early. Last year, I think everybody was just launching products that were like,
01:18:45
Speaker
Well, we can have it write stuff for you. We can have it write code for you. We can do auto-complete type use cases with a little bit more to them than I was mentioning earlier. RAG has come since then, has been a big thing people were working on. And so now you have this ability to constrain more, use this knowledge set, retrieve knowledge from here. I think taking more complex actions and more recursive iteration to
01:19:12
Speaker
automate way more things for you. I think we're in a really early stage there. And I wouldn't be surprised if we start to see a lot more of those types of things this year. That seems to be what a lot of people are working on in addition to leveraging retrieval and things like that. But I'm just really interested, I think I'm really excited about as we go on that journey with technology. Again, I work in software, so I'm thinking about
01:19:37
Speaker
how software are gonna change over time is I think the main user interfaces and ways that we interact with software is going to change a lot over the next five years. And I think we're in a really early part of it and like on both sides, both the people building the products are still figuring out like what clicks and like what are the really like compelling use cases of which I'm very excited about.
01:20:02
Speaker
And then humans who are using software have to adapt to interacting with software in different ways. And so I'm just really excited in this moment to be part of that journey that is like, it's going to look a lot different. It's not just going to be a chat bot for everything. I think we all know that's not the right solution to every problem. I like being able to click on things sometimes, et cetera.
01:20:29
Speaker
But on the other hand, there are really cool things that we're only just starting to do. I can leverage text as a way of... I can generate text based on relevant information to very quickly catch you up after you get back from vacation on
01:20:48
Speaker
what were the things I cared about before I went on vacation. Or there's just that part of it, just how people work with technology changing. That's what I'm really excited about. And I think this year, a lot of it probably is going to be people starting to figure out better ways to have either tasks get automated for you that are tedious
01:21:09
Speaker
or better ways of you being able to interact with the technology a bit less and get what you need much faster. I'm excited for that this year. Excellent. Well, Daniel, thank you once again for being on the show. I really appreciate it. Cool. Yeah. It's been great to come back. I hope to come back for a third time someday.