Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
#6 - Pieter Medendorp - Sensory Integration and Transformation in the brain
17 Plays5 years ago
Our guest today is Prof. Pieter Medendorp. He is the head of the sensorimotor (http://sensorimotorlab.com/) lab at the Donders Institute for Brain, Cognition and Behaviour.
In this conversation we talk about the science of sensory integration and transformation. Specifically, how these very fundamental processes impact us on a daily basis without us even realizing it. We discuss how our senses can fool us and how that experience can be fun. We also make links to industry and how forms of therapy and diagnosis can improve using these principles.
Recommended
Transcript
Introduction to Sensory Integration
00:00:00
Speaker
Hey, what's up everyone. This is Ajmal Savary and welcome to another podcast episode. In this episode, we talk to Peter Medendorp about how our brain is capable of integrating sensory information and how it is transforming it across modalities. Enjoy.
00:00:30
Speaker
Hey, everyone, and welcome to another podcast episode. If you're new here, my name's Ajmal. I'm a neuroscientist and entrepreneur. On this podcast, we explore the links between science, technology, business, and the impact they have on all of us.
00:00:45
Speaker
Our guest today is Professor Peter Medendorp. Peter is the head of the Sensory Motor Lab at the Donders Institute for Brain Cognition and Behavior, where he is also the chair of Sensory Motor Neuroscience and the director of the Donders Center for Cognition. Peter's research interests focus on the relationship between brain and behavior, and in particular, the neurocomputational coupling between perception and action.
00:01:09
Speaker
For his work, he has received the Career Development Award from the Human Frontier Science Program and the Radbout Science Award. He was also awarded the Dutch VD and VG grants as well as the European ERC Consolidator Grant. He is currently a board member of the Neurocontrol of Movement Society, member of the Dana Alliance for Brain Initiatives and member of the editorial board of the Journal of Neurophysiology and serves as a review editor of the Journal of Neuroscience.
Sensory Systems and Daily Life
00:01:39
Speaker
In this conversation, we talk about the science of sensory integration and transformation, specifically how these very fundamental processes impact us on a daily basis without us even realizing it. We discuss how our senses can fool us and how that experience can be fun. We make links to industry and how forms of therapy and diagnosis can improve using these principles. Enough background. Let's get into it, shall we?
00:02:06
Speaker
Hi, Peter. It's good to see you again. Yes. It's been some time. It's, uh, five months since we last saw each other. Yeah. Something like that. Something like that. And even back then I told you for quite some time that I wanted to ask you if you wanted to do a podcast session with me. Yeah. I think, I think you asked me already last summer. Yeah. So it's almost, um, but you're a busy man. Sorry, aren't you?
00:02:35
Speaker
Nothing wrong with that. It's good to be busy. I mean, in a good way. Of course. Busy, productive, not just busy for the sake of being busy. That's not good. No. So the podcast today is supposed to be about the science of sensory integration and transformation. To whoever I talked to before, most people don't even know what sensory integration is supposed to be. Could you explain that for us a little bit?
00:03:03
Speaker
Well, they asked me to talk about sensory integration or sensory motor integration, or how specific do you want me to be? Do you want to give me a broad scope? Yeah, let's start with sensory integration first. Sensory integration.
00:03:18
Speaker
Well, the sensory integration. Well, everyone knows, I think, that the human body, let's talk humans here, has multiple sensors,
Understanding Sensory Integration
00:03:30
Speaker
sensory systems. And we have, of course, we have our vision, our eyes, and that detect light from the environment. There is, of course, our ears, our audition. There is a sense of body. How is your body oriented?
00:03:48
Speaker
Where are your hands? Where are your fingers? That's proprioception or somatosensations you feel touch on the body. And the brain uses all these different sensory signals in order to
00:04:03
Speaker
infer the state of the body. So how is your body? What about your body? How do I perceive my body? And how is this body oriented, organized within the world? And so the brain builds up a percept based on all that sensory information. And that process is called sensory integration. So different sensory modalities need to be combined. Integration is a kind of combination.
00:04:31
Speaker
And one of the questions then is how is this combination, this integration of sensory systems, information from different sensory systems, being done? Because I can, let me give you an example. I can see a bird in a tree, but I can also hear it. So how do I build up that percept of that bird there in the tree? Is that because I see it or because I hear it, or do they combine different sensory inflows?
00:05:00
Speaker
And apparently, if you think about the human body, the human brain, the brain picks up multiple sensory cues in order to come up to a percept of that little bird there. And it not only uses the visual system, but it also uses the auditory system, and it uses all kinds of system sensory inputs that it can use in order to achieve such a percept. And I would say that is sensory integration.
00:05:25
Speaker
Because we do only have one percept, but that's how it feels to us. In the end, it feels like the brain builds up one percept. Even though it's different sources coming in. Different sources. Percepts can also be illusory, of course. But apparently, it seems that we have some integrated percept that is seamless, is not interrupted if there is an interruption in our sensory inflows. So now and then, if you feel that. But that process is sensory integration.
00:05:54
Speaker
And another complexity to that process is, of course, that the brain needs to take into account that the information that it achieves through the sensory system is not all equally reliable. Some information might be a bit more reliable than other information.
00:06:13
Speaker
And so then the question is, does the brain take that difference into account? And some people think, and I think that's a kind of a contemporary idea in the neurosciences that the brain does so. It takes multiple, it takes information into account, but it also realizes or takes notion of the fact that some information is a bit more reliable and other information.
00:06:37
Speaker
And that is also captured under that umbrella term of sensory integration. Is it explaining it? Yeah, I think that's explaining it very well. You definitely did a better job than when I tried to do that. Oh yeah? Yeah, yeah, yeah.
Perception and Behavior
00:06:51
Speaker
I tend to stick too much to the lingo used in the field of research, which of course is not very accessible if you hadn't had it.
00:07:03
Speaker
I always like- I'm still using a little bit of lingo here, but still. I think it's pretty understandable. Specifically, the example you gave, when it comes to localization, let's say, just with the auditory system you have there, where you know or at least have a feeling where something should be located. When you close your eyes, you can do this, but of course, your visual system tells you as well where something is located.
00:07:31
Speaker
And if those two come together, you don't have a percept. At least that's not how it feels to me. Oh, my auditory sense says it's over there. Oh, my visual sense says that it's there. I just know it's there. Yeah, you know, just, yeah, well, of course you can do experiments in which you try to put a discrepancy between these two cues and then you can see what does the brain
00:07:53
Speaker
Where does the brain go with? Does it go with the visual? Does it go with the auditory? And does that depend on the reliability of that information? And that's all that falls under this topic of sensory integration.
00:08:02
Speaker
And what type of sense can you mess with those two systems? I mean, it doesn't have to be these two systems, just any two systems that are supposed to give a very similar source of information for a percept. Yeah. And of course, you can mess around with that if you wish. Of course, you can control this experimentally. But also in the real world, the brain needs to take into account that not all the information that arrives in the brain is equally precise.
00:08:32
Speaker
Yeah. And it's like the classical example that you can find in literature, and that I also tell students a lot, is that if you have to do thermometers, and you want to record the temperature, and so you take your one thermometer, and you know it's not a very precise thermometer. So let's say it measures the temperature within five degrees Celsius. So you put it outside your window, and what do we know? We have February, so it tells you it's about six degrees.
00:09:01
Speaker
Then you take the other thermometer and you also know it's a bit better thermometer. It tells you the temperature within 2 degrees. So you also put it outside and it tells you 10 degrees. And actually now you have two measurements. What's the temperature?
00:09:19
Speaker
Well, if you work like the brain would say, well, let's not throw out information because I have two samples from two thermometers. I know that one is a little bit more precise than the other one. And so what the brain would tell you what you should be doing is weigh the two bits of information.
00:09:38
Speaker
but weight information that is a little bit more precise, put a little bit of heavier weight, and trust it a little bit more. But also don't forget about the other one. So if the one says it's six and the other one says it's 10, the one that tells you it's 10 is probably a little bit more, which is a bit more precise, well, rely a little bit more on that one. And so make an estimate, well, maybe it's nine degrees, right? So then you're a kind of, then you do integration, how the brain would do it. Right. So it's kind of,
00:10:05
Speaker
crudely said, four eyes see more than two.
Neuroimaging and Sensory Integration
00:10:08
Speaker
Yeah. But keeping in mind that, well, what if the other two eyes are, I don't know, they're wearing some blurry glasses or something. Yeah. Yeah. But rely then a little bit more on the better eyes. Yeah. But don't throw out information from the blurry. Right. Spectacles. Because any additional information is useful. It's useful. But how does the brain then know
00:10:30
Speaker
how reliable the sensory source is. Is that something we learn over our lifetime?
00:10:36
Speaker
Yeah, the brain might learn it. Basically, that's a very deep question already in the neurosciences. How does the brain know what the precision is of a signal, right? And we call that precision. We often make a distinction between, let's say, a bias in the system. So it's a kind of a systematic error relative to what we think is the truth. And we don't even know the truth, right? But suppose we assume that there's a bias and we called it
00:11:04
Speaker
In our lingo, we call that the accuracy. But let's say the reliability, we would refer to that as the precision. And the question is, how does the brain know about its precision? Apparently, the brain is able to learn, get a feel for that notion. Because if you make, if you experimentally make input to a particular sensory modality a little bit more precise,
00:11:31
Speaker
And the brain will start relying more on that information. See what you see in patient groups, right? If particular patients lack a particular sensory system because of a disease or what have you, then what you will be seeing is that the brain starts to re-weight, rely more on the information that remains, which makes sense, of course. Makes sense, yes. But at least we can track that process.
00:11:58
Speaker
Yeah. So you're saying it's possible to artificially, on a short term basis, make one perceptual, well, not one perceptual, one sensory source a bit less accurate, a bit less precise. And by doing that, you can then measure
00:12:17
Speaker
Just by knowing how accurate the other source is, you can then basically predict, okay, if things are weighed the same way, behavior should go that way. But if it's reweight, it should go in that direction. Okay. Wow, that's amazing. And that's what people have shown. You can see it in the behavior, you can even see it at the neuronal level, right? So in the brain.
00:12:42
Speaker
Yeah, but you make you make use of not only behavioral measures, right? You make use of neuroimaging techniques as well, like the fMRI, eg as well. Yeah. And you use also brain stimulation techniques, the TMS to disrupt processing. How
00:13:04
Speaker
I'm always wondering, the fMRI environment is a very constrained one. You pushed into this tube. It's very loud, so I guess. It's not always very pleasant there, no. But I mean, studying, for example, the sensory system of audition can be challenging, I guess, in such a loud environment. But you said you can also see it neuronally. How would those experiments look like? And what have you learned from those? What have we learned from those?
00:13:35
Speaker
Well, of course, most of these experiments, they are computationally guided. That means we try to guide them by setting up a kind of prediction based on some modeling exercise.
00:13:49
Speaker
And the model that we keep in mind is that the brain works like I just explained how this process of sensory integration works more, let's say, any optimal sense. The brain would be an optimal system, which is not a perfect system. It should behave like this. Maybe we can get back to that later.
00:14:06
Speaker
But then of course you can do experiments in a scanner, fMRI scanner, which is indeed very constrained. And I was just saying it's not very pleasant, but what I see if I put people, subjects, participants in an fMRI scanner, let's say later today, let's say at four o'clock in the afternoon, most of them fall asleep because of the very
00:14:29
Speaker
I'm a victim of that. Very sustained noise that comes up, all these beats that keep on drilling away.
Decision-Making and Sensory Precision
00:14:37
Speaker
It's really, it puts you asleep. That's right. So in that sense, it's not that unpleasant. They had to wake me up. You can do very... People nap there and you see it on their eyes. Of course, we track the eyes and we see if people are still...
00:14:53
Speaker
Still awake, not asleep. If they start falling asleep, you already see the eyelids, they go down and you try to wake them up. Come on, wake up. Anyway, so that's on the fMRI scanner. But what you could do, of course, doing auditory stimulation in fMRI scanner is not the easiest. And what you could do is then you can present the auditory stimuli within the
00:15:17
Speaker
Let's say within the noiseless spirits of the fMRI scanner, but it's still tempting. And of course you can use headphones and all of that. But most of these experiments are done using the visual system. Just looking at visual pictures and now what you can do is you can make these pictures a bit more noisy. You can blur them basically in your words. And then you can see how the brain responds to blurry images.
00:15:44
Speaker
with some, and that's what I call computational guidance, then you can make predictions about what would you find if the brain would be an optimal observer? If it has to interpret two images, what else? For example, would we find, let's say this precision that I was just talking about, would we see this encoded in neural activity? And some people claim that they can show this.
00:16:06
Speaker
But that's really ongoing work now. Many people in the field are working on it. And then the question is, of course, how does the brain do that? The brain is a bunch of neurons. How do these neurons encode?
00:16:19
Speaker
that information, and where is the precision encoded, if you can speak so, if you can say so. These are all questions that are currently being addressed. Because there's another dimension that can be played with when it comes to the precision, which is also of time, of course. It's not only is the image blurry, but how long were you allowed to see it? Of course. That's the way to manipulate, let's say, the noisiness or the precision of an image. If you feel it for 10 milliseconds,
00:16:48
Speaker
Of course, you are a bit less precise in what you think about the image than when you can view it for, let's say, two seconds, which is infinite time for the visual system. Yeah, that's right. And the experiment, to be fully honest, that I fell asleep in was- It's not ours, right? No, no, no.
00:17:12
Speaker
Well, I fell asleep in one experiment, but that was not in the scanner. It was a long day, let's say. But the one I fell asleep in was in the scanner and it was an experiment where it was a decision making experiment.
00:17:28
Speaker
where you saw a pattern of dots which were moving either left or right, and you had to basically decide they're moving right or they're moving left. But because you have all these dots, the level of noise could be very well manipulated, which was, to me, so difficult. Yeah, maybe it was just me. But they said my performance was very good.
00:17:57
Speaker
But to me, that's not how it felt at all.
00:18:00
Speaker
Yeah, well, this is a classical experiment. It's a very popular experiment in the neuroscience in which you look at a random dot pattern, as they call it. And what you can do is you can ask yourself, do the dots move to the left or to the right on average? And of course, if 50% of the dots move to the left and the other 50% move to the right, or they all move in random directions, it's your chance. And you'll find then, if you present those images multiple times, you will be a chance. Sometimes you say left, sometimes you say right. But on average, you will be.
00:18:29
Speaker
50% you make a correct response. But then you can manipulate, let's say a certain number of dots moves coherently in a particular direction, left. And then there is a level of coherence in which you always say, well, I now see it moving to the left. And so you can determine that decision curve as you refer to that.
00:18:47
Speaker
And of course, then you can look at where do we find this accumulation of evidence, if I look at that pattern, this process of evidence accumulation of coming to a decision. Do I say left? Do I say right? Where does it take place in the brain? And of course, there's ideas about, there's all kinds of structures in the brain that process
00:19:12
Speaker
have thoughts that move in a particular direction, they process visual motion as we call it, stuff that moves by. And you will see in those areas that there is that process of evidence accumulation that helps you to come to a decision, this stuff moves left, in this pattern it's an average moving to the left and in this pattern it moves to the right.
00:19:32
Speaker
And your perception, your uncertainty about your response is sometimes even not matching your behavior. Yeah. I think that's, so the first time I remember seeing that in a textbook, I was amazed that you could see, you could literally see in the neuronal activity, the accumulation in the specific visual areas of just the random dot motion.
00:20:01
Speaker
I was always fascinated by using sensory integration or this type of research for looking into decision making processes themselves. So not trying to take away from, let's just call it regular decision making where it's more about you actually have a choice between two things and you have to pick one of the two.
00:20:28
Speaker
To me it feels you just have so much more control and so many more knobs to turn that you can just, because I'm very convinced the principle mechanism should be the same.
00:20:40
Speaker
But that's just me. I have lots of debates with experts on this. I think that is actually mine. But of course, you have worked in this lab, so you also know a bit my thesis here. Of course. Yeah, that I think maybe, I believe is maybe not the right wording.
00:21:01
Speaker
I think that by studying, let's say, somewhat lower-level sensory motor systems, as I would call them, it's not only the sensory system, but also the connection to the motor system, so the sensory motor systems, they can really help you to understand particular principles that might be generalizable to other systems, and that's a little bit
Cognitive Principles in Neuroscience
00:21:30
Speaker
I've called this before and I think by studying sensory motor system, you basically open up a window into cognition. By looking at sensory motor systems, you can study how the brain analyzes information, but that information can be well controlled by an experimenter. So you get a handle on how does the brain deal with information or with uncertain information.
00:21:55
Speaker
How does the brain draw conclusions out of sensory information from information or multiple streams of information? And that's of course, we call this the inference problem. There's nothing else than drawing conclusions. So how do we draw conclusions? We draw conclusions every single moment in our lives based on
00:22:15
Speaker
multiple inputs, but came in now really pinpoint particular aspects of that process by studying the sensory motor system. That's what I believe in. Therefore, I'm studying the sensory motor system. Or how do we learn? Of course, there's a big gap between learning in the classroom and learning to control your body or to improve your forehead and in tennis or bring it even back to the lab.
00:22:44
Speaker
But I still believe that particular principles that we discovered there might be generalizable to more real world environments. And therefore I'm trying to study these systems. Right. No, I think so too. I mean, obviously I think so too. I worked here as well, but I just.
00:23:03
Speaker
The more I learned about the very fundamental nature of this type of processing, just because it's so basic doesn't have to mean it's simple at all. And the mechanisms that I've seen here and also the models we've worked on, I see now being used in other fields of research, they just call them differently. But the principles are the same.
00:23:26
Speaker
which I just like to think about it like this way. If you have one principle that you can apply at multiple different things, why using multiple different things? Exactly. And so far, the brain has been pretty efficient. Yeah, but I think that's also human.
00:23:42
Speaker
the human nature, we try to use particular semantics in order to put meaning to particular things. We talk about language, we talk about perception, we talk about motor attention, consciousness. But I think what we would like to find out are these more cognition general principles. And my point here is that
00:24:08
Speaker
But of course, you can have different opinions on
Modeling Brain Functions
00:24:12
Speaker
this one. But at least by studying, let's say, sensory motor systems, we can at least try to discover some of these principles. And then we can see whether they can be generalized to other cognitive domains. That would be, I think, very interesting, right? If we have understood the more general principles.
00:24:31
Speaker
and we have an insight in those, then I think we really push the science forward. Yeah, I like to think of it as layers of abstraction. If you look at, as you said, language and you try to explain language by just explaining how the muscles of the mouth work, you miss a lot. Sure.
00:24:54
Speaker
being able to move the muscles of the mouth is a fundamental part of being able to produce language. Sure. And that's, that's how I like to think about it. You have these very fundamental principles and then whichever layer of, let's just say cognition you put on top, they are required.
00:25:12
Speaker
And then other layers of abstraction might use exactly the same principles, not the same sources of information to the same degree, but that's what the abstraction is for. Exactly. Yeah. And therefore I also.
00:25:27
Speaker
think that that building up models about particular brain functions, models about the brain, of the brain, that they could be very useful. But then there's always the question, maybe a little bit in relation to your comment here, is that at which level do we build these models?
00:25:44
Speaker
And of course, the ideas in literature. David Marr, a famous efficient scientist, already proposed this idea of different levels by which you can describe a system. You can describe it more at the computational level. So what does the system need to do? What kind of computation does it need to make? How then do we implement these computations? That's the lower level. And what kind of algorithms make that link between computation and implementation?
00:26:13
Speaker
And so even that kind of approach can be very useful to understand sensorimotor systems, but it can also be very useful to understand language. Yes. Or whatever cognitive domain that we as humans, it's our taxonomy, right? Yeah. It's our labeling. It's our labeling, right. This is quite useful. It has appeared quite useful, but still the question if the brain
00:26:37
Speaker
operates like that in that sense. That's true. That's a tough one to answer, isn't it? Because we're the ones who try to figure it out and we need some type of, well, framework or language in that sense to describe it. We need a language in order to describe it and we need language in order to come up with some conception of what we think is going on there, right?
00:27:04
Speaker
And the more abstract we can make that language, the more formal we can make it. And you enter this idea of making computational models because they make very often precise. At least you formalize in a very precise manner what you really mean.
00:27:22
Speaker
And those models can make predictions and these bricks can be wrong, but at least then you've learned something, right? Exactly. I mean, that's the whole idea of science, right? You're supposed to disprove your models, not supposed to keep proving them. That's the whole proper idea, right? Disproof theories. Yeah, that's what I found difficult to explain to a lot of also listeners, because it often seems as if
00:27:50
Speaker
the scientist's job is to find evidence to support something. But it's rather find evidence to not support something. Because that's then when when things break down in air quotes, that's when it gets interesting. Yes. Not when when you keep confirming the same old story.
00:28:10
Speaker
No, of course, I think everyone would agree that gravity is a very useful motion in order to understand particular phenomena in nature. And replication is, I'm not trying to dismiss that, it's very important. No, but in the end, our scientific theories, they need to be disproven, right?
00:28:31
Speaker
So what I thought was interesting is I saw this TED talk from Daniel Volpert, and he's also a movement scientist.
00:28:46
Speaker
When he gave his talk, he was referring to, let me just call it the Bible of neuroscience, the principles of neuroscience, the book from Kandel and Schmidt and Schwartz. And yes, exactly. And he was referring to it and he just explained, okay, look, in 2000 or 2001, that book had X many pages.
00:29:15
Speaker
And now 10 years later, actually, I'm seeing it in your bookshelf. It's pretty thick. It's very thick. And that 10 years later, instead of the pages to become less, it became more. And his point was, we are actually, we are not, it looks like we understand more, but that's not the case, because the more we understand, as in,
00:29:40
Speaker
being able to, as you said, right, they put them in language to abstract these notions, to have principles that apply to many things, very, very exact definitions. They shouldn't become more pages. That's, that's what he said. Of course. What, what do you think about that? I think it was exaggerating. No, yeah. It's making the point. Right. Let's, let's be fair here. Yeah. Um,
00:30:08
Speaker
Well, I think sometimes also science needs to go through, build up a big database before we see, before we have a big breakthrough,
Interdisciplinary Neuroscience and Technology
00:30:17
Speaker
right? Right. And in the end, it also depends on what do you think or what do you...
00:30:23
Speaker
feel, well, what is understanding? What's your philosophy? When do we understand something? Do we understand something? There's also different ideas about it. Do we understand something if we can drive a model?
00:30:41
Speaker
Or do we understand something if we can build it as Feynman was saying? Yeah, Feynman was saying. So there's different notions of what people mean by understanding. And sometimes we might have to do many different experiments in order to come up with a new theory. So I'm not saying that it's
00:31:03
Speaker
It's bad that we add more data, but at the same time we should also work on trying to capture these data in some understanding that can make it a little bit more concise, right? I would agree to that. But I still find understanding is somewhat a loose term.
00:31:24
Speaker
Yes, that's true. But that's what I like about just neuroscience in general, because it's just interdisciplinary by its own nature. And not trying to bash any other sciences here, but I at the beginning, when I started out, I had no idea how big the philosophical component is in neuroscience, right next to the engineering component, right next to the computational, then the biological one. And
00:31:51
Speaker
I think this gives it, it doesn't necessarily make it easy to be coherent across all these sub parts. But I find that actually to be a strength, because every sub part might think about what you just said, what does it mean to understand something slightly different. And every part works on it, not necessarily on their own, but from their perspective, which can be quite enriching. I think so too. I think it's, well, the more diverse,
00:32:20
Speaker
The group of people is that works on a particular topic or field. I would argue that's the better and the higher the chance for a breakthrough in a particular domain. Yeah. Even though I still think that neuroscience is waiting for a very, we are not in physics yet, right? No, no, no. Far, far from that. Yeah. There's no known. Well, I don't know.
00:32:49
Speaker
We don't have the Newton's laws yet. No, no, no. Or the periodic table in chemistry or the genetic code from biology. But it also makes it interesting, right? Yes, of course. I think there's always...
00:33:09
Speaker
There are always things to keep pushing, which I find fascinating because it's such a personal thing to everyone. I mean, specifically perception. It just feels... This is what fascinates me. To us, perception feels so good, like we are very good at it.
00:33:29
Speaker
But when you start measuring, you start to see how bad we actually are at it. Or how optimal our perception is tuned in order to survive in the world in which we live. That's true. And how quick it can adapt.
00:33:47
Speaker
I think you always have to make a distinction between what's optimal and the brain seems quite optimal and what's perfect. The brain is not quite perfect, no. No. It seems to do what it just needs to do. Yeah. I would say that's a very specific trade-off. You can be perfect, but then it's rigid.
00:34:08
Speaker
Yeah. Or you're flexible and optimal for that specific situation to your capabilities, of course, of the system. But the good thing is that we are even not aware of particular sensory systems, right? We do lots of research, as you know, on the vestibular system. Yeah.
00:34:29
Speaker
It's even called hidden sense. People are not aware of it unless they're until something happens with the vestibular system and they get busy and nauseous and all of these effects that people find very unpleasant because the vestibular organ is not working properly. So we take perception kind of for granted or that our sensory systems function for granted.
00:34:52
Speaker
Well, you see it in disease that it's not obvious. You see it when you get older, it gets tough if your eyesight becomes worse, your hearing becomes worse. Yeah. This is what I actually find quite interesting is that, so lots of colleagues and friends are now getting children and it's just this thing that you
00:35:15
Speaker
They're so excited to see their child develop these motor functions, the perception, and it happens so quickly. If you think about it, just from a computational perspective, it's quite remarkable of what's happening. Especially in the first year. Especially in the first year. But then, just as you said, as soon as it's up and running, nobody thinks about it anymore. It's like, okay, now it's good. And it's, as you said, taken for granted.
00:35:44
Speaker
Yeah, take it for granted or push your system snowing in, right? Yeah. Yeah, participate in some, if you can, in some sports activity and you'll notice that your sensory motor systems, at some point there is a limit to them, right? Yeah. And that's not only the sensory motor systems as we talk about them, sensory modalities or the motor system muscles, et cetera. Of course, there's a whole lot of cognition that goes into it then.
00:36:11
Speaker
persistence and urine's motivation. Coming back to a previous thing that you said when it came to the breakthrough. I don't know if you heard of it, but so now you have this company that Elon Musk started as a neural link.
00:36:33
Speaker
Phillips working there now, isn't he? Sapes? Yeah. I've seen him in a few videos, which I think is great. The sewing machine. I'm sorry? The sewing machine. The sewing machine. Tell me about it. I don't know anything about it. How they bring electrodes into the brain, kind of sewing. What do you think about what they're doing in regards to the research?
00:36:55
Speaker
Well, I don't have a very strong... I'm not very informed, probably. I know a little bit the company. I know what they try to work on. What is that, actually? I don't really know that well. Well, I think what they try to, in the end, try to do is to...
00:37:19
Speaker
to connect, but it almost sounds a bit science fiction, to use the brain as a computational device in order to perform better, right? Even try to add some AI to the brain in order to make the brain an improved even superb system. And that's very science fiction. Yeah, that's very science fiction. But in order to get there, of course, they need to do all kinds of neurotechnology.
00:37:48
Speaker
How do we control neurons? How do we control how they generate their action potentials? What kind of action potential do they generate? I mean, with frequency, etc. Can we manipulate that? And we need a particular device in order to control those. And I think what they are currently working on most is on more developing neurotechnology as we talk about it.
Bio-Inspired Computing and AI
00:38:15
Speaker
That's the term that we use for that neurotechnology.
00:38:21
Speaker
So what I know, yeah, I haven't read much about it. But from what I recall is, of course, you hear the marketing side of it, which is the futuristic part that you talked about. But then you hear the technical side of it. And for me, that was
00:38:39
Speaker
more down to earth. Oh yeah, way more down to earth. Way more down to earth. There's very good scientists working there. Exactly. And then they know the problems in neuroscience. Yes, yes. But that's what I like. I mean, recording from single cells is not new. That's not new. But to do it in the way they are trying to do it, that's quite innovative. And trying to make it happen in the end, in the human brain, of course. We don't want to, of course, much of this work is currently being done.
00:39:09
Speaker
using non-human primate but in the end of course you want to
00:39:18
Speaker
at least society cares more about humans. Not to say that we do not care about animals, but we do studies in order to get insights in human diseases, etc. Improve the human brain or improve human function. Yeah, but these are very complex problems to solve still, that they are trying to solve there. Yeah, I mean, it just starts with what's the neural code? What's the neural code?
00:39:43
Speaker
But it also requires computation, it requires physiology, it requires understanding of biology. And the engineering. Engineering a lot. Yeah, not only how, this is what I find interesting, not only the biology of how the brain works or how plasticity changes might affect things, but also
00:40:06
Speaker
how the brain would react if you put something foreign in it. I mean, it's also, of course, a biological aspect that you need that you need to deal with. And you have all these other problems that come into just simply measuring from single cells. Sure. Which, yeah, I think they're tackling that quite well. Well, can we can we can we
00:40:31
Speaker
What can we use? Of course, that is all what we now call bio-inspired computation. Can we use the brain's principles in order to do computation? Yes.
00:40:47
Speaker
Even in computer tech, data storage. Can we make use of computations in computers? Can we use the same principles as the brain is using? Because that requires less power, it's more efficient. Can we rely on brain-inspired principles? That's why you also see a big branch in the field that's working on that now.
00:41:11
Speaker
Yeah. And do you have a few examples there? So now you mentioned our memory, right? Storage? Storage could be just computation. Just computation. Yeah. Do complex computations with our very power intents. They cost lots of kilowatts. If you could do them in the way the brain does its computations,
00:41:34
Speaker
And the brain is just using a few watts, right? Yeah, that's true. So if we can make that transition and you see that the field is now going a bit in that direction, then of course we also learn from the brain and have societal relevance in a different kind of field.
00:41:53
Speaker
So, what is your opinion on all the developments currently in AI going on, specifically when it comes to sensory integration? And, I mean, we skipped this now, but also the sensory motor integration. Maybe we should actually quickly explain the sensory motor integration part before we go into this. Sure. So, you explained what the sensory integration is.
00:42:19
Speaker
For simplicity, I would say now you also add the motor system for sensory motor integration.
00:42:30
Speaker
In the end, I cannot see perception completely isolated from, let's say, action or the motor system. So we perceive, and with that content of our perception, we try to do something. We act, we do not act, or we inhibit, or we do some. We might at least, something might be going on based on the contents of our perception.
00:42:53
Speaker
So basically you're saying we're not just perceiving for the sake of perceiving. Exactly. I don't think so. And we perceive in order to act. But if we act, there will be a consequence for our perception. And so that's a kind of closed loop. And so perception drives action, action drives perception. Of course there were the sensory systems.
00:43:20
Speaker
provide all kinds of information, some of which is more or less reliable. The same also in the motor system. Some signals that I send to my muscles are more precise than other signals. Some movements are more clumsy than other movements. I can improve on some movements. That means that I try to make them a bit more reliable. If I keep on exercising the surface in tennis, I will cut better because it will make this movement a bit more stereotyped. It means it's a bit less noisy.
00:43:49
Speaker
And so you have that loop between the sensory system and the motor system. But one of the problems is that the motor system can respond quite quickly and it can already make a change in
00:44:04
Speaker
in the world without you knowing that there's a change in the world because our perceptual system or sensory system, they are somewhat slow. Yeah. They have a delay. They have a delay. It takes 18 milliseconds before a visual cue is being processed.
00:44:20
Speaker
or maybe a bit shorter for an auditory cue, whereas the movement can already be over. And the idea is that the brain, that notion is referred to as the brain as a predictive machine or a prediction machine, that the brain uses the motor system in order to also make predictions about what the sensory systems are going to detect.
00:44:41
Speaker
And so basically there's two loops. Let's say there's the motor sensory loop, but there's also a motor predicted sensory loop. So now you have sensory predictions and you have new sensory information. And of course that also needs to be integrated.
00:44:59
Speaker
So, we also refer to that process of sensory motor integration. But the whole idea that the sensory and motor systems should not be studied or conceived in isolation, that is typically referred to as the problem of sensory motor integration. Yeah. Yeah. I mean, it makes sense. Specifically, if you perform a motor action and then you have like a sensory predictor being triggered by this,
Motor Control and Sensory Predictions
00:45:24
Speaker
you miss a big chunk of information if you were to study it in isolation, because this is quite important. So there's this one.
00:45:35
Speaker
one observation which i i find quite interesting every time i go to a bar and i see the waiter coming and they have something on the tray of course and then people at the table try to be nice so they take something off the tray they should not do that and i try to tell them why they should not do that they they just never believe me that if
00:45:57
Speaker
So just to explain, the waiter knows the weight of the tray and is very good at balancing it out. But if you take something off the tray, the sensory system of that waiter kicks in with this delay.
00:46:16
Speaker
while you know very well what you're picking up and when the waiter does not. So the arm of the waiter goes up, which can lead to imbalances and very often to disasters. While if the waiter does it
00:46:30
Speaker
him or herself, they very well know the weight and exactly at which moment it will leave, that's the sensory predictor you talked about, which then leads to the movement being perfectly, well, it actually leads to no movement because it's completely compensated. This is very smooth act in there. Yes.
00:46:46
Speaker
Well, it's the same reason that you can't tickle yourself. A very popular example. Why can't you tickle yourself? Because you can make, based on how you control your fingers, you can derive the sensory predictions in the brain and tune them down. Exactly. Whereas you are going to tickle me, I can't make these predictions because I have no access to your motor system. So it feels very ticklish.
00:47:07
Speaker
But same idea. Exactly. Same idea. But I just find it so fascinating that whenever I bring this up in wherever I am, people just don't believe me. They just, because they feel as if everything is so perfect in our sensory processing and control, that they just don't believe me. And we end up with a tray and everyone trying this 20 times because they are very convinced they will get better at it. But I try to tell them, it's like, there's nothing you can do about that.
00:47:36
Speaker
Well, I think if you do it in a kind of stereotype manner, of course, you can learn a bit. Sure, sure. But it's indeed. Which is then another sensory prediction, actually. Of course. Yeah. But at a different level. At a different level. Yeah, yeah, yeah.
00:47:51
Speaker
Yeah, but that is sensory motor integration. All right. And thanks for that explanation. And now taking that into what do you think about all the AI development, specifically in robotics, how can they well benefit from this actually? It depends a bit on how you approach those problems. Do you want to
00:48:15
Speaker
build bio-inspired robots or do you just see it as an engineering problem in which you just try to utilize the correct sensors and add them to your robot and try to solve a control problem?
00:48:38
Speaker
I think there's a world to gain by trying to make the connection between how humans behave, move, perceive perhaps, and trying to incorporate that into robotic devices.
00:48:54
Speaker
And that's subfield of AI, right? And trying to develop these robotic devices or robots. If you talk more about machines in general, then the interest is also going, what kind of algorithms do we need to develop? Can we take benefit of brain function?
00:49:18
Speaker
on how to develop these algorithms, or can we use these algorithms in order to decode brain function? And take, for example, artificial perception, a perception in an AI system. How do you want to develop it? Well, do you want to mimic?
00:49:35
Speaker
how the human brain does it, and that means that you're building neural networks. And that's what people have done. But now people see, well, you have to build these deep neural networks, deep learning, multiple layers, as we see in the brain, in order to set up vision. And so you see that there's lots of borrowing back and forth. And I think that makes the link between AI and neuroscience quite strong.
00:49:58
Speaker
But what would you say that, so the one big criticism I heard from the academic side about these deep neural nets that are being used in industry just as a, almost as a one size fits all solution. Like, oh, we have a problem. Just train a deep neural net. It's fine. Yeah. And if you have not a problem, train a deep neural net. Yeah. And that's a bit of criticism, but of course I think the AI field is working on that. One-shot learning. Right. Yeah. And we call, and that's how it's called.
00:50:25
Speaker
can we learn from just giving one new example as the human brain tries to do or in a few examples, right? We can generalize quite quickly. Well, there's all these examples and how deep neural nets can go wrong if it's trained on pictures of cats and you still see, as I would still recognize as a cat, but it's slightly different and the whole deep neural net is already, it goes awry, right? It's completely off.
00:50:53
Speaker
But I also have somehow, and I don't consider myself a deep expert there, that the field will go solve that.
00:51:02
Speaker
The question still is, do we in the end understand more about brain function? That is even a philosophical question. I think it goes back to a question of the difference between the theoretical understanding and the application, because I see a lot of people that like to have this in their arsenal of tools.
00:51:26
Speaker
But they don't care if, you know, if they understand a bit better how the human brain works, they couldn't care less. Yeah, exactly. They just like, I have this toolbox and I just use it to solve, to solve my, my problem, your problem, your categorization problem, your classification problem, what, what, what, whatever your problem is, right? And that's, that's totally valid.
00:51:46
Speaker
The question is, I think for academia, it's more important to develop new algorithms. I don't think that in academia we should build self-driving cars, right? Because the whole industry is working on those questions. But what academia can do is trying to push the science forward. New algorithms, new techniques.
00:52:14
Speaker
You mean new algorithms as in how the deep neural net can be trained? The whole field of machine learning. Let's develop that from a more theoretical side. And of course, if we train students, people in those fields, they can also bring it into society, right? And of course, we can make connections to society. But I see that more as the purpose for academia than really trying to
00:52:40
Speaker
start competing with with industry all these developments that are already going on in society yeah yeah I don't think that's necessary it's think everything everything has its place as long as well I see the biggest downfall if just specific parties don't talk to each other anymore that's that's when we talked about just before we started this whole conversation yeah exactly so who
Ethical Considerations in AI
00:53:09
Speaker
Who's in control here? I think that's a very fair point, a valid point, and that needs to be thought of deeply. And that not only requires people that work in AI, but it also requires philosophers, politicians, basically the whole society, right? It's a societal issue. Yes. And that needs to be addressed.
00:53:36
Speaker
But all the regulations that need to come in place and to guide these developments in a correct manner, I think there's really work to do as far as I can see it.
00:53:50
Speaker
But how would you go about it? It seems so vast of a problem. That is, of course, the problem. It goes so fast. And if it can only be tracked by a handful of people, of course, then that automatically enters the complexity, right? If society cannot keep up, that's not good. But I still think it's...
00:54:15
Speaker
It's quite crucial that we put lots of thinking on how we would like to have these developments, how we would like them to happen. And well, society should also have a stance in how these developments happen, and that should not be surprised by everything that comes to society, right? Yeah. And therefore, I think it's important that there is, but you're also arguing that we need connections.
00:54:42
Speaker
But of course, there's also different goals. There is no company as a business case. Sure. There's an economic incentive. Makes a total sense.
00:54:57
Speaker
And of course that a politician or society also needs to take that into account, but there's more to a society than just running it by economy. And for that, I think we need to bring in philosophers and maybe people move from other fields that help us all think about these problems.
00:55:20
Speaker
I mean, specifically, you mentioned self-driving cars before. That's just an example. Yeah, yeah, just as an example. But even there, it's how do you... So let's take an extreme example here. Let's say you have, just as a quick segue, you have a situation in traffic where the car just can't avoid
00:55:44
Speaker
causing damage. Let's say it can hit two humans or one human, but there's no third option. What's the car supposed to do? And there, I think is where the philosophical questions
00:56:02
Speaker
about morals and ethics coming to play, which is often overlooked as the consequence of these rapid developments. Because we have to deal with this all the time, and hopefully nobody ever has to make that decision.
00:56:19
Speaker
But still in the end there, so I read the books by Harari, I don't know if you know them. He wrote all these books on developments in biotechnology and neurotechnology, AI, etc. Very famous, this writer, author, opinion, he's driving forward opinion, but he's also pointing this out there, so even your example.
00:56:41
Speaker
What do we decide? So do we then need a branch of philosophers that say, well, this is the rule. In the end, it's still a human question too, right? Exactly. What I heard that how they are trying to go about it is they pose this as a question to normal humans, and then they take that and put that on the algorithm.
00:57:03
Speaker
But still... So in order to get these systems accepted in society, and maybe not the self-driving cars, of course, but also other systems, well, they take away some of the control by humans, and it might be good, as examples. An AI machine might be able to see more on x-ray image.
00:57:29
Speaker
or an MRI image than a human doctor. But in the end, the AI doctor needs to work together with the human doctor because there's more to treatment than just diagnosing or to healthcare or to medical systems than just diagnosing a disease. But it's very difficult. It is. And also your point of... Well, let's ask...
00:57:57
Speaker
a group of people, how they would respond and let's build that into an AI machine. Because there's many differences in how humans respond. That's what we know. And let's start out with that.
00:58:08
Speaker
And we have to deal with very noisy information. Information is uncertain. And some people make different choices in uncertain situations because we do not know all the inputs to the system in the human case. So how do we know what's the right decision? I think this is a philosophical
00:58:33
Speaker
an issue that needs to be dealt with. And we can all say, well, this is the law and follow the law. But the question is, are the current laws and regulations, are they apt? I think the current laws are written for humans, for human cases. And soon there will be more. Exactly. And the development goes so fast, it needs to be, I would actually say this should have been the discussion about all of this sort of been started 10 years ago.
00:59:03
Speaker
Well, I think it's very difficult just to give the politicians the benefit of the doubt. I normally don't do that, but let's say to do that, it's very difficult to see how fast development goes if you're not in the field itself and see every day, oh, we're here already. Yeah, I agree. But rather than when the self-driving car is on the road. I agree. And in that respect, it's
00:59:29
Speaker
society needs to have some control on these developments in order to protect society also against the erroneous transitions. Yes. Yeah, exactly. This is also something we need to get used to. Just other transitions happen much slower. Yes. Significantly slower actually. Yeah. That was nice. Yeah. Well, it depends who you ask. If you ask the business.
00:59:59
Speaker
In the end, we're all human beings, right?
Visual Stability and Sensory Predictions
01:00:02
Speaker
Exactly. I have a few more questions. Just to get back to...
01:00:10
Speaker
How good the sensory system is to compensate for predictions that it may, for the, no, let me start over. How good the sensory system is by taking predictions of the motor system into account, which is, so one of them that I was always very fascinated by was the eye itself, right? You study this extensively.
01:00:31
Speaker
And one of those parts is the, and if I make them, if I overgeneralize this, please correct me, is the visual stability thing. I mean, we don't, we see everything very stable, but we make eye movements all the time.
01:00:46
Speaker
But we don't really realize that those eye movements don't change our percept of the world. But if we, if somebody ever tried to take a picture or a video with their camera, they immediately see when they're shaking. Sure. But so the, but if you think about it, the eye is like a camera. But how come with all these eye movements, there's no camera shake for us? Okay, could you could you explain that?
01:01:12
Speaker
Well, that fits that whole notion of the brain being a prediction machine. So indeed, we have the eyes and the back of the eyes. We call that a retina. There's the lay with photoreceptor, so they detect light.
01:01:30
Speaker
And the eyes, they can move very fast. We can categorize eye movements in different categories, but one of which is these very rapid eye movements, which are called sec hands. So they can go up to 500 degrees per second, 600.
01:01:47
Speaker
And so you make this very rapid eye movement. But if you move your eyes, of course, also the photo receptive layer, that takes a different orientation. And so if there's an object in front of you that stimulates the photo receptive layer at one location, but now if you make an eye movement, the same object stimulates the photo receptive layer at another location. And you're not aware of that at all.
01:02:14
Speaker
And then the question is, how come? Well, if the brain needs to wait every time that set of photo receptors is stimulated to process that information, it takes about, as we said before, 80 milliseconds. So you would see snapshots of the world, kind of.
01:02:35
Speaker
And that's what we do not do. And so the idea now is that the brain at the moment, it tries to, it prepares for making these rapid eye movements, these seconds, and it makes about three to four per second. So you make a hundred thousands of them every day. So at the moment that the brain starts preparing, getting the system ready to make the eye movement. And there's already a little bit of activation that is
01:03:01
Speaker
helping the eyes to move, so it's already sent to muscles at the same time, the brain uses that signal, so it's a motor signal, in order to set up a prediction of what the retina is going to receive. And so the brain is not surprised that the object first stimulated that set of photoreceptors, and now it's stimulating this set of photoreceptors, so it's not surprised.
01:03:25
Speaker
about this new sensory input. This is what I knew, I anticipated that, so the world has remained stable. And so that's one of the ideas. And so this is called the problem of visual stability. Of course, you have that not only for eye movements, head movements, same problem in the auditory system, touch system, every system needs to deal with that.
01:03:48
Speaker
And one of the ideas is that the brain uses these sensory predictions based on motor signals. And that's visual stability. Is that explaining it? Yeah, I think so. But does that then also mean that because, so if it can't rely on the feedback, because you said 80 to 100 milliseconds, and the saccade is so fast that I can't see anything during the saccade eye movement,
01:04:17
Speaker
So I'm actually blind while my eye is moving. Kind of, yeah. Kind of. Yeah, try to make eye moves while you're in front of a mirror. You don't see your eyes move.
01:04:29
Speaker
I never tried that. But it also means that I have to be the one making the eye movements. So if there was external control, things would feel very different. Yeah. If you would push your eyeball with, let's say, with your index finger and you push a little bit on your eyeball, you will see that the world shifts a bit.
01:04:49
Speaker
Because you cannot make these predictions. It's not a motor signal that's sent to the eye muscles. It's not a motor signal. Our muscles and brain cannot use that signal in order to make sensory predictions on vision. But the idea is then, so maybe to take it one step further, so the eye.
01:05:08
Speaker
the system, the visual system makes a prediction about this is, this is what my photoreceptors are going to detect. Then if the eye has made its movement, the photoreceptors will detect that object that is now stimulating it. So the brain basically almost takes two samples, it makes a prediction and then it gets the new
01:05:33
Speaker
input on the retina. And so suppose that the object would have moved, even its very rapid eye movement does not happen. Then the brain say, well, this was my prediction, but apparently my photoreceptors have not detected it. And so now it can conclude because the object has stimulated a different set of photoreceptors. Well, it must have moved, right? And so by so doing, the brain can create stability, but also knows when there's instability.
01:06:01
Speaker
Right. And can you mess with it in a sense of you can move? So basically, so the brain makes the prediction, and it says some, I move my eye, then I should see this. But then if you just also slightly change it, not too much, but also slightly that the prediction itself is going to change.
01:06:23
Speaker
You could try. You could try if the prediction is going to change. But the movement of the eye follows that. It's consistent with that prediction. And of course, you will still perceive stability. But what you could also do, as you can say, well, at the moment that I move my eyes and I start making the eyes, I just, and these experiments have been done, I stimulate at the level of the muscle and that the muscle contracts slightly more.
01:06:52
Speaker
So then the eye has moved more than you would base your prediction on, then of course you would create a percent of instability. Right.
01:07:01
Speaker
But then would your muscles adjust to that to try to reduce instability or just find it? The brain will try to map. If this happens all the time, the brain will start learning. And it will start to create a new mapping between, let's say, the motor signals and the sensory prediction. And people refer to that as the Ford model, as you know, as you have worked on that. Right. I did work on that.
01:07:27
Speaker
But, but I should be talking here myself. No, I understand. I understand. But this is, this is the idea. So if the sensory, if there's, if you continuously disrupt the sensory signal in relation to the sensory prediction and the brain will start to adapt. So you would say, maybe I go too far with this, but actually our percept of a stable world is an illusion.
Applications in Sports and Therapy
01:07:57
Speaker
It's something organized internally. Yeah. So it's a construct. It's a construct. Perception is a construct by itself, of course. Yeah. And I think that's what often people miss is the difference between sensation and perception. Yeah. Well, make that distinction. So what we added...
01:08:17
Speaker
There's, of course, famous illusions that show you already. The same stimulation on the retina, on the photo receptors, can create different percepts. Why is that? Because there's a difference between perception and sensation. I always found that very fascinating. It is very fascinating. Especially because it's so low level.
01:08:41
Speaker
And it's so replicable. Yeah, of course. It already happens at the level of the retina. In the retina there are so many cells. There's a whole neural network in the retina that already starts processing the visual input. Yeah. To take it, so now going a bit away from the hardcore research, more into what can we potentially, except of course AI and robotics, gain out of this type of
01:09:07
Speaker
knowledge, understanding these mechanisms. You mentioned before, of course, when we do sports, we get better and better and we realize, okay, our sensory system is not perfect in this new situation that we throw ourselves in. Let it be, I don't know. Even throwing darts, something very simple. I mean, I'm terrible at it. But you can very quickly see improvements.
01:09:35
Speaker
Do you think these mechanisms that we then start to understand better and better can also aid into potentially diagnosing disease? That's what I would be hoping. At least that's also what I'm thinking. You see all these developments also even in psychiatry, this whole notion of computational psychiatry, which is a new field, kind of.
01:10:03
Speaker
And the idea is that if we can capture particular processes by models or by principles, another assumption that everyone relies on the same principles, but let's say the
01:10:19
Speaker
the knobs in the system are just slightly differently tuned in particular between humans between different individuals or different if you compare disease to health then if you if you get a handle on
01:10:34
Speaker
what the value must be on these knobs or the range of these knobs, then you could perhaps also try to at least could help for diagnosis, but you could also help to track the outcome of therapy or medicine, etc.
01:10:56
Speaker
Yeah, because what I heard very often is that specifically now that you mentioned computational psychiatry, just psychiatry in general, the DSM book, I mean, just as it should, right, it goes through specific iterations.
01:11:11
Speaker
but often that's just a feeling of mine. I actually don't know if this is any correct. So people please take this with the benefit of the doubt. But what I mean is that sometimes I have the feeling that in psychiatry, some diagnosis feels a bit like, okay, this doctor would make that diagnosis and that doctor would make that diagnosis because you have the symptoms of course, which are listed.
01:11:39
Speaker
But specifically for certain doctors, you have some might weigh certain symptoms more, some might weigh them differently. So you might end up in a very different category. But what I think is very beneficial about the, let's call it the sensory motor approach into this diagnostic, it makes it very quantifiable. And maybe that's a computational aspect of this. Yeah, it's completely off guard.
01:12:06
Speaker
It's also known that all models are wrong. Let's keep that in mind too. And then you can still...
01:12:16
Speaker
You can still shift. Well, you can use the same arguments as you now made for the DSM in relation to the models. This is the wrong model, right? And I'm relying more on that model. So you get the same. So it's not an easy problem to solve, but I still believe, at least I feel that by taking computational approaches, we at least are very formalistic in
01:12:46
Speaker
in phrasing and conceiving what we think we are measuring or we're quantifying. And that could be useful. But that's not to say that this is the perfect approach, but it's an approach that now you see this developing in the field. It's more computational approach and it's applied now in psychiatry. But you also see it more and more in psychology, of course.
01:13:09
Speaker
No, I think you're right. I think I don't mean necessarily saying the model is better than the doctors themselves because obviously they're applying a model as well. I think I mean the standardization of quantifying. You get a different type of standardization. Yeah, I think that's what I meant. Yeah, that's what you meant. Yeah. No, I think that's definitely a big leap forward and
VR and AR in Sensory Research
01:13:33
Speaker
can be. It can be. It still needs to prove itself probably. That's true. But I also am not an expert here.
01:13:38
Speaker
That's true. Well, but still, um, as one of my last questions, I see an industry VR a lot and I hear, I mean, these, these predictions are now a little, maybe I should say outdated because they didn't come to, come to life yet, but that we, VR will be all around us. We will do everything with VR. And then they, they came AR being augmented reality.
01:14:09
Speaker
How much do you think those things, those approaches, those devices will mess with our minds? Of course.
01:14:24
Speaker
Well, it depends on how you want to use them, right? If you want to create VR to simulate the real world and do things that you do not want to do yet in the real world for Champlain therapy, you could use VR in order to teach patients to walk again, for example.
01:14:43
Speaker
In particular patient groups, for example, stroke patients, you do not dare to put them on the street yet because it's too dangerous. Then you could set up a VR environment. It simulates the real life situation.
01:14:59
Speaker
And then all of that and the whole three-dimensional visual display that simulates the real world in order to set up a therapy or do experiments that are unsafe to do yet in real life.
01:15:15
Speaker
Of course, it's never completely real life. I still believe in VR as making the connection between, let's say, that's also how I see it now and also the direction that we take as a group, to make a connection between what we do in the lab and principles that we have discovered in the lab and theories that we have tested in lab environments.
01:15:37
Speaker
We would like to know whether they also hold up in real life, but to make that connection, I think VR and AR and everything related is excellent in order to serve that goal, right? That in VR
01:15:52
Speaker
we will not completely mimic the real world. That's totally true. But we still, I think, we have somewhat of an experiment to control that mimics the real world, which we do not have in the real world. And then we really live in an ambiguous, noisy environment in which we do not completely control the environment. In VR, we can mimic a real world.
01:16:16
Speaker
But still do this from an experimental control type of site, which I strongly believe in. But that we, of course, that we... Isn't that the meaning of VR and AR? That we try to make sensory stimulation as...
01:16:34
Speaker
well, as rich as possible. And of course, we can also make it overly rich or bias, sensory systems are. Yeah, of course, it depends on you. Yeah, that's the entertainment side I see is going for that. It's not trying to see how can we control the environment, but more
01:16:57
Speaker
How can we get the person more sucked into this environment? Yeah, that can be very good. Yeah, that can be very good. Of course you make it way more a real life experience.
01:17:10
Speaker
That's maybe even over. Yeah. Beyond real life. Right. Yeah. That's, that's what I, um, this is what I like about the VR, especially the Oculus Rift. When, when those started, those VR glasses, when they had their roller coaster demo, so everyone is listening, you can find those, uh, example video examples on YouTube. I'll put one in the show notes as well. And what you see is the person standing.
01:17:37
Speaker
and having those VR glasses on and going through this roller coaster environment. And you as a spectator see on a TV what they see. But you can see them being so immersed in that environment is that they cannot keep their balance, which is
01:17:56
Speaker
coming back to what we started talking about, is that the visual system says one thing, and your balance system, the vestibular system, says something very different. Of course, but that is at least one of the big business models in amusement parks, right? Yes, that's very true. You're right, I never thought about that. You're right, it's actually the discrepancy between and messing with them, which can cause pleasure.
01:18:22
Speaker
Yeah, yeah. Not to me, I really dislike roller coasters, but most people love them. Well, in the Afteling, I forgot the name, this attraction, and we see really make, create confounds, the conflicts between, let's say, visual and vestibular systems.
01:18:42
Speaker
Yeah. Yeah. And you also have these, um, I don't, I don't know if they still exist, but when I went once they were still a thing and I could roughly deal with that is when you're on this chair and you see something happening in that environment, but your chair is basically moving in that same way. Sure. Which is same, exactly the same idea. The only difference is you're not, there are no loopings involved or anything crazy like that. So it's fine for me to deal with.
01:19:10
Speaker
flight simulators, car simulators, and they try to mimic the situation as much as possible. And it's very immersive. You have tried them out a few times. It's very immersive. It's very immersive. And I think it's only getting better and better. But this is one part where I think that the fundamental research that you're doing is really, it has actually shown to be important when it came to making them more immersive, because you cannot just make them more immersive from an engineering perspective.
01:19:40
Speaker
Because that's not how the brain processes the information. That's what happened with the Oculus at the beginning. They had these problems. They made it engineeringly perfect. But humans didn't experience those environments as real until they were modified to take into account how we process information. And then they were insanely immersive, which I thought that was beautiful.
01:20:09
Speaker
Yeah, but I think that's also realized now in also neuroscience and neuroscience can go without understanding behavior.
Human Perception in Design
01:20:16
Speaker
And there was a time in which neuroscience was really more into left a little bit of out the behavior. Yeah. But of course, the brain is shaped in order to control our behaviors.
01:20:26
Speaker
So we also need a handle on how behavior can be quantified, can be characterized, and we have to bring it into our system in order to also understand how the brain works. That's also what I strongly believe in. And for that reason, I think it's important to also go to more real world experiments.
01:20:44
Speaker
Yeah. How much do you think then can this research about sensory knowledge, about sensory integration, sensory motor integration, perception, how much do you think designers for anything, let it be apps, let it be therapy, let it be sitting in a restaurant enjoying a meal, how much
01:21:07
Speaker
Should the people that design those experiences take this into account or do you think it's too low level? I don't think it's too low level. I think I'm not saying that every designer needs to know how the brain functions, but I think it's important that he or she knows how human behavior comes about.
01:21:25
Speaker
And how humans might be very useful to know a little bit about perception and action in order to understand the human condition and make tools or interfaces that link up to those conditions. And if you only approach it, let's say from one side,
01:21:45
Speaker
I think you lose on the other side a lot. And I think that's what I would suggest to prevent yourselves from. Right. Are there some books that you can recommend for people who want to learn more? I mean, you mentioned Harari, was it? Yeah, but that's more on Jofaal Harari. He's a very popular author these days. He wrote very... Well, I would say I personally like the books a lot. Yeah.
01:22:15
Speaker
on how this whole development in AI, biotechnology, how that will change the world and how we deal with data, human data, et cetera. I think that's usually if people would like to know more about those developments and how they affect, how they could affect society and what could be coming. Let's phrase it like that.
01:22:41
Speaker
If your question would be more from a neuroscience perspective, what kinds of books should people read? Is that your question? Yeah, that might want to learn more about sensory motor integration.
01:22:56
Speaker
Yeah, of course, a few good books have been written a bit more for students in neuroscience or students in sensorimotor neuroscience. There are some good books. I'd rather start me. You know, Emma has written some good books, but I would argue they are a bit more textbooks and that they would serve, let's say, a designer.
01:23:19
Speaker
There must be human factors books that people could look into. I have to think about it a bit more in order to help you with a good suggestion. No, no, no. If you find some later, I will simply add those to the show notes, even if you don't know them right now by heart. Well, there's many great books that I can highly recommend, but if you really ask for, let's say for the UX type of
01:23:44
Speaker
Yeah, I don't know any. That's my problem. That might be a bit more difficult. Yeah. Yeah. So whoever is listening and knows about motor and sensory motor stuff. In that application, that would be very interesting. Yeah, because I don't know any. I was hoping you know. Of course, my field is a bit more on the
01:24:07
Speaker
on the other side of that question probably. But yeah, I'll look into it. Let's see what we can find. Are there any more things that you would like to add? Do you think there's something else that is very exciting in the field of sensor motor integration that recently happened that you think would excite the listeners?
01:24:29
Speaker
Well, I think I really would, if I would advise the listeners, stay tuned on what's going on in this field, because I think what I now see is that field is also developing quickly, and that field is moving away a little bit from pure lab-based experiments, trying to do studies and develop theories that also hold up in the real world.
01:24:51
Speaker
And that is, I think, exactly what also a group, at least some of your listeners might be interested in. And I see that coming up in the next few years. Yeah. Yeah. Oh, I think so too. Well, Peter, thanks so much for taking the time.
Conclusion and Further Engagement
01:25:09
Speaker
If people want to get in touch with you, can they find you on Twitter or how can they get to you?
01:25:16
Speaker
on Twitter, through our website, through email. What is your Twitter handle?
01:25:27
Speaker
All right. I will also put this in the show notes so people don't have to write anything down. Again, so thanks so much for taking the time. It was a pleasure to talk to you. Yeah. It's been way too long, as usual. Yeah, well. It's been too long time, I mean. And now we go to the pub. Yeah, we could do that. All right. Okay. So to everyone listening, yeah, you have a great day.
01:25:51
Speaker
Hey everyone, just one more thing before you go. I hope you enjoyed the show and to stay up to date with future episodes and extra content, you can sign up to the blog and you'll get an email every Friday that provides some fun before you head off for the weekend. Don't worry, it'll be a short email where I share cool things that I have found or what I've been up to. If you want to receive that, just go to ajmal.com. A-D-J-M-A-L dot com. And you can sign up right there. I hope you enjoy.