Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Episode 15: Three! Easy! Rules! About! Research!
190 Plays2 years ago
Welcome to Episode 15 of the Movement Logic podcast! In this episode, Sarah discusses the lima beans of learning: research. But guess what? Reading research doesn’t have to feel like a chore, and the more you understand what different aspects of research mean, the more interesting it becomes.
- Three easy rules to follow for quoting research as if it’s facts
- Why we get so excited about some studies - and why we need to pump the brakes
- How to know the quality - and thus the value - of the research you are reading
- Why it’s important that movement teachers quote research responsibly and what can go wrong when studies are not good quality
Reference links:
Yoga Reverses Osteoporosis Study
Sign up here for the Movement Logic Newsletter for course discounts and sales and receive a free mini Pelvic Floor course!
Watch the video of this conversation at: www.movementlogictutorials.com/podcast
Want more Movement Logic?
- Follow us on IG: @movementlogictutorials
- Get on our mailing list for tons of free content
- Contact us (we're super friendly)
Thanks for listening!
Recommended
Transcript
Introduction to Hosts and Focus
00:00:02
Speaker
Welcome to the Movement Logic podcast with yoga teacher and strength coach Laurel Beaversdorf and physical therapist, Dr. Sarah Court. With over 30 years combined experience in the yoga, movement, and physical therapy worlds, we believe in strong opinions loosely held, which means we're not hyping outdated movement concepts. Instead, we're here with up to date and cutting edge tools, evidence, and ideas to help you as a mover and a teacher.
00:00:34
Speaker
Welcome to episode 15 of the Movement Logic
Research Challenges and Misinterpretations with Dr. Sarah Court
00:00:37
Speaker
Podcast. My name is Dr. Sarah Court, physical therapist and motorcycle enthusiast. And today I'm going to talk about research now before you run away screaming. Please know that I get it.
00:00:51
Speaker
trying to understand research can be really hard. It can feel frustrating and it can feel like it's just, you know, better left to other people. I had a really good example of this experience. I run a small mentorship group for movement teachers. It's like a monthly ongoing content course and I tried to get them excited about doing one meeting.
00:01:11
Speaker
about one meeting, not like 15, one meeting about research. And the feedback that I got was a resounding, loud and resounding, no thanks. So I get it. It can feel like it's just something that you don't wanna do, but the problem is a lot of research that we hear about in the movement world has been interpreted correctly.
00:01:38
Speaker
has been misinterpreted. I think is that actually the word I'm looking for? I don't think you can interpret something miss correctly. Anyway, it's been interpreted incorrectly and then that incorrect interpretation gets spread around as if it's gospel. So I want to help you avoid that error.
00:01:57
Speaker
And in order to do that, I'm going to give you three easy rules to follow, as well as the three most important things to be aware of when you are actually looking at a research paper. And I'm hopeful that after you listen to this episode, you will be less intimidated, you will be better equipped, and potentially you might even be like,
00:02:19
Speaker
kind of a little bit jazzed about reading research.
Verifying Research Sources
00:02:22
Speaker
I'm not gonna hold my breath on that last one, but a few of you, a few of you might be like, you know what? Actually, this is kind of cool. All right, so let's get into it. Three easy rules. So rule number one, if you read about a research paper in an article, you are not allowed to quote anything just from that article.
00:02:44
Speaker
So what does that mean? So for example, let's say you're reading yoga journal and they publish an article that says a recent research paper found a connection between practicing yoga and reducing depression. That sounds really cool. However, you cannot then, after you read your yoga journal,
00:03:00
Speaker
go teach your class and say, a recent research found a link between yoga and reducing depression. Maybe it did, but my point is you have to go and look at the actual research paper itself that the article is quoting.
Impact of Misinterpreted Research on Society
00:03:18
Speaker
And if the article does not provide you with a direct link to that research paper, you have to say, automatically no. That's a David Rose, Schitt's Creek, immediate hard pass, automatically no. If the article itself is not showing you where it got the information from, that's got to set off like a big old alarm in your head. Most articles, most well-regarded journals will give you the link to the article. They just, they should, they have to.
00:03:48
Speaker
However, very often what's actually happening is the article itself will write a very clickbaity or kind of hyping title to get you to read the article. But when you then go click on the research paper and read what the research is actually saying, it doesn't actually say the same thing that the article is claiming that it does.
00:04:14
Speaker
There's an example of incredibly destructive repercussions that are still being repudiated 20 years later.
00:04:25
Speaker
That is the women's health initiative study that started in the late 1990s looking at menopausal women and hormone replacement therapy and what some side effects or if there were any negative side effects that were taking place. The study itself was flawed and the authors have since retracted and repudiated their own results. They are now saying we misinterpreted the results.
00:04:49
Speaker
what the media did was they saw some of the results which were there's a tiny number of people that had cardiovascular disease or breast cancer. The media took a hold of those and blew it way out of proportion. The repercussions have been that almost overnight at that point, something between 70-80 percent of women who were on hormone replacement therapy stopped.
00:05:17
Speaker
To this day, so that was 20 years ago to this day, I think there's something like five or 10% of women who could be benefiting from hormone replacement therapy during menopause who are actually doing it because that misinformation spread so fast and so wide and
00:05:35
Speaker
As someone who is doing a small amount of hormone replacement therapy, the amount that I'm permitted to as a breast cancer survivor for my own medically-induced menopause symptoms, which are significant,
00:05:48
Speaker
I can't fathom going through this without the help that I'm getting.
Questioning Unverified Claims
00:05:53
Speaker
And the number of women who are suffering needlessly, continue to suffer needlessly for years and years and years from all of the symptoms of menopause that can be treated, it just makes my head want to explode. So that's a really strong example of how a misinterpretation by an article or some other piece of media
00:06:18
Speaker
and cause huge ripple effects into society. So if you're like, well, okay, but you're telling me, don't just read the article, go back and read the research paper, but I don't know what I'm supposed to look for in the research paper, don't worry. I am going to tell you in a little bit, but at the moment we're still on our three easy rules. So rule number two, if you are taking a class,
00:06:47
Speaker
And the class teacher says, research has shown, and then they say something that is either super vague, like deep breathing is good for you, or super specific, like this yoga asana pose prevents diabetes, or
00:07:07
Speaker
they can't really explain what they mean. Ask them afterwards if they can send you a link to the research, or where they read about it, or where they got this piece of information that they are disseminating in class. And if their answer is something like, oh, my teacher told me, or I read this thing online, you do not get to repeat what they said and call it facts.
Examples of Misleading Studies
00:07:31
Speaker
People say ridiculous things in class that they have heard from, who knows where, from teachers, from other friends, a headline on an article that they didn't read. And these are not facts.
00:07:45
Speaker
My friends. I can't remember if I've told this story in the podcast or somewhere else, but I took a, I think it was a bar class one time, and as we're all like sweating and working, and I really don't enjoy it personally. I'm not sure why I was there.
00:08:04
Speaker
Sometimes I'll keep taking classes just to be like, wait, do I really hate this? And then I'm in the class and like, yes, I really hate this. So anyway, I was there not especially enjoying what was happening, but the teacher then said at one point, this kind of like bar exercise, this kind of exercise burns the fat that's on the inside of your body. And I was just like, what are you talking about?
00:08:30
Speaker
as opposed to the fat that I brought in my purse, like all of my fat is on the inside of my body. So it made no sense at all, right? People say weird things. Anyway.
00:08:44
Speaker
So an example of something where people get really excited about a study but they don't actually understand what the study meant or they don't have the opportunity, I should say, to really pick it apart and figure out if it's a good study or not. There was one about five years ago that was really kind of like popular and viral in the yoga and movement world. And it was a study about getting up and down from the floor
00:09:11
Speaker
and the link between if you had to put a hand down or a knee down or how much you had to help yourself as a direct link to mortality. The way that it worked was if you could stand up without using your hands, that was the best score. And then every sort of hand or elbow or knee that you had to put down, you got points detracted.
00:09:37
Speaker
So there are a few things when I looked at the study that are really just kind of off. First of all, if you're not walking around practicing getting up and down from the floor without using your hands, it's not actually that easy to do. And you would probably put a hand down whether or not you actually need to from a strength or flexibility standpoint. You would just do it because it's easier. And the instructions, the verbal instructions that people were given
00:10:02
Speaker
were, without worrying about the speed of movement, try to sit and then rise from the floor using the minimum support that you believe is needed.
00:10:12
Speaker
which to me is super vague because the minimum support I believe is needed, I would just sort of like, well, I believe I'm gonna put my hand down first because that's gonna be the most comfortable way to do this, right? So I'm not getting information around, I understand that they're trying to not bias people about how to do it, but at the same time, I don't know, just to me, the wording of it is very vague and misleading a little bit.
00:10:39
Speaker
the criteria for the people who were included in the test, they specifically excluded people who played sports. And by excluding people who played sports, they were automatically skewing their results towards less functional, less strong, less mobile people.
00:10:59
Speaker
They also excluded people who had a musculoskeletal injury that would get in the way of the test and that I wanted to make a point that's actually I think okay because within that group of people like let's say someone had an ankle sprain or they had knee surgery or something like that.
00:11:15
Speaker
there's going to be a variety in that group of fitness levels. Some people might have injured themselves, you know, being active. Some people might have injured themselves just in an accident. So that's actually okay. But what the test, you know, quote unquote, found was that the older that people were, the less well they performed the test. And then they correlated that to a higher all cause mortality.
00:11:39
Speaker
But the other thing that puts you at risk of mortality is just being older. So the study showed that the older people were, the less well they performed this test, the older people are, the more likely it is that they're coming towards the end of their life. So everyone was freaking obsessed with this proof for a while. And I think it's not in small part because
00:12:06
Speaker
It was very social media friendly. You could film yourself getting down to the floor and back up again without using your hands. You could engage your followers. You could have them try it. You could create a hashtag. You could get lots of likes and comments and shares. And you could also sound sort of semi-intellectual that this came out of a research paper. And, you know,
00:12:35
Speaker
I think a lot of people leaned into that, but the problem was the research was deeply, deeply flawed. So it would be more accurate to show the exercise and be like, hey, this is hard and fun. Try it. Tell me what you think.
Hierarchy of Research Quality
00:12:51
Speaker
That's literally all you could actually claim about it as far as it's showing you at what age are you going to die.
00:12:59
Speaker
Okay, rule number three, if it sounds too good to be true, it probably is.
00:13:07
Speaker
There is a study that came out a few years ago about yoga for osteoporosis. And the title of the study is very misleading. It suggests that all you need is 12 minutes of yoga a day and you can reverse bone density loss. But in the study itself, they say that they are looking at using yoga to prevent fractures from falling, which is not at all the same thing. Those are two very different things.
00:13:34
Speaker
The methods described are kind of iffy and hard to understand. And I don't want to go into it too much because Laurel talks about this study in her episode on bone density. I think that's episode five. Jules Mitchell and I discuss it in our episode. I believe it's episode 12 as part of the episode. And so if you want to hear more about what about the study is it like not so great, you should go back and listen to those episodes.
00:13:59
Speaker
It's hard because this would be, you know, if this actually was proved to be true, it would be fantastic. What an easy and cheap solution that doesn't involve, you know, medication or other expensive, you know, you just do some yoga every day and you could reverse your bone density loss. But it sounds too good to be true because it is too good to be true. OK, so those are your three super easy rules. Number one, read the actual paper, not just the article about it.
00:14:29
Speaker
Number two, if there is no source for the claim that you hear from a teacher, read in a book,
00:14:37
Speaker
over here in a coffee shop, you cannot just swallow it wholesale. You need to assume that it's just not true. And number three, if something sounds too good to be true, you need to have a super, super hypercritical eye about it. Okay, so let's say I'm at the point where I read the article, the article said something intriguing, it was interesting. And then I clicked through and the research paper was there, the whole thing.
00:15:06
Speaker
Well, so now what do I do? Because let's say I don't, you know, when I was in PT school, we had multiple courses about research, how to read research, how to create it, how to make our own studies. We had to design and run our own research studies. So I have a lot of experience around that. But that's from PT school. When I was a yoga teacher, I had exactly zero experience around it.
00:15:30
Speaker
If that's you
Assessing Research: Key Factors
00:15:31
Speaker
and you're like okay well great now I clicked through the research paper and it's just a bunch of words that are like swimming and there's like statistics and something about a power and a significance and a little you can you can look at a paper.
00:15:44
Speaker
and still have that hypercritical eye and just understand what things you need to pay attention to. And I'm going to give you three things to pay attention to while you're looking at a study. So the first thing is knowing that not all studies are created equal.
00:16:04
Speaker
by that I mean there's a hierarchy in terms of how quote-unquote good research is based on the kind of study or analysis of other studies that it is. And essentially the ones that you can put more faith in are the ones that are higher up in that hierarchy.
00:16:26
Speaker
So the categories that I want you to keep an eye out for are three. The first one is something called a systematic review.
00:16:37
Speaker
So with a systematic review, you are not creating your own new research study. You're not getting volunteers and all that kind of stuff. That's not what you're doing. What you're doing is you're coming up with a question and then you are going through the existing research and using existing research to answer it.
00:16:56
Speaker
You pick out you know 1000 studies that are on that topic and then you exclude the ones that aren't specifically related to your question and then you go through and you pick out the ones that are well designed well put together and you leave out the ones that are not well designed not well put together.
00:17:12
Speaker
And then you use that to get the answer to your question. And so what's good about that is instead of it being a solo study, you are looking at possibly hundreds of other studies on this topic. And so there's more data and so we get better results that way. Within a systematic review, you would also be very likely to do something called a meta-analysis. A meta-analysis is our second category.
00:17:41
Speaker
So we've got systematic review when you're taking, you have a question and that you're going to get it answered by looking at a bunch of other studies. Within that systematic review, you may do something called a meta-analysis, but you could also do a meta-analysis just by itself. So a meta-analysis is when you take a bunch of studies that are all on the same subject
00:18:02
Speaker
And you just, you go through the same process where maybe you exclude them if they're not as good or if they're not quite on the same subject or things like that and you get sort of a core bunch of studies. And then you extract all of the results from all of the studies and kind of blend them together and do your statistical analysis on that big pool of data.
00:18:27
Speaker
So what's really good about that, which hopefully makes, you may already intuitively be like, oh, that sounds good. Again, instead of, let's say your study that you ran that had a hundred people in it, now we have a data pool of 50,000 people. And so the outliers become much smaller in terms of how often they pop up when we get more and more data like that.
00:18:51
Speaker
So those two, systematic review and meta-analysis, those two are when you are looking at research that has already taken place. The third category that is of the best types of research is something called a randomized controlled trial. So this is the gold standard.
00:19:13
Speaker
of running a study. What's so good about it is that people who are in the study are allocated randomly into treatment group or control group, meaning there's going to be people getting whatever's happening and there's going to be people who aren't. So you have that comparison.
00:19:31
Speaker
In addition, the people running the trial, running the study are blinded. It just means they don't know whether the person they're working with is getting the trial, getting the treatment, or getting the control. So any bias that they might have unconsciously brought can't be there. All of these things are supposed to help control for any sort of accidental bias that might come in that would change the results. So basically it's just about making everything very
00:20:02
Speaker
random in terms of who gets treatment, who doesn't, not letting people have access to pieces of information that might change the way that they run the study, things like that. And so that's the best possible way to run a study. There are other types of studies, like there's a cohort study, a case control, a case study. Those all may, they may not be bad. They may be actually very well run. But
00:20:27
Speaker
In terms of how much you want to give them value, let's say, they're just not as good as the others. So for your purposes, you want to really keep an eye on systematic review.
00:20:43
Speaker
meta-analysis and randomized control trial.
Critical Analysis of a Fitness Study
00:20:46
Speaker
And you'll know which one it is because it'll say it in the title or in the very early part of the study. So you're not gonna be combing through being like, oh my, what is, I can't, I don't know what this is.
00:21:01
Speaker
Hey guys, it's Sarah. Laurel and I really hope you're enjoying the new Movement Logic podcast. We are having such a good time. We both really love sharing ideas with each other and getting sparked by things that the other person has learned. Our goal for the show was to help you feel the same way so that you can feel excited and inspired by what you're learning and even maybe take some of these ideas into your teaching. That would be amazing if that's what happened, I'd be so happy. Because I know, oh my God, we both know
00:21:30
Speaker
what it feels like to be uninspired to be stuck in a rut desperately trying to come up with new ideas so you take another training and it just ends up you fall back into your old habits the things you already know how to do because it's too hard to change who you are as a teacher we've all been there the whole reason why we created the movement logic tutorials
00:21:51
Speaker
Was so that you can enhance what you're already good at instead of trying to be some other different kind of a teacher every movement logic tutorial contains so much to help you do that hours and hours of anatomy kinesiology Myth-busting myth-busting is maybe my favorite part of the whole thing but most importantly dozens of exercises that help you
00:22:12
Speaker
with strength or flexibility or functional movement, whatever you and your clients want to do in their life. Because we're so grateful that you are listening to our podcast, we have a podcast exclusive discount to say thank you for supporting our efforts with your years. What you can do is you enter the coupon code podcast at checkout to receive 10% off of your entire purchase.
00:22:37
Speaker
You heard that right. You go to movementlogictutorials.com, take a little scroll through all of our different tutorials, stick some of them in your cart, the ones that you're like, ooh, pelvic floor, ooh, shoulders, and then enter the code podcast at checkout and you'll receive 10% off your entire purchase because we appreciate you. So thank you and go forth and save.
00:23:05
Speaker
So second criteria is has anyone else replicated this study? So let's say it's a study that's a randomized control trial and you look at it and you're like, wow, that's really good. The way in the research world, one of the ways that research is found to be valuable, found to be believable and accurate is if somebody else can go in and take your study and do the exact same thing and get the same results.
00:23:36
Speaker
There was a while ago a study about something called power positions that I think a lot of people have probably heard of where it's claimed that if you adopted a position of confidence, power of taking up space before a stressful event like a job interview, that it would boost your confidence, your self-esteem, that it would even improve your memory.
00:24:03
Speaker
your pain tolerance and decrease your fear. And this was that thing of like standing with your feet wide and your hands on your hips or with your arms up and reaching in like sort of a V shape. And it got a lot of immediate attention as well, because it sounded like, well, that seems pretty easy and just go in the bathroom before my interview and put my hands on my hips and take up a lot of space. And then I'm gonna march into this interview and nail it. That sounds great, but
00:24:32
Speaker
nobody was able to replicate the study and get the same results.
00:24:37
Speaker
And in the world of research, that's kind of considered the same as like, you made this up. I'm not trying to make any sort of claims about any of the research that I'm talking about today. I'm just presenting you with my interpretations of information. But it is a really, really big deal if you can't replicate someone's study. There's a great example of a study that has been replicated many, many times.
00:25:05
Speaker
And that is when they do an MRI of sort of randomly collected people who don't have back pain and they do MRIs of their low back and they find a huge number of people have things like a disc herniation and stenosis, disc degeneration, all kinds of degenerative issues, injuries or just sort of repetitive overuse, all that kind of stuff.
00:25:30
Speaker
And none of these people are having any back pain. And so that has really drawn a line through the idea that your back pain is being caused by your disc herniation or your back pain is being caused by your bone spurs or any of those things. And that study has been replicated many, many times. So that is evidence that you can, you should still go read it, but that is evidence that you can say, okay,
00:25:58
Speaker
more than one person has been able to do this and they get the same results. Therefore, this is pointing towards this being really very accurate. And then the third thing is you want to look at the methods that they describe in the study. And I have to say a big thank you to Jules Mitchell. And if you are actually interested in learning more about research, she does a whole section of it in her 300 hour yoga teacher training. And that's something you can go look up on Jules's website.
00:26:26
Speaker
But the methods, so there'll be a section of the study, it's usually pretty early on, just called methods. And methods is supposed to describe how they did the study. So in theory, you should be able to look at the methods and replicate it, right? Because we know now that's one of the ways that we can ensure that the research that we're reading is actually good quality research. But if you read the methods and you can't replicate it, or you read the methods and you're like, this is not going to, there's nothing about this.
00:26:56
Speaker
explains how you are then extrapolating this other piece of information, then that's another really good thing to be able to do. Don't worry about all the statistical analysis and the discussion and all that kind of stuff. Feel free to read it, obviously, but it can get super intense. I still hate statistics. I couldn't stand it at school. I still cannot stand it. But
00:27:21
Speaker
being able to look at the methods is one sort of, it's an easier entry, I think, to understanding research. So what I thought I would do is take the methods from the study about getting down to the floor and back up again without using your hands or your knees or whatever and go through it and just sort of go through it line by line and tell you what the issues are that I see.
00:27:48
Speaker
So in that study, they used a specific test that was already in existence that had been used in other studies. And that's actually a really good thing to do because it just gives you more of that accuracy and authenticity in what you're doing. You're not making up a new trial, a new test that has no backing. So it's a test called the sitting-rising test or short-handed as SRT.
00:28:16
Speaker
So they're using a specific test that's already been used, that's good. We already know that they have filtered out basically the more fit people, which I find problematic. Okay, and so now I'm, this is from, I'm just gonna quote the methods.
00:28:33
Speaker
The SRT assesses the components of musculoskeletal fitness through evaluation of the subject's ability to sit and rise from the floor, assigning a partial score for each of the two required actions. So, okay, already this test is described as assessing musculoskeletal fitness. It's not assessing mortality.
00:29:00
Speaker
So your ability to do the test or not do the test does not automatically reflect mortality. All it reflects is musculoskeletal fitness. So already I'm running into, this is hugely problematic, but let's continue. So I'll keep going.
00:29:19
Speaker
SRT was administered on a non-slippery flat surface in a minimal space of two by two meters with the subject standing barefoot and wearing clothing that did not restrict body movements. Okay, that sounds fine to me. I don't have any problem with that. Before the SRT, the evaluator instructed, without worrying about the speed of movement, try to sit and then rise from the floor using the minimum support that you believe is needed.
00:29:50
Speaker
And I talked about this a little bit earlier, but I would listen to that instruction. I would think, hmm, this floor looks hard. I'm barefoot. I don't think I want to go down hard on a knee. I think I'll put a hand down.
00:30:02
Speaker
And to me, that's the minimum support needed. But if someone told me to try and sit down without using any support, then I would attempt to do that. So what they're claiming is to know why someone is putting a hand or a knee down. That the why of it is directly related to unsteadiness, not related to, I don't know, preference.
00:30:28
Speaker
I would look at that hard floor and be like, why would I try to sit down without putting my hand? Anyway, SRT partial scores begin with a maximum of five points separately for sitting and rising. So getting down to the floor, you get five points, getting up from the floor, you get five points. One point was subtracted for each support utilized, that is hand, forearm, knee, or side of leg.
00:30:53
Speaker
and an additional half point was subtracted if the evaluator perceived an unsteady execution which they describe as partial loss of balance occurring during the action. So evaluator perceiving it
00:31:11
Speaker
is a bit iffy. A, they could miss it. B, they could over interpret a wobble as like, oh, this person lost their balance. Also, partial loss of balance is not full loss of balance. People partially lose their balance all the time and don't fall. So partial loss is actually a recovery, which is a successful not falling event.
00:31:35
Speaker
me partially losing my balance and then recovering my balance while I'm doing something doesn't mean that I am a false risk. And it also, you know, it doesn't, I clearly have a lot of feelings about this, but this idea of like unsteadiness and again, interpreting the meaning behind the unsteadiness or instantly saying, oh, well, that's like, you were going to fall because you got unsteady. I don't, I find that problematic. Okay, here we go. I'm going to keep going. In addition,
00:32:05
Speaker
One point was subtracted if the subject placed one hand on the knee in order to sit or rise. I have mixed feelings about that. Lots of people put their hands on their legs to stand up or to sit down. Honestly, not that they necessarily need to, they just kind of think maybe they need to. Crossing the legs for either sitting or rising from the floor was allowed while the sides of the subject's feet were not used for support.
00:32:32
Speaker
And I just find that confusing. How am I supposed to be sitting cross light on the ground and not have the side of my feet touching and then somehow get on the soles of my feet to stand up? That seems like a lot of, I'm just confused by that. I find that weird. If a five score was not obtained, the evaluator provided some advice that might assist the subject to improve their SRT score in other attempts.
00:33:00
Speaker
If you're watching the video of this, you can see that my eyebrows are essentially at my hairline, and I'll tell you why. At no point in the study do they tell us how many attempts each person got. Just that they take the results, they take the score from their best attempt.
00:33:20
Speaker
So does everybody get the same number of attempts? Is there a, you know, you get three attempts and we take your best one. You get five and we take your best one. Do some people get five and some people go like that already there's so much room for question in there and it's not clearly delineated. And then finally, there is a video and it's on YouTube and I'll put the YouTube link in the notes where you can, where it shows you the SRT test, the performance and the scoring.
00:33:50
Speaker
And the only new piece of information from that video was that the quote unquote evaluator advice was basically, instead of doing that the way you just did it, try it like this instead. So they essentially demonstrate how they want the person to do it. So that's an example of going through the methods with a fine-tooth comb and really just
00:34:18
Speaker
you don't have to have a ton of experience reading research to get the sense that it's a bit fuzzy. You might have thought that some of the things that I was picking apart was being a little too picky, but again, we're assuming you are going to go and try to replicate this study and that would give us proof that it's a good study. You actually can't replicate it because
00:34:44
Speaker
we're missing information about how scores were given and it's just very it's very vague a lot of a lot of what the methods are saying so you wouldn't be able to successfully
00:34:56
Speaker
replicate it. And so already that tells it's not great. But then also the correlation that they're making, the biggest red flag to me is that the test that they use is a measure of musculoskeletal fitness. It's not a measure of mortality. And that they are then extrapolating mortality rates from it, when in fact, mortality rates also correlate with just how old the person was. So there's a big leap
Importance of Critical Assessment
00:35:19
Speaker
in the conclusion that they're drawing in this study. Okay. Hopefully you are still awake and I haven't lost you completely because I want to talk about one more thing. What is all of this matter? If you're sitting and you're listening to this and you're like, I'm still not convinced that I need to be able to do all of this or that it's really important. But if you have the ability to look at a few things in a study,
00:35:44
Speaker
and get a broad sense of the quality of the study, that is going to tell you a lot about how much faith you should put in the results of the study. And then, whether that is something that you should be including in either the way that you teach or the subjects that you talk about with your students, the way that you talk with your peers. If you're in a class and you hear someone reference a study, but you know that you looked at it and you're like, it's really sketch, you could then go up to that person and say, hey,
00:36:14
Speaker
It does sound cool, doesn't it? Have you read the paper? Let me give you this link because actually it doesn't do what it claims it does. And then we are stopping this misinformation campaign one person at a time, right?
00:36:31
Speaker
or maybe if you have a large social media range, you can do it lots of people at a time. If you come across some interesting research or you hear about a lot of people talking about some new study, the most important thing that you can do is not just go, oh, wow, cool, and then quote it in your next class. Keep your critical eye and your critical ear, both of them, and keep them open. If you learn that there are not already
00:36:56
Speaker
multiple other researchers who have replicated that study. If this is a one time and there's 20 people in the study and that's all we have, it is too early to start talking about it like it's facts. And in a situation like that, even the researchers themselves should be saying more studies are needed. Okay.
Conclusion and Listener Engagement
00:37:20
Speaker
I hope that this has been informative and helps you understand the value that research has, but also that it's really important not to just swallow everything you hear and learn and read, hook, line and sinker, just because it's called a paper.
00:37:35
Speaker
A note to you listeners that you can check out the show notes for links to the references that I mentioned in this episode. You can visit the Movement Logic website where we have a mailing list for free content and our all-important sales on tutorials, all that information. You can watch the video of this episode if you want to see what my face looks like while I'm recording. And that is at movementlogictutorials.com forward slash podcast.
00:38:05
Speaker
Thank you so much for joining me today. It really, really helps us if you liked this episode to subscribe on Apple Podcasts, Stitcher, Spotify, or wherever you get your podcasts.
00:38:20
Speaker
and then to review it on Apple Podcasts, et cetera. If you can write us a review, we would be so appreciative. We read them. It makes our little hearts so happy to hear that you are listening and enjoying it. And if there's something that you would like us to talk about on podcasts, stick it in that review and then we'll definitely see it for sure. Join us again next week for more of Enrollment Budget Podcasts and more of our loosely held, yet very strong opinions.