Zencastr
00:00:00
00:00:01
Speed1x
Format
Share
Embed
Report

You've Been Played: The Pros & Pitfalls of Gamification w/ Adrian Hon

Human Restoration Project
Human Restoration Project

105 plays · May 6, 2023

Gamification is a long-standing practice across lesson planning and educational technology, but it doesn’t always work out the way we expect. At the end of the day, if the nature of the task is not interesting, then what we’re creating is more about compliance than engagement. In this session, we will host a conversation on what it means to gamify content, learning, and pedagogy: recognizing potential success while advising for potential pitfalls. This episode is a panel discussion we had with game designer & author, Adrian Hon, on the pros and pitfalls of “gamification” as part of our EduFuturism Learning Series. You can find all of the previous events including Innovative AI Tools for the Classroom…and their Dilemmas and Learning From Video Game Tutorials, as well as register for upcoming events in the series @ humanrestorationproject.org/learning. You can also find this video and others on our YouTube channel by searching for Human Restoration Project.  Guests Adrian Hon is an award-winning video game designer and is the CEO and founder of Six to Start, co-creator of the world’s most successful smartphone fitness game, Zombies, Run! He previously was the director of play, creating alternative reality games, at Mind Candy. Resources * Watch on YouTube [https://www.youtube.com/watch?v=dJpfut98Bc0] * You've Been Played: How Corporations, Governments, and Schools Use Games to Control Us All [https://www.hachettebookgroup.com/titles/adrian-hon/youve-been-played/9781541600171/?lens=basic-books] * Adrian Hon on Twitter [https://twitter.com/adrianhon] * Conference to Restore Humanity! 2023 [https://www.humanrestorationproject.org/conference]

Transcript

Speaker: Conference to Restore Humanity 2023 is an invitation for K-12 and college educators to break the doom loop and build a platform for hopeful, positive action.

Speaker: Our conference is designed around the accessibility, sustainability, and affordability of virtual learning, while engaging participants in a classroom environment that models the same progressive pedagogy we value with students.

Speaker: Instead of long Zoom presentations with a brief Q&A, keynotes are flipped and attendees will have the opportunity for extended conversation with our speakers.

Speaker: Antonia Darter, with 40 years of insight as a scholar, artist, activist, and author of numerous works, including Culture and Power in the Classroom.

Speaker: Cornelius Minor, community-driven Brooklyn educator, co-founder of The Minor Collective, and author of We Got This.

Speaker: Jose Luis Vilson, New York City educator, co-founder, and executive director of EduColor, and author of This Is Not a Test.

Speaker: and Iowa WTF, a coalition of young people fighting discriminatory legislation through advocacy, activism, and civic engagement.

Speaker: And instead of back-to-back online workshops, we are offering asynchronous learning tracks where you can engage with the content and the community at any time on topics like environmental education for social impact, applying game design to education, and anti-racist universal design for learning.

Speaker: This year, we're also featuring daily events from organizations, educators, and activists to build community and sustain practice.

Speaker: The Conference to Restore Humanity runs July 24th through the 27th.

Speaker: And as of recording, early bird tickets are still available.

Speaker: See our website, humanrestorationproject.org, for more information.

Speaker: And let's restore humanity together.

Speaker: Hello and welcome to episode 131 of our podcast at the Human Restoration Project.

Speaker: My name is Nick Covington.

Speaker: This episode is brought to you by our supporters, three of whom are Christina Danielle, Russell Walker, and Laura Henry.

Speaker: Thank you so much for your ongoing support.

Speaker: The trailer you heard in the intro is for our Conference to Restore Humanity, a fully virtual conference that runs July 24th through the 27th.

Speaker: We've got a stellar lineup, so we hope you'll join us.

Speaker: Tickets and info can be found at humanrestorationproject.org slash conference.

Speaker: This episode is a panel discussion we had with game designer and author Adrian Hahn on the pros and pitfalls of gamification as part of our EduFuturism Learning Series.

Speaker: You can find all of the previous events, including innovative AI tools for the classroom and their dilemmas, and learning from video game tutorials, as well as register for upcoming events in the series at humanrestorationproject.org slash learning.

Speaker: You can also find this video and others on our YouTube channel by searching for Human Restoration Project.

Speaker: And with that, I'll hand it over to Chris.

Speaker: Enjoy the episode.

Speaker: Welcome everyone today for our edu futurism learning series, the last one on video game design.

Speaker: We're joined by Adrian Han.

Speaker: Um, Adrian is the recent author of you've been played how corporations, governments, and schools use games to control us all.

Speaker: He is an award-winning video game designer and the CEO and founder of six to start.

Speaker: Uh, he's the co-creator of one of the world's most successful smartphone fitness games there, zombies run.

Speaker: Um, and he was previously the director of play at mind candy, where he created alternative reality.

Speaker: games.

Speaker: You've been played.

Speaker: It's a really fantastic work.

Speaker: It outlines how gamification has often become a form of boring, behaviorist style control.

Speaker: And it recognizes how gamification has essentially made its way into all forms of our life.

Speaker: And we wanted to invite Adrian on the series because gamification is so present in classrooms.

Speaker: That really took hold

Speaker: I'd say like the late 2000s and remain strong now, whether that be adding points and badges to existing lessons or creating apps that track reward or punish students or teachers.

Speaker: So in our conversation today, we'll be having a question driven discussion of what it means to gamify content, learning, pedagogy and everything and anywhere in between.

Speaker: So welcome, Adrian.

Speaker: Adrian, we appreciate you being here.

Speaker: If you want to do like a short intro and we'll go from there.

Speaker: Sure.

Speaker: Well, thanks for having me.

Speaker: It's great to be here.

Speaker: Yeah, you know, like I, you know, usually have a kind of feel about gamification.

Speaker: You know, I first kind of got into this because

Speaker: You know, I've been designing games for a long time.

Speaker: A lot of those games have been what people would call serious games, you know, educational games, games of like a non-entertainment purpose.

Speaker: And those kind of fall within this idea of gamification, which is using ideas from game design for, you know, serious things like education or health or for politics or training or anything like that.

Speaker: And, you know, the term gamification really only came about like about 15 years ago.

Speaker: It's a pretty it's a pretty new word, even though the concepts behind it are very old.

Speaker: I mean, people often tell me as if it's kind of a gotcha.

Speaker: Well, people have been doing this for hundreds of years.

Speaker: It's like I'm aware that people have been doing this for a long time, but there's a reason why.

Speaker: this word gamification really only started taking purchase, you know, 15 years ago, because it started getting much, much more widespread with the spread of the internet, with apps, with Web 2.0 and with smartphones.

Speaker: And I think, you know, when I first came across gamification, I thought it, I didn't love it because it felt, it didn't really feel fun.

Speaker: I think people had the idea that if we add points and badges and leaderboards to activities, then it would be, they would make them fun and engaging.

Speaker: And that did not seem to be true.

Speaker: But I thought it would basically go away of its own accord.

Speaker: Basically, people would try lots of gamification, it would be a big bubble, and then it would go away.

Speaker: And I noticed probably about five years ago that I just started seeing in more and more places.

Speaker: I started reading about gamification being used in Amazon warehouses.

Speaker: I started seeing it being used for controlling Uber drivers.

Speaker: I started seeing health insurance everywhere.

Speaker: Instead of going away, it just seemed to be getting more and more common.

Speaker: And, you know, even though I hated the idea of kind of thinking about it, I wanted to write a book about it because, you know, I wanted to sort of explore how is it being used in practice?

Speaker: Is it good?

Speaker: Does it work?

Speaker: And how widespread it is.

Speaker: And I think people often ask me, what is the most kind of surprising thing, example of gamification that I found?

Speaker: And it has been ClassDojo.

Speaker: It was, I'm not just saying that because I'm talking about education here.

Speaker: It really was.

Speaker: ClassDojo, I think if you're a teacher, I'm sure you've probably heard about ClassDojo or people who use it.

Speaker: It's a classroom management app, behavior management app that basically

Speaker: those two things it's kind of like a private social network you know for for parents teachers and students but it's also a way for teachers to kind of reward or punish students for a range of behaviors for points and I think when I first heard about it I thought this can't actually be true like I don't really believe it really um and then I started reading more and more reports of it I started reading papers on it I was like I

Speaker: I've literally never heard of this and I spend my time making game education.

Speaker: I don't understand like how there's such a disconnect here.

Speaker: I would talk to parents about it.

Speaker: I talk to teachers and they'd say, well, of course, don't you know about Class.jr?

Speaker: I was like, no, I don't know about it.

Speaker: People don't talk about this thing.

Speaker: And so there's a whole section in the book devoted to the cast dojo and is one of the examples that I used and also happens to be one of the things that gets people most vocally responsive, you know I've done interviews about the book and on radio and

Speaker: Most of the calls in are by people, by teachers complaining about Glassdoor.

Speaker: And the other calls are from Uber drivers and Amazon workers.

Speaker: People have direct experience with gamification.

Speaker: So that's kind of my interest there.

Speaker: I think it's really...

Speaker: a little bit shocking.

Speaker: I think it's a little bit sad because I think there are ways to use gamification, to use video games, you know, ideas and games for good in education.

Speaker: But actually in practice, like,

Speaker: If you are a student, you know, the gamification that you're most likely to encounter, you know, at school is not great.

Speaker: And, and, um, you know, speaking as someone from the video games world, I think, uh, we'd like to talk a good game about how games can be good, but actually in practice, I think, I think, um, a lot of stuff that you see is, is not good.

Speaker: So that's my, that's my sort of intro piece, I guess, about gamification.

Speaker: Awesome.

Speaker: And I figure, I mean, at this point, we're just going to open up the floor.

Speaker: We've pre-prepared some questions, but maybe to get us started, I'll ask one.

Speaker: And then from there, feel free just to dive in, either ask questions in the chat or you can unmute and talk that way.

Speaker: But something I wanted to open up with is...

Speaker: Speaking as your experience as someone who developed like one of the arguably early gamified apps that's that's widely successful.

Speaker: And I know you talk about this in the book and what makes that like good game design versus what we're seeing today.

Speaker: What is an example of gamification as good practice.

Speaker: So there's one thing I think all of us are probably familiar with class Dojo and we recognize the issues with that platform.

Speaker: But.

Speaker: There is an argument that perhaps there is a use for gamification in classrooms as well as a good practice.

Speaker: What does that look like to have a game that's actually well gamified that promotes positive behaviors without it turning nefarious or dark or eerie?

Speaker: I mean, I think that the best examples of gamification that I can think of are things that are really specific to a subject rather than like, here's one weird trick that will improve all learning of all kinds.

Speaker: And so one example that I use

Speaker: is this game called Kerbal Space Program, which is a kind of like simulator.

Speaker: You know, you build rockets in this game and you launch them and you try and put these little Kerbal, you know, people into orbit and you can build, you know, space stations and moon bases and so on.

Speaker: I played it and...

Speaker: You know, it's fun, because it's fun just building stuff and seeing rockets blow up, you know, and and like doing silly things.

Speaker: But it's also an extremely accurate physics simulator.

Speaker: What that means is it is able to explain to you a lot of really counterintuitive things about physics and about orbital mechanics, which is to say, okay, well, why is it so hard to get into space?

Speaker: I mean, space is only 100 miles up.

Speaker: You have to go into orbit.

Speaker: You have to go really fast.

Speaker: How do you do plane change maneuvers?

Speaker: How do you...

Speaker: You know, if I'm here and the space station is here, how do I get there?

Speaker: Can I just go and like put my stresses in that direction?

Speaker: Actually, no, you're going to do this other thing.

Speaker: And just by the process of playing that game, you learn about this stuff.

Speaker: And I think that, you know, there are other games like that, you know,

Speaker: like Factorio or puzzle games where they sort of teach you to kind of think in different ways and kind of, you know, there's some games that essentially teach programming, even though they don't really talk about it that way, you know, like how to construct loops and how to construct if statements and things like that.

Speaker: And I think what is distinctive about them, I mentioned that they're specific.

Speaker: They're not trying to be like, well, here's one game that is going to teach you English and French and physics and geography.

Speaker: It's like, no, we can only really do one thing at a time because it's really hard to make anything fun at all.

Speaker: And so it is really hard to make orbital mechanics fun.

Speaker: It's really hard to make programming fun.

Speaker: They mentioned it for one thing.

Speaker: um and i think that's the other thing which is like it i mean it sounds obvious but like the the the mark i think of good gamified product or experience is that people would do it even if they weren't being told to right um and people would pay for it but kids would pay for it you know i see so many examples of gamification out there where they say well it's so popular it's like

Speaker: If you have to give it away a little bit, then maybe people, I'm not saying that you shouldn't give things away.

Speaker: It's more just like, if you're able to sell something to someone as they're able to do for games like Kerbal Space Program, it means that it's of high enough quality and it's fun enough that people are like, no, I want to spend my time on it.

Speaker: And so many gamified experiences I see are pretty shoddily built so that the only way they can actually get people to use them is because they're free.

Speaker: Because if you had to pay for it, people would be like, well, it sucks.

Speaker: I'm not going to use that.

Speaker: So those are kind of two quick examples.

Speaker: If I could, I'm glad that we started with the positive examples of gamified experiences because I fear that the vast majority of them are not necessarily that.

Speaker: And you had mentioned in that introduction, the rise of social gamification, really kind of coming up with the age of smartphones, the ability to track data from your pocket more than we had the capacity to before.

Speaker: But I wonder if the user experience

Speaker: of it tends to be framed negatively.

Speaker: So either through Class Dojo or through Uber, Amazon, et cetera, what is like the inherent promise of gamification?

Speaker: Who is it intended for and why did it appear to fail in these big examples in which maybe it didn't live up to its promise?

Speaker: I mean, I think that's a good question.

Speaker: I think there's lots of, you know,

Speaker: you could sort of classify gamification by, in a way, who's paying and who's benefiting for it.

Speaker: And so if you look at something like your Apple Watch, you're kind of paying for it.

Speaker: Apple doesn't really get any money if you use the gamified features on your Apple Watch, like scores and achievements.

Speaker: I mean, they kind of like it because maybe you'll buy another Apple Watch, but it's not really that direct.

Speaker: And so I think in that case, you know, things where you're paying for it and you're kind of choosing it, I would say, you know, those,

Speaker: In those cases, people like them or people use the gamification because they think it will motivate them and reward them to do something they already want to do.

Speaker: So an example would be Duolingo.

Speaker: No one is making you use Duolingo.

Speaker: And so they see the gamification, which is very clearly marketed in Duolingo, and they think, well,

Speaker: I find it hard to learn French and to sort of, you know, read vocabulary books.

Speaker: And so maybe, you know, turning it into a game will work.

Speaker: That doesn't mean it does work.

Speaker: Right.

Speaker: I think Duolingo is actually extremely problematic.

Speaker: But I think that the idea of that is like, well, OK, it's all right.

Speaker: And you are buying into that.

Speaker: And so that is very different, however, from something like Uber or working for Uber, working for Amazon, where

Speaker: if you work for Uber, you don't really have any choice but to add your working conditions gamified.

Speaker: But if you're working for Uber, you will be offered quests, right, to do 50 trips this week.

Speaker: And if you do that, then you'll earn an extra $50.

Speaker: And you'll think, wow, great, you know, extra money for me.

Speaker: I said, well...

Speaker: Not really.

Speaker: I mean, like, they could just give you the money anyway.

Speaker: You know, it shouldn't have to be contingent on hitting this target.

Speaker: Right.

Speaker: And so even though it's framed as a bonus, it's not really a bonus.

Speaker: It's kind of more, you know, because your pay is like not particularly high, it's more like withholding money.

Speaker: I mean, certainly if you look at it that way.

Speaker: And so I think the problem there is that basically the gamification is obfuscating your pay and your kind of working conditions, you know, and it's also degrading your working conditions.

Speaker: And so those are kind of two different things.

Speaker: You know, I call it in the book coercive gamification.

Speaker: You know, these are these are forms of gamification where you go into work and you

Speaker: hey, now you have to play this game, whether you like it or not.

Speaker: And then you have what I call lifestyle gamification, which is like Duolingo, which is the Apple Watch and that sort of thing, where it's probably better, like no one's forcing you to do it.

Speaker: But in practice, I think it doesn't actually, usually it doesn't actually work that well, usually it's kind of oversold.

Speaker: And often, actually in the case of Duolingo, it ends up being, I think, kind of like quite antisocial and counterproductive because

Speaker: just a short aside on Duolingo, if we've got time, my partner's mum uses Duolingo and we notice that occasionally she just disappear off into another room, like in the middle of a conversation or dinner for like an hour.

Speaker: and then there'd be like Duolingo sounds coming from the other room and then it would turn out that she got a notification about XP happy hour from her Duolingo app and so basically the idea is that for that hour you get double experience points for the Duolingo lessons you complete.

Speaker: And it's usually in the evenings.

Speaker: And it's just, I think that's terrible.

Speaker: I think it's such a terrible thing.

Speaker: Like, I don't think anyone who signs up to do it and then goes thinking, I want to go and learn Mandarin.

Speaker: thinks I also want to be interrupted during my dinner and have my kind of value system like manipulate to the point where I'm going to like just depart because now like I value the XP more than I value like actually learning you know this language but that's what they've been able to do and

Speaker: Do Lingo is fascinating because the designers will spend a lot of time actually on social media, very openly saying, here's how we maximize engagement by doing all these tricks.

Speaker: I'm like, you're just saying it in the open and people don't seem to mind.

Speaker: It's very strange.

Speaker: So that's a bit rambly, but those are the distinctions I've made there.

Speaker: Yeah, as like a really early version of that, I think about in the 90s and maybe the 80s, the

Speaker: like library book reading programs, where the second that a kid is introduced to myself included, the second that you learn about those programs is like, well, how many books can I quickly get through in order to earn that pizza ASAP?

Speaker: Because that's really all that matters at the end of the day.

Speaker: The book doesn't matter anymore.

Speaker: And yeah, it's interesting to note how similar that is.

Speaker: And I want to make sure too that we hit on

Speaker: kind of the origins of this.

Speaker: You talk about the quantified self movement, which I wasn't familiar with that at all.

Speaker: But the more and more I looked into that, like the more and more I was like, man, this is really similar to how schools function in terms of assessment and testing.

Speaker: You quote in there, one of the originators says, unless something can be measured, it cannot be improved.

Speaker: So we are on a quest to collect as many personal tools that will assist us in quantifiable measurement of ourselves.

Speaker: Can you just talk a bit about that?

Speaker: Yeah.

Speaker: So the quantified self movement is a very kind of Californian thing, which more or less came about when we had the technology to track elements of human behavior and human health automatically.

Speaker: And so, you know, a good example would be, okay, maybe we had pedometers like in the past I had, I was like a really nerdy kid and I had a pedometer for some reason, but like, you know, you get a Fitbit, right?

Speaker: And a Fitbit, you know, can measure your steps and your distance and it can sync it to a computer.

Speaker: And so the quantified self movement was, okay, let's try and hack together devices and software to do like a homegrown Fitbit before Fitbit exists.

Speaker: And so someone who's really into quantified self would be weighing themselves every day and putting in Excel.

Speaker: They'd be measuring their steps.

Speaker: They'd be just measuring everything they can.

Speaker: And it was possible to do that because the electronics were starting to come out and starting to get cheaper.

Speaker: And it sort of married with this very, you know,

Speaker: this sort of like feeling in the 2000s, this kind of supremacy, I guess, of, you know, science.

Speaker: And I mean, I say that as a scientist, former scientist, you know, like this kind of supremacy of like objectivism and like, okay, the only thing worth kind of studying are things that we can assign numbers to.

Speaker: And I think when it started, it was, you know, really exciting, I think, for the people who are really involved, because maybe they've had bad experiences with the traditional healthcare system, you know, they go to their doctor, and they'd say, I have this particular problem.

Speaker: And the doctor would be like, well, you know, like, I don't have time to talk to you, like, it's probably all in your mind or whatever.

Speaker: And so quantified self, basically, from this, it would let people sort of take control of, you know, the information about their bodies.

Speaker: And then,

Speaker: sort of run experiments on themselves, you know, and see the effects.

Speaker: So they might say, okay, now I'm going to stop eating wheat.

Speaker: What does that do to my energy levels?

Speaker: Now I'm going to start doing this.

Speaker: What does it do to that?

Speaker: Obviously not very scientific in some ways, but like I get the appeal, which is like, well, I want to, like, I don't see why I can't have information about myself.

Speaker: And I think where...

Speaker: But as with a lot of things with good intentions, it ended up becoming extremely financialized where now the quantified self movement kind of doesn't really exist because it just got built into Apple Watches and wearables.

Speaker: and the data is a bit harder to sort of get hold of and it's all become monetized.

Speaker: But the other thing is that, you know, it's one thing to have the data about yourself and to see that you're walking 4,000 steps, you know, a day and maybe you want to get to 10,000 steps a day, right?

Speaker: And so what people,

Speaker: kind of all immediately landed upon as a mechanism to motivate themselves to change their behavior was gamification.

Speaker: They're like, well, okay, the only thing I can think of that I can use to make myself change my behavior is gamification.

Speaker: But I will, of course, I will give myself stars.

Speaker: Of course, I will give myself points.

Speaker: You know, I'll give myself achievements and needables, right?

Speaker: And...

Speaker: I just found that kind of really fascinating that like, you know, it was just like, well, that's obviously the only thing I'm going to do.

Speaker: And even today, if you go and look at people talking about occasionally, if you go and look on like hacker forums and technology forums, you'll see these like end of year posts by people who like measured 50 million things about their bodies.

Speaker: And invariably they'll say, well, I gave myself 50 points for doing this and I gave myself minus 20 points for doing these things that I didn't like.

Speaker: And so that's how I kind of, that's how I motivated myself to change my behavior.

Speaker: And I think those, you know, those sorts of behavioral change mechanisms ended up

Speaker: in consumer products being used by hundreds of millions of people.

Speaker: Because the people who design Quantify Itself and who use Quantify Itself back in the early 2000s

Speaker: are the exact same people or friends with the people who designed the Apple Watch.

Speaker: You know, they all, I mean, I'm not like suggesting any conspiracy here.

Speaker: It's just, they all live in Silicon Valley.

Speaker: Like, you know, they go to the same conferences.

Speaker: Like if you're a designer at Apple, you don't get ideas from nowhere.

Speaker: You know, you probably read about it somewhere.

Speaker: And so that's, these are the ideas that were sort of floating around.

Speaker: This is a quick follow up as you're explaining this.

Speaker: It makes me think a lot about digital LMSs or learning management systems in school and how.

Speaker: Yes, grades and tests and assignments have always been like handed back to kids and made them feel a certain way.

Speaker: But now we're at the point where you instantaneously know when that assignment's been graded, you get a notification on your phone and you check it ASAP.

Speaker: And it completely changes how a kid feels about themselves and their emotions in real time to the point where you have...

Speaker: family members and students complaining that like 4 p.m.

Speaker: like they just got home or maybe they're eating dinner with their family and now they know that like you know Mr. Smith gave them an F on an assignment and there's like a

Speaker: a feeling of control and dominance there on like how much academia has become part of just like the the daily life of as like as a kid and the exact same way that class dojo like sets norms and cultural norms through discipline or as adults like you're constantly assessed on your performance at your job or even like email something as simple as that like responses yeah

Speaker: I think that's really important because, you know, whenever I talk about the subject, you know, especially when I talk about class dojo or about, you know, people getting, having their workplace gamified, you know, is some smart ask will always come along and say, well, haven't we always done that?

Speaker: You know, didn't we always get grades?

Speaker: It's like, there's a big difference between getting a grade at the end of a semester or the end of the week or the end of the month.

Speaker: and getting it microgrades every 10 seconds.

Speaker: And if you can't see that, then you're sort of being a little bit willfully accused.

Speaker: Because I think I would sort of tie it back to this sort of idea of quote unquote play in a sort of game design sense.

Speaker: You know, people talk about fun and play coming from being able to have a sense of freedom within a sense of constraints, right?

Speaker: Like, you know, school is going to have some constraints, a workplace is going to have some constraints, you know, in terms of rules and things like that.

Speaker: But otherwise, it'd just be complete chaos.

Speaker: But within that, you ideally want to give people some freedom in how they approach their work and how they do their job, unless you want to treat them like robots.

Speaker: And I think the process of grading people continuously and then seeing that feedback

Speaker: eliminates that ability to play because there's essentially much more sort of a close, you know, finally grain control being put over their behaviour.

Speaker: And it, yeah, you know, I mean, I think it's bad.

Speaker: I think it's very different from, I think it's a very different experience than from getting a grade more infrequently, even though it is still getting grades, kind of.

Speaker: Yeah.

Speaker: And that is a technological part of it.

Speaker: You cannot do that kind of fine-grained feedback where people will get grades at 4pm or whatever without having the internet, without having smartphones, without having AI or things like that.

Speaker: It's just not really possible to do that at scale unless you have technology.

Speaker: So that's why it is different.

Speaker: I wonder if there is, since you were talking about the difference between gamification in the game design sense, you know, we think about mechanics, we think about events, we think about those kinds of things versus the vision of gamification that you're addressing with all these things, which seems to be like the Skinner box aspect of things, right, involved in behaviorism and reinforcement.

Speaker: But I wonder, have you thought, looked, or do we have any, I hate to ask about the data around this, but like about the cumulative effects, right?

Speaker: The macro effects of living a life in a society with government agencies, you know, for-profit companies who have all kind of bought into this gamification.

Speaker: We're all on the receiving end of this.

Speaker: What are those big macro level impacts?

Speaker: So, no, is the answer.

Speaker: I don't think we have a lot of data about that.

Speaker: And partly that's just because, you know, the research is not funded into this, you know, and it should be.

Speaker: You know, I was astonished at how little research there is into class dojo, given how pervasive it is.

Speaker: There's very few papers, really, about it.

Speaker: And, you know, the other problem with gamification of any kind of fast moving technology is that as soon as you study it, it's like out of date, you know, and God knows, you know, it takes like three years to get a paper published or, you know, to do a study.

Speaker: And so everything I'm reading is completely out of date.

Speaker: And then you sort of put on top of that the burden, you know, people obviously want really accurate studies.

Speaker: They want double blind studies.

Speaker: It just takes forever to sort of run those.

Speaker: So, but then I would sort of pull back one step and I would say, okay, so imagine we have a study that said, actually, if you go and monitor kids every like five seconds and give them like a grade into their augmented reality glasses, their sort of performance goes up 5%.

Speaker: Would we say, well, OK, that's great.

Speaker: Actually, we should do that because the data says yes.

Speaker: Or would we go and say, actually, no, that's sort of against our values.

Speaker: And I think that those are kind of two different things.

Speaker: It's like you sort of have to decide,

Speaker: Will we do anything, you know, to to increase, you know, students performance at standardized testing?

Speaker: Or are there certain things where we're like, actually, like our values are sort of against that.

Speaker: We don't think that we should be putting that on kids, even if on average, it improves, you know, their performance.

Speaker: And so my argument would be.

Speaker: Michael Boucher- Like firstly we don't have the data, but even if we have the data I don't think that is actually how we think people should be learning like you know.

Speaker: Michael Boucher- And so that is a harder stance to take because because then you're kind of saying well no like I think there are certain things which we which i'm not prepared to do, even if even if the data points in that direction.

Speaker: Yeah.

Speaker: And I would imagine to just interestingly enough, even if the data pointed the opposite way, that because there's a financial incentive in doing this type of work, like why would I follow it anyway?

Speaker: In the exact same way that I mean, we have 99.9% of climate change papers point one way, but we still don't actively act on those things because there's a financial incentive at a macro level to not follow those things, at least not according to the companies that are involved.

Speaker: Right.

Speaker: Like there's a there's a huge monetary component there.

Speaker: Yeah, I mean, I don't want to be, I definitely don't want to be like a conspiracy theorist about it at all.

Speaker: Like I used to be a scientist, you know, I think data is important, but I think that the, the, um,

Speaker: the role in which sort of like research plays, and usually it's not very powerful research in terms of like it's sort of explanatory power, statistical power, you know, is usually, I mean, the way I've seen it in my book, you know, where I look at companies who are selling lifestyle gamification, like brain training apps, like education apps, they'll go and say, well, if you go and use our brain training app, you're going to increase your, you know, maths ability by like,

Speaker: And then you go and look at the paper and like this paper sucks.

Speaker: It's just terrible paper.

Speaker: You know, you're sort of misrepresenting the benefits, but, but you're kind of saying that the data points in that direction.

Speaker: And like you clearly do not actually, you're not interested in what the data says.

Speaker: It's just a marketing technique.

Speaker: Right.

Speaker: And if there were,

Speaker: data that pointed in the opposite direction, you wouldn't share with us, right?

Speaker: So, I mean, that's kind of where we are, where like, I kind of feel like there is this burden of proof being placed on things where it's like, well, you know, heads I win tells you lose.

Speaker: But it's like, well, if I have a positive study, I'm going to use that.

Speaker: But if that's a negative study, I'm just going to ignore it.

Speaker: And so, like, we kind of have to get around this idea that like, well, we can only act if the data says so, because like, it's going to be very difficult to do that.

Speaker: Sure, that makes sense.

Speaker: Let me shift gears here to read next question here in the chat.

Speaker: He asks, Is there a different vocabulary we can use to better differentiate the types of gamification that are coercive or extrinsically incentivizing from ones like Kerbal Space Program and simulations narrative driven games?

Speaker: that are more intrinsic in motivating the player?

Speaker: Like as in gamification is gimmicks and carrot on a stick stuff, whereas the new word would be a more ludic, playful human.

Speaker: So changing up the terminology.

Speaker: Well, I mean, you know, we can.

Speaker: I think that there are, you know, I think people have used different words.

Speaker: You know, I've tried to classify it as

Speaker: I mean, I guess in my book, I just classify as bad gamification as good gamification, honestly, if I'm being direct about it.

Speaker: But that would be the same as, can we use a word for good games and bad games?

Speaker: We just say they're good or bad.

Speaker: I think, I mean, I guess I would go and say that...

Speaker: I call it generic gamification, you know, in my book, if I'm being specific, which is like when you have something like Class Dojo or a lot of other

Speaker: apps where it's basically saying I'm going to give you points for good things you do and minus points for bad things you do and I'm going to give you achievements for good things you do and not achievements for bad things you do and I'm going to go and rank you on a leaderboard.

Speaker: I call that generic gamification and that's not necessarily a judgment that it's bad it's just like well you're just using the same thing and so

Speaker: David Olusogaertsson- You know, I think it's possible to more easily dismiss generic generic gamification because it's like well it's probably not going to really do anything for most people might do something for some people, some of the time, some people do actually respond to that but.

Speaker: David Olusogaertsson- I it's generic and therefore it's only good to have a limited amount of power compared to more specific forms like Kerbal Space Program like other things and unfortunately.

Speaker: A lot of companies are just really cheap companies.

Speaker: A lot of organizations are just cheap and they would rather buy one thing rather than buying something weird.

Speaker: I mean, a global space program is weird.

Speaker: I can see why schools would be like, I don't know why we should do anything for this.

Speaker: Can't we just go and get something that will work for everything?

Speaker: And this sort of goes back to, I mean, I'm not like an educator, but I know that people are different and they like different things.

Speaker: So I go and look at our own game, Zombies Run, which tries to make running more exciting.

Speaker: I'm, I didn't imagine that this is going to work for everyone who's a runner, even, you know, because a lot of people don't like zombies, right?

Speaker: Or a lot of people don't like listening to stories, or they don't, you know, like when they're running, or they don't like, you know, games or things like that.

Speaker: And so, you know, like,

Speaker: I think that there is this search for kind of like a silver bullet, you know, in terms of gamification solutions, when actually you can't get that because, I mean, I don't want to talk about learning styles.

Speaker: It's just more like some people like some things and other people like other things.

Speaker: And you will find, we found that Zombies Run has worked incredibly well for some people to the point where it's really transformed their lives and we've had 10 million people download it.

Speaker: But other people, they're like, yeah, I don't want to use that.

Speaker: And I prefer doing Park Run or I prefer using Garmin watches.

Speaker: And that's great.

Speaker: And so it's just like we can't do a one-size-fits-all.

Speaker: What we need there to be is like a thousand games, right?

Speaker: And help make it possible for people to go find the ones that are going to work for them.

Speaker: And, you know, that's where we need to be.

Speaker: instead we've kind of got like one game or zero games and we're just hoping that it's going to work for everything and so that's kind of that sort of feeds into my frustration with the research angle where people will go and say well we've like studied gamification to see whether it like it helps at this you know and it's like I just don't

Speaker: You know, we sort of tested zombies run on university students to see whether it made them run more.

Speaker: And like, yeah, but I don't want to force zombies run on people.

Speaker: Like, people should just choose whether to play it or not.

Speaker: And I sort of trust them to kind of know whether they're going to like it.

Speaker: And of course, you know, that's a problem in schools where it's like, well, we just more or less like force people to kind of do something.

Speaker: And yeah, you kind of want them want to help them discover, you know, things they might not might like that they might not know about.

Speaker: But yeah, I just wish there were like more options.

Speaker: That is the fascinating thing is that we've, in education, and I'm trying to pull in some of the pieces into private sector stuff too, but in education, we go immediately from the research about what works.

Speaker: And you had just said what works with the game that you've created might not be the game for everybody, but we've decided systemically that say something like Class Dojo must be the thing that works for all kids or all classrooms or those places in which

Speaker: in which they are adopted.

Speaker: So it really is that interesting move from like, here's what we have the data around, here's what we're going to assume what works, and then here's how we're going to, like it really gets quickly into that course of space, that course of gamification that you were talking about there earlier, that in order to achieve what works, it requires students to

Speaker: Uber drivers, Amazon employees, et cetera, to participate in these coercive structures.

Speaker: You know, there's not a lot of data, I don't know, that I've seen, a research that I've seen that says, like, here's what works is, you know, I don't know the bits about agency or autonomy or anything else, because then you're responding to

Speaker: dozens, like an infinite amount of ends.

Speaker: It's all about aligning means and ends through these sequences of coercion.

Speaker: And perhaps I'm just, I'm doing one of those things in a talk where I'm thinking out loud and trying to make connections, but that is sort of a connection I think I'm making on my end there too, is the

Speaker: that part about gamification or that part about coercion and gamification kind of go hand in hand, because as soon as you introduce the autonomy and agency components, I don't know, then people can start to make decisions about what it is that they want to pursue, what it is that they want to get done in there.

Speaker: Right.

Speaker: And it's so, you know, the difficulty with, you know, trying to make these decisions, you know, on a kind of mass level is that

Speaker: you end up kind of watering down the sort of effects that things can have.

Speaker: Like you can imagine that there's a game that will help kids get more excited about English literature or about learning French or something, but it only works on 0.5% of kids, right?

Speaker: And so like, how are you really gonna like figure that out?

Speaker: I just don't, you know, it's gonna be really difficult if you went and tested it on everyone, it'd be like, well, it does nothing at all basically.

Speaker: Like it's not statistically significant.

Speaker: But if you, if those kids were able to find that game and use it, then it might be like genuinely transformative.

Speaker: I remember, you know, I studied, you know, I learned violin when I was growing up.

Speaker: And it was just horrible, horrible experience because like, it's like the violin just sounds terrible for the first five years you do it.

Speaker: And I remember coming up with an idea for a game.

Speaker: I was like, I think I could design a game called Violin Hero that would like make, that would like make learning the violin like way more fun.

Speaker: And, you know, the issue is that it would be difficult to sort of like go and like how do you make money from it?

Speaker: Like it would be hard to sort of find the people that would like it.

Speaker: But I think for the people who do like it, it would be transformative.

Speaker: And so it's, you know, all of this comes down to a question of scale, which is just like we're trying to do things at massive scale because it's easier and cheaper than trying to be specific to, you know, different people, to different subjects, you know, to, you know, different,

Speaker: learning styles or whatever we call it now and so I think that that

Speaker: know the good news is that people really like making games i mean you know like that you know game designers love making games and and they often do it for like you know for the love of it rather than for money and so you know i i would you know like i i see like a great future you know within grasp where you know we we go and you know i don't know how this would work but we go inside the

Speaker: we would like there to exist games that would sort of help people get excited about certain educational things.

Speaker: And we want to sort of help those things find an audience.

Speaker: And there are some games like that.

Speaker: There are games that are better than Duolingo or more interesting Duolingo.

Speaker: like massively multiplayer games or 3D immersive games where you learn, like, you get sort of dropped into some mystery, you know, environment and they're only speaking in French and you have to sort of go and figure things out in that way.

Speaker: That sounds great, you know.

Speaker: But there's kind of a mismatch because, you know, the kids certainly don't have the money to go and pay for these games, you know.

Speaker: And God knows, like,

Speaker: trying to sell games to schools is impossible.

Speaker: I remember we made these health and fitness games and people would always say, why don't you go and talk to the NHS, the National Health Service in the UK and try and get these games in front of people who could help

Speaker: who they could help.

Speaker: I'm not even going to explore that because that would be a good idea, but there's no way I can convince the NHS to give us a time of day.

Speaker: And that's partly because of scale and partly because of just cultural issues where I think they think games are a bit silly and probably they would think a zombie game is just absolutely insane.

Speaker: And that would change eventually, but I just wish it would happen faster.

Speaker: Are there areas of policy that you see being influenced by, you know, my earlier question about macro effects, do you see government officials in different places, the NHS, elsewhere, actually, like, buying into the gamified parts and implementing them into policy writ large?

Speaker: I mean, you know, I used to, I mean, I sort of searched for like gamification, you know, on Google News, you know, all the time.

Speaker: And like, you know, you will see, like, I think there's like a UK local authority that was trying to use gamification to encourage people to, for example, you know, use public transport or walk, you know, instead of driving.

Speaker: But then I looked into it and it was like another generic gamification thing.

Speaker: But I was like, well, 10 points if you walk today.

Speaker: I was like, this sucks, you know, put it in the bin, you know.

Speaker: And so...

Speaker: That is where they are, which is very kind of nudge-based, you know, quantified experience, where, again, it's an issue of scale where if you're a local council or local government, you know, like, well, we want to, like, achieve this thing.

Speaker: what is a piece of software which we can deploy, you know, quickly that costs like £20,000 that, you know, may or may not work, but we hope it works.

Speaker: I'm sure they hope it works, but, you know, involves, you know, that can be used by literally everyone and involves no change of any other services whatsoever, you know, basically a silver bullet.

Speaker: And so,

Speaker: I don't see, I don't actually see any change happening there at the moment.

Speaker: I think that the change will come.

Speaker: generationally, you know, when you have kind of like policymakers, you know, who, and legislators who have like grown up with games.

Speaker: And to some extent they have, but also I kind of think, unfortunately, that a lot of policymakers and legislators are just really boring people or really busy people, and they've never really like actually played games.

Speaker: And so they just, they've never really kind of connected that, you know, the dots there.

Speaker: um like i look the book to try and get people to think about this and to you know and to think about the harms of gamification but also the positive aspects and people are you know like people some people are thinking about it it is happening but it's just happening um you know very slowly um and i think that it's it's

Speaker: If I was like a different person or we, you know, I was like more motivated by making money a different way, I guess I would be sort of like paying some pressure groups to sort of go promote zombies rather than our fitness products as I guess a local government intervention.

Speaker: And then I would be like, oh, can I go and get like 50 million pounds or $50 million from the US government to like do an experiment?

Speaker: Like I could do that.

Speaker: I just find that like soul destroying basically.

Speaker: So I don't do that.

Speaker: I'd rather just go and sell it directly to people.

Speaker: even though, you know, I sort of agree that it would be nice if, you know, there were sort of more, you know, you know, government to support it.

Speaker: Right.

Speaker: There's, there's two things that you're bringing up and they're, they're very different from each other, but,

Speaker: It's interesting to note, I think about how the EU has relatively recently implemented those regulations on games regarding gambling and loot boxes and how we know that like gambling and loot boxes are a really big deal to young people you have kids that are becoming addicted to these things they spend their parents credit card and now.

Speaker: It's it's fairly difficult for game creators that operate in those countries and even now in the United States to do those things because they they want to be able to sell their product and the form of regulation actually is working, even though they are always trying to find different ways to get around it.

Speaker: That's the first thing.

Speaker: I think highly relates to all this is I was just recently at ostensibly an ed tech conference like yesterday.

Speaker: And I saw Thomas linked in the chat there, one of these softwares that are often sold.

Speaker: I've never seen PBIS rewards, but it's like they're always the same story.

Speaker: You assign kids a number, you scan them, you say if they're doing something right or wrong, or you track them, you surveil them.

Speaker: Right now, what's really hot are the

Speaker: like student tracking apps in the hallway so that you can prevent meetups and so kids like can't go to the restroom for too long.

Speaker: And it's really creepy, honestly, when you start really thinking about it.

Speaker: I have a list of bad gamification.

Speaker: And so like, thank you for that.

Speaker: I'm putting it in my list.

Speaker: And what's interesting to me is one, obviously, like the EdTech version of this is very Black Mirror-ish.

Speaker: EdTech has historically been, and I'm sure you're probably familiar with like Audrey Waters' work with Hack Education.

Speaker: EdTech has always historically been like some of the worst possible examples of Black Mirror S technology because they all are designed through a tool of compliance.

Speaker: So that's kind of like the first thread.

Speaker: But the second thread, I don't have a chance if we'll have a chance to get to it, but we're always interested with our org talking about beyond ag tech.

Speaker: Like what does gamification look like when it comes to a meta component of school, for example, grades and testing?

Speaker: Because there's an element there of you call the scalar effect.

Speaker: Is that what it's called?

Speaker: Where you're taking a very complicated issue and simplifying it.

Speaker: Yeah.

Speaker: For example, like wine tasters, where they'll give it a grade or a score, but realistically, it's very hard to distill something that's that complicated down to that.

Speaker: And at the end of the day, it means nothing.

Speaker: So to rephrase that, first thread is like what this looks like in ed tech and tracking and behaviorism and all that kind of stuff, like a virtual tailors and digital tailors and second thread is like.

Speaker: What are your thoughts about school in general?

Speaker: I don't know if you have kids, but like, just like the idea of grades and testing and how that relates.

Speaker: Yeah.

Speaker: So, I mean, you know, the first red, I think with ed tech, you know, ed tech is the same as tech, you know, which is basically, it's like people trying to create a system, you know, the, the, um,

Speaker: It's easier to make the design a technological system that treats everyone the same and just throws people into a database and doesn't cater for edge cases or things like that.

Speaker: And so the same sort of issues that you see with technology that tries to sort of monitor employees, you know, that tries to maximize certain outputs or number is the same problem you see with edtech, which is just like, and the reasons why it gets adopted are the same, which is just like, well, you know, institutions would prefer just to like have one piece of software, you know, to use, even if it's not very good because it's just easier to do it that way and to administer.

Speaker: And so I think that one can hope for better versions of EdTech, you know, and I think that certainly one can hope for EdTech that is, you know, one thing I sort of talk about is, at the very least, if a school is going to adopt ClassDojo, can we not come up with an open source version of ClassDojo that schools can, like, customize?

Speaker: You know, we should do that, right?

Speaker: And I've seen this technology.

Speaker: None of it is really that hard to make.

Speaker: Like, go and throw like $50 million at some team of crack government programmers, and they will be able to replicate and improve on all the stuff open source.

Speaker: Like, they would love to do it.

Speaker: Like, programmers love to do, you know, would love to sort of do something that is good for people.

Speaker: Just go and pay them a decent wage, and then we can wipe out all these companies.

Speaker: It'd be amazing.

Speaker: The...

Speaker: second thread about I don't have kids, you know, but I think that, you know, we probably all know about Alfie Cohen, you know, and Punished by Rewards and stuff.

Speaker: And like I found his book, you know, arguing against, you know, grades and scores and values incredibly persuasive.

Speaker: I thought it was a little bit

Speaker: I sort of, I admire the fact that he's kind of so, I was going to say dogmatic, I think that's unfair.

Speaker: He's just so, he's not willing to kind of give an inch.

Speaker: He's like, I don't think you should go and praise people at all.

Speaker: Like, stop praising people.

Speaker: I was like, I don't know, man.

Speaker: I praise people, but maybe I should stop doing that.

Speaker: I sort of admire that way.

Speaker: He's like, well, he knows what he thinks, that's for sure.

Speaker: I...

Speaker: You know, like I think that, you know, yeah, I think again, grades are the product of the system, which is just trying to, I'm sympathetic with it because it's just like, well, we're just trying to like figure out a way I'm dealing with all these students and we don't have enough resources and we're just trying to like identify people who need more help, right?

Speaker: That is what, you know, like I think if you like go, if you give them like the most sympathetic, you know, read of grades, it's like, well,

Speaker: We don't need to use grade to go and screen people and different things, but if someone gets an F, then we know who needs more help.

Speaker: But then I think there are ways of achieving this all without...

Speaker: without using grades, it just costs more money.

Speaker: And so, you know, it sort of comes down to that.

Speaker: If teachers had more time and had more money, then they wouldn't do it this way.

Speaker: They can give more personalized feedback, which would better.

Speaker: I think we all sort of understand that, you know, but it's just like we're in a situation where people or governments sort of don't value, you know, that kind of a...

Speaker: you know, they don't believe that it's going to have benefits, which I think is a shame, you know, that, you know, like, so, you know, that's, you know, I run a games company and we have like 30 people and we hire a lot of people.

Speaker: And I see a lot of, I see a lot of, you know, CEOs,

Speaker: complaining about like how difficult it is to go and hire people.

Speaker: And they say, oh, we, you know, we advertise this job position.

Speaker: I promise this is relevant.

Speaker: And we get like all these, you know, CDs in and job applications in.

Speaker: And I just don't have time to, I don't have time to go and to go and read them.

Speaker: So I just go and like see which university they went to.

Speaker: I just go and apply this automatic grading procedure to the job applications.

Speaker: And so that is how I deal with it.

Speaker: And I just, I look at that, I think, I just think you're not doing your job.

Speaker: Like, you know, I think you're just being really lazy.

Speaker: Like, you know, and when we have people who apply to our company, we read through every job application.

Speaker: Like, it's not that hard.

Speaker: Like, you know, that's literally what we're here for.

Speaker: And it allows us to identify people who, you know,

Speaker: David Olusogaertsson, You know, maybe would not pass through like an automated you know grading process right who maybe didn't go to the right university we didn't have the right grades or things like that, and I don't know how we would be able to go and formalize like our.

Speaker: are criteria for that, because in some ways, you know when you see it, when you kind of see someone who's kind of interesting.

Speaker: And so I think that this sort of like reliance on grades and is a product of just people being like, I don't know, I just don't want to have to like make the decision myself.

Speaker: I'd rather like someone else do that for me faster.

Speaker: Yeah, I think that there's there's so many different threads to explore.

Speaker: And I, I really honestly think that this book is is influential for educators in the exact same way that like Alfie Cohn's work is or many of the other folks that are talking about the ways at which

Speaker: Well, corporate structures and schooling structures are very similar in terms of like their culture and ways that they interact with students and with employees, but also just the concepts of extrinsic and interest and motivation, ed tech.

Speaker: I mean, there's a lot of different themes to explore that I think are very powerful.

Speaker: So, Adrian, again, I appreciate you being here.

Speaker: All of us, I think, are learning a lot.

Speaker: Your book, again, is You've Been Played, How Corporations, Governments, and Schools Use Games to Control Us All.

Speaker: Definitely check that out.

Speaker: Anything else that we should know that's coming up, Adrian, that folks should check out?

Speaker: I just started a new newsletter because I have too much spare time, apparently, where I am trying to... It's at adriinhong.substack.com.

Speaker: And basically, I am trying to talk about games and essentially review games in a kind of more critical lens, but in a way that is not really boring to read.

Speaker: And because I think that...

Speaker: that games are really important, that they're probably this century's most important form of art.

Speaker: And I wish people would take it more seriously.

Speaker: I wish gamers would take it more seriously.

Speaker: I want to take it more seriously.

Speaker: And so if you're interested in not educational games, I mean, although some of the ones I will talk about are kind of educational and like, okay,

Speaker: The way I would sell it to people is like okay you probably don't have enough time to play all these games, but you probably interested in know what to think about them.

Speaker: Well, this is this is um the newsletter to be so that's what i'm pitching at the moment.

Speaker: Awesome well definitely link in the the notes final question just out of curiosity quick quick response what's your favorite game of all time video game.

Speaker: You know, I think that, I don't know why you would say this because I didn't really play it that much, but I really love this game Into the Breach, which is kind of like a puzzle game.

Speaker: And I played it for several hours and I loved it so much I don't want to play the game in case I don't like it as much as I used to.

Speaker: But basically it's a bit like chess in some ways, but with robots and aliens.

Speaker: And I usually hate those kinds of games, actually.

Speaker: That's why I love it so much.

Speaker: I'm not really very patient.

Speaker: I'm not very patient.

Speaker: I'm a gamer.

Speaker: I get bored very quickly.

Speaker: and I don't like playing chess.

Speaker: And so when I got this, I was like, oh my god, this is like so, it's so amazingly done.

Speaker: And I think if you're kind of interested in game design, I would actually really look at it.

Speaker: I think it's one of the best game design games I've ever seen because it managed to get me to play like a tactical turn-based game when I hate those games.

Speaker: And it's also a great game because it's not really that addictive, you know, and I think actually good games probably shouldn't be addictive.

Speaker: I know people like say that as like a kind of compliment.

Speaker: Maybe we should be careful what we praise.

Speaker: Sure, Factorio is a good example of addictive games.

Speaker: But yeah, at risk of this becoming like another hour about talking about indie games.

Speaker: Thank you again, Adrian.

Speaker: It's been incredible.

Speaker: We'll be in touch soon and the recording will be live on YouTube shortly.

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Speaker

Recommended