Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
From Chaos to Control: Christina Camilleri on Building Safer Games by Design
36 Plays7 days ago
Episode Summary:
Christina Camilleri leads trust and safety product strategy at Netflix Games — but her journey started as a shy kid finding refuge in online worlds. In this deeply human conversation, Christina shares how her love for hacking, community, and chaos turned into a mission to build safer systems in gaming. We talk about safety by design, the monetization myth, building meaningful controls, and why your systems are only as strong as the people behind them.
If you care about retention, LiveOps, or what makes a game feel safe (but still intense), this episode is for you.
Timestamps:
00:03:00 — Christina’s hacker origin story and gaming as escape
00:12:00 — Why safety in games is getting harder (and why it matters more)
00:19:00 — League of Legends, low-trust users, and the culture of competitive games
00:33:00 — Safety is a retention strategy: what the data actually says
00:45:00 — Feedback loops, reputation systems, and when reports fail players
Guest Bio:
Christina Camilleri leads trust and safety product strategy at Netflix Games. She’s a former ethical hacker and security engineer, with experience spanning BAE Systems, Riot Games, and now Netflix. Christina is passionate about building safe, inclusive player experiences — and believes good design can create both tension and trust.
🔗 LinkedIn
Key Topics:
- Safety by design in game development
- Player behavior and trust systems
- Reputation scoring and false reporting
- Game moderation tooling
- Accessibility through player control
- Real-time safety vs. reactive systems
- Why toxic behavior kills retention
- Community-led systems that scale
Credits Line:
Hosted by Greg Posner, founder of Player Driven — a podcast about the business of games and how smart people scale them.
Recommended
Transcript
Introduction to Christina Camilleri
00:00:00
Speaker
Welcome to PlayerDriven. Today we are joined by Christina Camilleri, a hacker turned builder who now leads trust and safety product strategy at Netflix Games. Christina's background security engineering and ethical hacking gives her a unique lens on how to design safer, safer more inclusive game experiences from the ground up.
Gaming as a Lifeline
00:00:19
Speaker
Games were a lifeline for her growing up, and now she's making sure others have the same joy without the toxicity.
Focus on Safety and Studio Culture
00:00:25
Speaker
We're diving into what safety by design actually means, how studios can fix culture midstream, and why trust and safety isn't just a checkbox.
00:00:34
Speaker
It's a retention and monetization strategy. So, Christina, that was really a long intro. Thank you so much for joining us today. How are you? Good. Thank you. how are you? Thank you for having me.
00:00:46
Speaker
Yeah, no, thank you so much. i'm very excited about this.
Passions Beyond Gaming
00:00:49
Speaker
We got so much cool stuff. And the first thing I want to ask you is I was scouring LinkedIn and I saw you are into bikes and cars and I believe restoring them.
00:00:58
Speaker
Is there any projects you got going on that are super cool? Oh man. So I used to ride motorcycles, but I had to give it up after an accident. um I almost got into restoring a Datsun and almost got a old Chevy Stingray, but nothing at the moment. My current, I guess, half project is my truck that I've kitted out to go overlanding, which is quite the change from classic cars and motorcycles, but it's been fun.
The Allure of Overlanding
00:01:27
Speaker
I don't even know what that last part means. What was that? You were with the truck? Overlanding. So it basically is a crypt to go off-road on, ah yeah, to wild spots in the wilderness so I can get away from people in the city.
00:01:44
Speaker
My last podcast I did, and someone was talking about restoring a pickup truck, but he was putting like the giant tires on it to go to something in the desert where you like ride them through the dunes in the desert. yeah Is that similar?
00:01:55
Speaker
Probably sounds like overlanding. Yeah. Mine's a pickup truck, which now has a camper on the back so I can work and live out of it when I want to get away. sir it's fun. Can I ask how this hobby of restoration came to be?
From Hacking to Security Engineering
00:02:10
Speaker
um I think it stems back to enjoying taking something old or that was meant for something else and repurposing it to something that I like or enjoy more.
00:02:25
Speaker
um I think it also stems back to um my love for just taking things apart and figuring out how they work. And then in that process, when putting them back together, you realize you can put it back together sort of however you like um and customize it. So that's probably where it stems from. I've always had a bit of chaos gremlin in me. Again, I loved figuring out how things works by breaking them.
00:02:48
Speaker
And yeah, that naturally translated into security. um I think there's a thrill in learning how something works and then learning how to bypass them, um especially systems that were supposedly locked down.
00:03:03
Speaker
yeah. yeah So let's back up even more, because when we had our pre-call, we talked about your introduction to gaming. It started very personal at a very young age. and you know ah you have such a cool You have so much cool experience. You were with BAE, you were with Riot, and you have so much security.
00:03:19
Speaker
And you came into the gaming world and brought that
Online Communities and Personal Growth
00:03:22
Speaker
experience there. And you were telling me your story of how this all started. And I thought it was ah Beautiful story. If you wouldn't mind sharing kind of why is gaming or how did gaming get introduced to your life and how did it help you kind of find a way?
00:03:36
Speaker
Yeah, um i I spent a lot of time alone as a kid. um My mom works three jobs. And so when I was young, I was home alone a lot.
00:03:47
Speaker
um So games were around. I think my first console that my mom spent a long time saving up for was an Xbox. And then I sort of naturally transitioned to a PC.
00:04:00
Speaker
And they weren't just entertainment for me. They were, um as cheesy as it sounds, I guess my second home. um I spent, when I went, when I started playing on the PC, I spent a ton of time in games like Starcraft Brood War, Dark Souls, Half-Life, and then eventually sort of pivoted into online games like RuneScape.
00:04:23
Speaker
I think WoW took a decent chunk of my life and MapleStory was, I think, the first MMO that I played as a, when I was much younger. So that's sort of how I got in. I think just a lot of time home alone and then needing some kind of entertainment that allowed me to connect with either um like and a different world or other people.
00:04:44
Speaker
Yeah, you know, I mean, the Xbox is one thing getting started there. But when you start talking about your PC games, a lot of them just had to do with community. Right. It's just kind of. It's a weird question to ask. I don't even know how it comes across on podcast. But me personally, you know, in in real life, I'm very introverted. I'm shy. I'm quiet, right. But online, you have the ability to kind of mask yourself and kind of be someone else. Is that the feeling you had growing up?
00:05:08
Speaker
Yeah, I was definitely a bit of an awkward and shy kid. I um actually went to an all-girls school, and why that matters is they it was very difficult for me to find other like-minded kids that were into sort of the same nerdy, geeky stuff that I was, especially at that age.
00:05:25
Speaker
So it was also a way for me to connect with other like-minded people online,
Finding Community in Gaming
00:05:30
Speaker
too. Um, I actually made some of my, uh, really important and influential friendships. My first relationship actually came from someone that I met online, which is crazy to look back on.
00:05:42
Speaker
Um, and I, I got to meet some of them in real life too. So it, yeah, I was a shy and awkward kid, but that was sort of ah a way that I could meet others both online and in real life that I could relate to a lot more.
00:05:55
Speaker
When you've turned on that Xbox for the first time, do you remember that first game that you played? Does that memory kind of sit in your mind? Oh, man. I think the first PC games were a bit more coded into my memory, but I do remember it was one of those Xbox packages that came with two games um with it. I think it was grabbed by the ghoulies and i don't... Brute Force, that's what it was. It was some war fairly cheesy...
00:06:23
Speaker
Sorry to anyone that loves that game. um FPS, um where you could play as different characters. This is going off memory over 15 plus years ago, more than that, 20 plus years.
00:06:36
Speaker
um I do remember in enjoying them, but I have much more fun memories of of PC games. it's funny i mean brute force i remember that game so well they were like how get the next table i think there's like an alligator guy and there was an alligator guy he's my favorite wow that was i remember getting in i liked it but yeah no that that didn't work out apparently how they planned it to be no did they even make a second one or was that feel like it was planned i don't know if it made it to market maybe if someone's listening and actually knows that they they can let us know tell us i will nostalgia play that game
00:07:07
Speaker
So you started PC gaming, right? And at some point you get into security. Well, you started, it but before we get there, did you do any kind of the UGC route? the community You were obviously part of a community, right? I guess, how did you find the community?
00:07:22
Speaker
Did you participate? don't know the right questions to ask and kind of how did you interact with your communities? Yeah, um great question. Back then, um one of the strong memories I have was in MapleStory, I was part of a guild and the guild predominantly used IRC, internet relay chat, for communication.
00:07:45
Speaker
um And they also had a really active online forum, which I miss those as well. I don't think they're nearly as as active as they used to be. But it was through IRC that I weirdly formed my community around security. It was on Freenode at the time. And I think Freenode was kind of the go-to server for a lot of InfoSec and um nerdy folks.
00:08:10
Speaker
um And that's actually how I got connected to some security folks and then ultimately ended up at my first conference when I was really young, security conference. um But yeah, I found my community through IRC.
00:08:23
Speaker
and ia see Oh, man. was talking with someone about IRC the other week, and this was such like the Wild West of the internet back then. You'd had to find random IRC channels to join. and You don't know what you're joining. And it was just like so far before the Discord times now where it's a little more organized Wild West, but like...
00:08:43
Speaker
Yeah, and it was Wild West in in like in a lot of ways in that i I was, I mean, obviously a younger female in those spaces and there weren't really many, if any, safeguards or safety nets on on these games or on IRC.
00:09:01
Speaker
So it wasn't always safe. And um i realize now I think that the duality of being both um games and communities, both being place like a safe haven and and a place of risk was what shaped my passion for safety now.
00:09:19
Speaker
um but I don't think I realized that then. Is there a point where you think it clicked or you thought to yourself, because you know, back in the day, security probably wasn't even in the top 10 to 20 things you're thinking about when you're talking online with people. Like, was there a day when you look back you're like, holy shit, that could have been something that was really bad?
Safety in Online Interactions
00:09:35
Speaker
think Yeah, there's probably multiple memories I can recall on that. um The first thing that came to mind was um I met quite a few people in real life online. Actually, right when I graduated high school, I went on a trip to...
00:09:55
Speaker
around the US and quite a lot of the people I stayed with were people i I met on MapleStory or IRC, combination of different games that I've played. um And I was traveling solo um and that probably could have gone really horribly, me meeting and staying with people I met online for the first time.
00:10:15
Speaker
um But it ended up being a really, really great experience and one that I'll remember forever. I got to meet a lot of amazing people and see a lot of the states. Amazing the quality of people that come out. I mean, i'll even think back to like back in the day, right? Was it as unsafe? And I'd probably say no, it probably wasn't as unsafe just because it wasn't as accessible by everybody. Right. And now it's easier to get into a game. It's easier to get access to these things in it.
00:10:42
Speaker
it becomes scary and understanding why you might want tool cells tool sets to help protect that. But in all honesty, like it seems like right now we're going backwards and it's kind of like this, Holy shit moment. What is happening? Like what's, is it crazy time? Like is the world truly just cyclical? Like we do the same thing over and over again. And I don't know. That was kind of an off topic rant, but like you, you kind of hear about the stuff, you see it, you live it, you grown. And now we're doing it again.
00:11:11
Speaker
Yeah, I think a lot of things are a cycle. It just evolves or or a pendulum swing, I like to say. um But yeah, it's online communities. I mean, games online, things are way, way, way more accessible now. I think when I was growing up,
00:11:26
Speaker
on games and internet it was a little bit more community focused and it was new enough that people were still figuring out how to use it and figuring out how to connect with people online but now it's in front of everyone from the time ah kid is born like iPad kids exist and you're just very used to being connected with others and safeguards have gotten better but there's also just a lot more people online and other ways for exploitation and unsafe things to happen. I think a lot of that is just tied to human nature, unfortunately.
00:11:58
Speaker
You put a system in front of them, they'll find a way to get around it. it um There's a lot of lovely things that come out of human connection, but there's also a lot of awful things when you put two humans together.
00:12:10
Speaker
So yeah, I think it's, the landscape's a little better, but there's a lot more people online. There's a lot more surfaces to interact on today as well. There's ah something you said, and it resonated with what I just spoke about one of my podcasts, is you build constraints around something. And by nature, people are going to test to the constraints of what they can do and what they can't do.
00:12:32
Speaker
And I think you know when you have more people testing the constraints, you're going to find more problems here. And you went into security at BAE fba is one of your first jobs, and you've seen how this process works. and And when you start thinking about security, does that come to mind? Like, hey, I have to test...
00:12:49
Speaker
all the possible constraints here when it's time to start looking at that. And I'm rambling on question, right? How do you know what these constraints are or where to start looking? Oh, that's Loaded, I'm sorry.
00:13:01
Speaker
No, it's good. It's a good question. um i think it depends what you're testing. um But for me, what I loved about being a consultant or some people might call it like an ethical hacker is putting yourself in the, I guess, hacker's shoes or that offensive mentality where your goal is to try every way to to break into the system or to circumvent those controls.
00:13:30
Speaker
um I don't know if there was a place where I, actually there is a place where I always started, even when attacking um technical systems like web applications or servers or um infrastructure.
00:13:45
Speaker
think I learned over time that the weakest link was usually the human behind it. So you can have a super sophisticated lockdown system. But if, I don't know, Molly in HR is trained to help people and um to respond
Transition to Riot Games
00:14:02
Speaker
to people's needs on the phone or over email, she could be the weakest link into a system. Or um humans are also very gullible. So pretending to be a higher up or someone in charge and getting in that way was something I really enjoyed and that's called social engineering.
00:14:19
Speaker
um But yeah, sort of testing the whole gauntlet of everything from let's try the most common vulnerabilities, like maybe they haven't patched their system and then getting a little bit more sophisticated into like cross site scripting or SQL injection and then um Yeah, the whole the whole range. But my favorite one was usually trying the like social engineering, phishing, um pretending to be someone else, um that that route.
00:14:47
Speaker
I think that's fascinating. I love love that quote that the weakest link is the human behind it. ah That makes sense based on those tests that you got to take online to make sure you're security and basic stuff. um I think, I mean, all that stuff you're saying is so cool. I remember back when people were saying, hey if you want to find out people's like Facebook passwords, go hit forgot password. It's be like, what's your mom's maiden name? And then just go find out what their mom's maiden name is and you can figure that stuff out. And then, I don't know, kind of...
00:15:16
Speaker
How did you realize that this was a passion that you, I guess you kind of answered this, but like, how did you realize that this was your passion that you wanted to go down this route? um I think it all comes back to the the chaos gremlin really loved figuring out how things worked and how to bypass them.
00:15:33
Speaker
um It was also just a ever fast evolving, creative, constantly surprising field. You always had to keep up with the latest exploits, the the latest weaknesses.
00:15:46
Speaker
And that was really fun for me to to keep up with that industry. So you're at BAE and you decided to take ah job at Riot. And Riot, I think, is known for their player safety, player support. I think there's a gentleman there, Wes, who I follow on LinkedIn. Wes is great.
00:16:06
Speaker
Like awesome stuff that he puts together. His blog posts talking about the the different ways they do matchmaking and security. It's just yeah mind-boggling stuff. So from your time at Riot, you kind of tell us what your responsibilities were there? And and was there any key things that you learned there about security that you're like, OMG?
00:16:21
Speaker
Yeah, before I get into that real quick, I think there was a shift for me going from consulting and ethical hacking into Riot, which was while I really loved the thrill of breaking things over and over and finding exploits and weaknesses, what often happened is we'd hand the report over to the client that we were working for.
00:16:40
Speaker
And sometimes they would care and they would patch those systems. But most of the time, um i guess it felt like we were giving them their dirty laundry and saying like, this is terrible analogy. Hey, here's your dirty laundry, go clean it. And then we come back a quarter later or a year later, and then we'd see the same pile of dirty laundry and that um over time made me start feeling a little bit hollow like it breaking is fun but what's the point if no one is taking it seriously or no one is fixing it and i think that's what got me transitioning from consulting to going to riot games was wanting to help companies fix some of the issues that they had
00:17:21
Speaker
um and work more closely with the company to help them care and help them um be less likely to be attacked by by external folks.
00:17:32
Speaker
So um then to answer your question, what I was doing there, I ah had a bit of a weird hybrid role. I was focused on both employee security, so helping employees um um understand how to improve their security posture, but I also had a player investigation angle um And you can probably imagine that this the reports that came in and the the things that the tooling picked up at the time on League of Legends chat was probably the some of them the worst parts of player behavior I've seen.
00:18:08
Speaker
um but I was responsible for investigating high sensitivity threats there,
Understanding Toxicity in Competitive Games
00:18:15
Speaker
um then figuring out who was behind um the threats, and then sometimes escalating that to law enforcement. So it was very player play a safety focused there.
00:18:30
Speaker
Were these processes that you are already in place there? Did you help have a hand in I think it's just fascinating to kind of hear. mean, first of all, League always, for years, was known as kind of the toxic place to be, right? So, I mean, there's a few questions I have based on this. I guess the first one, I'll hear somewhere different, was like,
00:18:49
Speaker
did this open your world to a new type of nastiness that you didn't knew existed yeah I don't know if it was new. I think um this is maybe sad to say, but when you spend so much time online, especially me getting online as a kid, um you see ah lot of shit and I quickly became numb to it. And I became numb to a lot of things just based on the work I did. Even at BAE Systems, we had a we had a for a digital forensics case where um we were supposed to just be um confirming whether or not someone was watching adult content at work, but it evolved into something much worse.
00:19:32
Speaker
My point being like you, If you spend enough time online, you're bound to see all the horrible stuff. I think what Riot Games opened my eyes to, though, is the types of badness you can see and ah better understanding into what motivates some of that.
00:19:49
Speaker
um A lot of people like to make fun of League of Legends as like being one of the most toxic games online. I think they've done a lot to to try to improve that.
00:19:59
Speaker
But um I like to think a lot more about why that game was one of the worst. And um a lot of it comes down to you're putting low trust users into high trust situations, which is you're putting people sometimes that have never interacted with each other before um So low trust users into a high trust situation, meaning into a situation where they're expected to perform well.
00:20:29
Speaker
It's a competitive environment. You need all of your teammates to work well together. And the goal is to win. So that's a lot of pressure on someone to do well without really knowing anything about them.
00:20:41
Speaker
And that can create toxicity because if someone lets you down, maybe, you know, the reaction is going to be pretty poor or you're going to put a lot of heat and hate and pressure on that person that isn't performing well.
00:20:54
Speaker
And they're going to get defensive because everyone is in a high stress situation. And that just sort of. cultivates a high stress, aggressive environment for players.
00:21:07
Speaker
Um, there were certainly other things that contributed to that, like the lack of safety controls at the time. And, um,
00:21:17
Speaker
I don't know, Toxic Game is online playing competitive games. I feel like it's not, well, it's fair, but like if you look at a sport, ah thinking basketball, hockey, baseball, right? like It's a sport, and once in a while, people get aggressive, and they'll let their energy out, and they'll hit each other. but they'll esports is a sport as well. There's competition there There's people that need to get their anger out and and some people do it the wrong way and it's terrible and some people are just trolls. like But maybe there needs to be a better outlet for people to say, you know what, I got to get my aggression out. Instead of saying something mean to the person across the screen, maybe
00:21:52
Speaker
Maybe I just got to have I have these little like little grippy things here. Well, I just because I yeah always got be moving. Like maybe i need something like that. But I think, you know, at the end of the day, it's a competitive game, especially all of the Riot games and especially that like it's high stakes. and It's also one of the most popular games out there. So, yeah, you're going to be the spotlight just because you're probably one of the biggest.
00:22:14
Speaker
yep Yeah, that's exactly right. and And your outlet thing reminds me of something I also like to mention, which is also giving players controls over some of the experiences.
00:22:25
Speaker
Maybe um when you're thrown into a competitive match, you don't want instant access to like completely open text chat. Maybe you just want to talk to your team, um or maybe you just want to talk to certain individuals. Not not a good example for this situation because communication is...
00:22:41
Speaker
such an important aspect of competitive games. Sorry, my cat is... I have like a curry puff on my desk and he is very sneakily creeping up to it to try and get a bite.
00:22:52
Speaker
Christina is trying to wrestle her cat right now for our for our listeners. You can watch the video. ah um you some this is You're talking about your outlet and I think that's a great, great transition is like when Christina has a, oh shit, it's one of those types of days.
Managing Stress Through Nature
00:23:09
Speaker
What is your outlet?
00:23:11
Speaker
Oh, man, it depends on the type of stress. My go to now. So we sorry if this is getting too off tangent. It used to be Souls like games and Dark Souls um because this might sound really weird, but after I had a stressful day or something made me upset or angry.
00:23:32
Speaker
I would go take it out on a game that punished me in return. um and then it gave me a goal to like, even though this game is smacking me down and I'm dying over and over, it would give me something like a goal to try to get over a boss fight or ah or an area that I was struggling at.
00:23:52
Speaker
But that stresses me out more than anything now. i Huge fan of Dark Souls. I can't deal with it after a stressful day now. So now here's my cat, by the way. This guy helps me. with um with bad days. i mean, look at him.
00:24:05
Speaker
And then the real answer is probably getting out of the city and just disconnecting. Going back to talking about my truck, um I recently got back into fishing and recently have gotten really into fly fishing.
00:24:20
Speaker
But um I think just disconnecting, getting into nature while still having something to do. catch fish is is my disconnect now. Nice. You have a pair pair of waders you can wear into the water and, uh, go fishing.
00:24:37
Speaker
Yeah, I actually just got my first pair of waders two weekends ago. um i wanted to make sure I loved fly fishing before starting to invest in all of the gear.
00:24:48
Speaker
um And yeah, got waders. I went to, just got back from a trip from Italy a few weeks ago. And one of the things I did there up in the Dolomites was fly fish for a day with a guide, which was a magical experience. And that was a point where I was like, okay, it's time. I'm getting waders.
00:25:05
Speaker
There you new hobby. Your transition from BAE to Riot when you were talking about companies just wanted to do that checklist of check we did this. It reminds me of my old days of having to do RFPs.
00:25:20
Speaker
They'd come back after an RFP saying, you need to do this, this, and this, and this. And I'd give it to my products team who would show it to the engineering team and be like, yeah, we'll do that next quarter or the quarter after that. And like, did it happen?
00:25:32
Speaker
Maybe eventually, but it's just kind of like you want it to happen, but like I had no power to make it happen. Yep. Yeah. I think that was a tipping point for me right at the end of my career in consulting.
00:25:45
Speaker
We had this really, really big project that I think pushed me over the edge where we found probably the worst vulnerability we've ever found in that um there was an exposed admin panel that basically gave us access to every American citizen's PII and social security numbers. And we very, very quickly, yeah, minor, tiny little thing.
00:26:08
Speaker
um We very quickly reported it to the company. And then, you know, one month went by, two months went by, six months went by and it was still online. And it made me... What was different about this was like, wow, them not fixing this puts a lot of people in danger. um It's some really personal information just sitting there online behind a very, very weak barrier.
00:26:32
Speaker
um And I don't know if it was apathy or like lack of leadership or resources or they didn't care or i don't I don't know, but it didn't sit right with me. And that's sort of what pushed me into like, okay, I think i need to go work on the other side now.
00:26:49
Speaker
Yeah, I love it. It's that day you realize you don't want to report problems. You want to make solutions, right? And I think you then took that experience to Riot. You grew up from Riot and now you had this opportunity at Netflix. Can you tell us why you decided to make a move? Why it has made sense to your own personal journey?
00:27:07
Speaker
I'm going back. Yeah, i I realized that I wanted to be on the other side. I wanted to build systems that um would help and protect people. And that that extended into into games as well.
00:27:20
Speaker
um
00:27:23
Speaker
I think also i had a moment where I realized that I could combine two things that I really loved, which was video games and security.
Integrating Gaming and Security
00:27:31
Speaker
And Riot gave me an opportunity where I could combine both of those.
00:27:36
Speaker
um And because I believed in the potential of online games and online interactions. um Going back to the stories I was saying about my childhood and and meeting, having some of those really lovely experiences, meeting some of my closest friends, but also seeing the other side of that was a really good driver ah of, you know, I come from a place where I've experienced some of these things. I know how they, how to break them. I know how to, how people get around them. And both of those were motivated to
00:28:10
Speaker
I don't want other people to experience some of the things that I did online. And I want folks to have at least the levers and controls to have a better experience online because games are such a wonderful thing. um um And that's that's sort of what pushed me there.
00:28:27
Speaker
So I guess when it comes time for you to start building safety by design, right? Let's talk about what safety but design actually means. You've had plenty of experience over time from from learning about defensive systems and now Rai, and now you're up here, right? What does safety by design mean to you, k Christina, and how do you start putting this into effect?
00:28:50
Speaker
Great question, huge topic. um To me, it's safety by design is about intentionally shaping systems to prevent harm before it happens. So not just patching things when they go wrong or after things go wrong.
00:29:04
Speaker
um It's a much more proactive mindset, not just a feature set. so But the caveat I put there too is this doesn't also mean um only designing for pro-social behavior.
00:29:18
Speaker
um I think when some people think of safety by design, they think like it, it um you have to box yourself into only having safe experiences in that.
00:29:30
Speaker
But I don't think a design always means for um forcing purely pro-social behavior. think it's much more nuanced. um While games should promote respectful play, ah part of the beauty of gaming is that emotional range, going back to what we were talking about with competitive games, that tension, that rivalry, that chaos um is a very important part to gameplay as well.
00:29:54
Speaker
So um I think Smart Design acknowledges as well that not all intense or competitive interactions are toxic and we should support rivalry without without necessarily the real world harm aspect, um without that pressure or panic and um still giving the ability for players to be anonymous, but not also be cruel behind that too.
00:30:19
Speaker
When you start thinking about the larger picture of of monetization, right? Some people might say, why am I going to spend money to kind of worry about these solutions right now and not just build it from when it's time?
00:30:29
Speaker
Others will debate that you can monetize it from the beginning, right? What kind of factors do you take into account, whether it be maybe, hey, how does someone view this person's profile? or see like And i don't even know if this makes sense, right? But like how would you say to teams that, hey, it's important to...
00:30:45
Speaker
factor this design in now because down in the future it will protect your community yeah yeah that's a good question i have i have material on um like how can you how can your games still be safe and engaging um while still being profitable and i think one um misunderstanding it's the wrong word um misconception of trust and safety is that It inhibits fun and it's a cost center and it's similar to how security is viewed and that's something that the company has to do for legal reasons or safety reasons, but not necessarily something that contributes to the profitability or oh the success of the game, which I think is a really unfortunate and false statement or thought to have because...
The Business Case for Safety in Games
00:31:31
Speaker
um It's the opposite. I think safe safe games are sticky games. um When people feel respected and welcome, they stay. um i recently read research around, like, ah that just got published this year, that 65% of players would quit or ah would quit a game because of harmful interactions and 61% of those would who stay still reduce their in-game spending when when um faced with inappropriate behavior. And there's another study too.
00:32:04
Speaker
um There's a game called Sky Children of Light, very cozy, wonderful game. um And they showed that generosity in that game is very contagious. So players who observe or receive pro-social acts engage more and engage more frequently.
00:32:20
Speaker
What mistakes have you seen studios do um even when they have the right tools or in general? There's sort of three buckets i There's more than three, but the three I like to focus on of where devs go wrong is failing to address the potential for harm at its core. So proactive and moderate proactive and reactive moderate.
00:32:43
Speaker
Oh my goodness. Proactive and reactive moderation tooling is getting better and better at preventing and removing harmful content, but it still requires players to misbehave or something to go wrong before you can do something about it.
00:32:57
Speaker
So I like to have devs shift the mindset of a little bit of instead of anticipating bad things to happen, let's instead look at what be what might be causing that bad behavior to happen. And that leads us to the next thing, which is um trying to design features without considering social dynamics or proper controls is another thing.
00:33:20
Speaker
So, for example, um a game prototyped or tested by a small trusted group of people might have a totally different reception when released to a larger audience.
00:33:31
Speaker
um especially when you're releasing to an audience potentially full of anonymous strangers, not like a trusted testing group um who have never interacted with each other before and there's no established trust.
00:33:43
Speaker
um And I think I like to mention is it's it changes with a different
Designing for Social Dynamics to Prevent Toxicity
00:33:48
Speaker
genre of the game. So the type of behavior you see in a competitive game versus a cozy game will look very different. Competitive elements, like I was talking about before, can increase people's aggression. um and players can display toxicity um to try to urge their teammates to play well.
00:34:05
Speaker
um One real-world example I want to give here is I read a case study around Among Us a while ago, and it was initially prototyped and tested with like a trusted group of friends and developers, and then and then they launched it, and then it got massive reception. I think COVID had to do with some of that too, and it blew up in popularity.
00:34:27
Speaker
but they released the game with no um chat filtering and no basic reporting or blocking. And um the whole game is about murdering people and lying about it.
00:34:38
Speaker
So the game had a massive surge in toxic behavior and griefing. um I think largely due to the amount of strangers now interacting with one another and because it was a competitive game.
00:34:51
Speaker
Another common theme theme I see where developers go wrong or can consider doing better in is user controls. While they're great and they do allow people to um control the interactions a bit more or take action on bad behavior, think sometimes they feel like they don't have any real impact.
00:35:12
Speaker
um So a lot of trust a player has in a game's ability to address harm, i think, can be tied to features like ah reporting systems and feedback loops.
00:35:25
Speaker
So when we deploy controls like blocking, muting, reporting, they often fall short on actually combating the harassment. um think there was a study that shared that um one of their perceived effects that perceived ineffectiveness leads to a lack of trust in those systems, which then perpetuates that toxicity.
00:35:48
Speaker
So a real world example, Rainbow Six Siege had um some controls. They had muting and reporting in game. And then that would send send a report off to Ubisoft support.
00:36:01
Speaker
And then they said, you know, the controls were very, very easy, but they felt temporary and they did not feel like it actually combated the harassment at all. Simply ignoring chat and then moving on caused a lot of people to believe that it was okay to continue making the comments and harassing others because there's no impact there.
00:36:18
Speaker
So um making your controls feel impactful, sometimes having feedback loops to the player. i believe Overwatch has a system now where when you file a report, if an action is taken on that player, they'll actually send you a message in game a few days later saying, hey, thanks for sending that report in.
00:36:37
Speaker
We investigated and we took action on that player, which made two things happen. One, the reporter feel like their reports were were taken seriously and actually something was done about it.
00:36:47
Speaker
and to that then reinforces the community of like oh this company actually does something to stop and prevent bad behavior from happening so maybe maybe i should cut down on what i say or what i do um in game that might not be super safe um yeah i love that second point i mean both points are great but i i'm always a big fan of communication and i say over communicate right i mean I'd rather you hear you're doing something. And even if you can't figure it out, I'd rather you hear you say you can't figure it out than just ignore me as a gamer, right? That feedback loop is so important. And when someone takes the time to reply, this is why I got into customer support is like my first job. And I loved it. I thought like, it's important for you to hear back from me as I want it. Like, yeah and I love that. I think that's so important that what you're talking about is like, you report this, you get that feedback loop.
00:37:36
Speaker
On a kind of different note, if I'm the one making the report, is anything going to happen to me or come back at me? Like what happened if it's a false report by mistake? Am I going to get in trouble? um That's a great question.
00:37:48
Speaker
um so We don't usually want to have any retaliation, but um back... Sorry, this is going a little bit too far back maybe, but we were talking to me a bit before about testing systems and where do you start and what do you test for?
00:38:05
Speaker
And one thing I love to stress to other people is when you're designing even safety systems like block reporting functionality, you have to test those too because users will find ways to...
00:38:18
Speaker
um repurpose systems that are designed to keep people safe into weapons as well. um And one thing that we've seen with reporting is, yeah, people submitting um massive amounts of false reports.
00:38:33
Speaker
There's even been games that will have some automated action on a user when you submit like a certain amount of reports on them and players will quickly usually usually quickly find out and then use that against players that they don't want to um continue being online.
00:38:47
Speaker
But Yes, I always encourage that um games with large online community or online, an online interaction aspect have some kind of reputation system.
00:38:59
Speaker
So you want to track um both the good and and bad. So how many times someone's been reported or how many times they've been action against but also positive contributions like how many reports have they submitted, where it has been accurate and has led to action.
00:39:14
Speaker
um on the On the opposite side of that, um also tracking, has this play a does player have a history of submitting many false reports that don't have... I'm losing my words.
00:39:28
Speaker
Has that player submitted a lot of false reports and is demonstrating bad behavior through that feature? yep. so yep I've seen more tools come to market now that are working to kind of create a united user profile.
00:39:43
Speaker
so you can kind of sign up for one user profile and be age verified and check all those boxes. Would it be to the point you were just saying where, you you know, players have reputation, would it be better to have a shared reputation across the entire gaming universe or per publisher or however you want to look at that?
Debating Shared Reputation Systems
00:40:01
Speaker
Oh, that's a tricky question. um
00:40:06
Speaker
i don't I like the idea of having a shared reputation system, but then it sort of creeps into the credit score feeling world, and I'm not that sure how much that would benefit games, especially across different markets and regions and even studios would would help.
00:40:28
Speaker
um I do encourage that if you're a studio with a wide range of games that you do track that behavior across your portfolio of games um with also considering that the behavior a player might show in one game could be very, very different in another game.
00:40:50
Speaker
um for example, when I'm playing with a trusted group of friends in a competitive game, like the worst parts of me come out or very different parts of my behavior come out.
00:41:03
Speaker
I um all ingest in jestine and friendliness ah toward my friends. But um when I switched to then like a cozy farming sim or Valheim or something.
00:41:15
Speaker
um the best parts of my, like, okay, let's build a base and, uh, gather these resources and just cozy things come out. Um, why I bring this up is,
00:41:27
Speaker
Yes, maintain a reputation score system for your players because it can help you make better decisions about actions to take against those users. If they have a proven track record of being a terrible person, maybe you don't want them to have access to really rich online communication features in games.
00:41:44
Speaker
um but
00:41:48
Speaker
I don't know where I was going with this. I would take love where you're going. I mean, just because you're a troll in one game doesn't mean you're troll in another game. It doesn't mean you're going to repeat that bad behavior. Yeah, so I think also um focusing the actions you take on players to specific games or to specific features or in response to specific kind of behavior makes more sense than trying to do it across the board.
00:42:15
Speaker
How do you start to justify the appropriate action to take on a person?
00:42:22
Speaker
The general things i will say here, ah try to map the action to whatever bad behavior they demonstrated. So if, shitty example, but if the game, if the player is really, really generally positive player, um good at teamwork, um not abusive um generally, but for some reason when they're on voice chat in a competitive session, they have,
00:42:50
Speaker
a tendency to, i don't know, be passionate on voice is the word I'll use. and they're doing this repeatedly, instead of actioning against their ability to play the game at all, like prevent suspending or banning them from the game, um try to map them more specifically to the behavior they're demonstrating that you want them to stop or change.
00:43:12
Speaker
So um don't allow them to use voice chat or put restrictions on who they can use voice chat with. Another thing i want to do I want to mention here too is there Sorry, I want to find this quote because I really loved it.
00:43:28
Speaker
When players understand what is and isn't acceptable behavior and you tell that to them proactively, they're much more understanding and much more likely to want to change that behavior or adapt that behavior.
00:43:44
Speaker
Usually what it is is just someone's lack of understanding of what is and isn't appropriate in a specific game or so situation. Because there's a lot of other things that come into play here, like people's culture, how are they how how they were raised, where they're from, um that the types of people that you typically play with.
00:44:01
Speaker
So if you sort of set the boundary or set their expectation early with the player, they're more likely to follow that than if you punish them for doing something wrong that they didn't know was wrong later.
00:44:12
Speaker
um Fascinating. I'm thinking back to my kids and how I react on something they do. And it's kind of the same thing. Like, how do you react when someone makes that first mistake? Do you scream at yeah them the first time they make that mistake or let them know?
00:44:24
Speaker
Let them know. Yeah. Tell them why it wasn't appropriate, what effects it has on other people. And i maybe I'm being a little bit too um optimistic about what, how humans behave. But I generally think that if you set that expectation and they understand why it's wrong, they'll be less likely to to do that.
00:44:43
Speaker
I think on the other side, though, it can also motivate people to, oh, this is what will stir the pot the most. So this is what will get more of a reaction from this person. um So let me go do that. And yep, it's a delicate balance.
00:44:57
Speaker
Not related to gaming at all. I remember when I was very little, i was in my house and my dad said we had a cactus out front, like a
Balancing Punishment and Education in Gaming
00:45:03
Speaker
small cactus. He's like, don't touch the cactus. What do I do i go outside, touch the cactus, I get a splinter, right? It's that whole idea of like, you're going to tell me something. I'm going to- have to test it. You got to test it, right? And hopefully ah you only end up hurting yourself with no one else.
00:45:17
Speaker
Yeah, going back to the like reporting thing I was talking about before, there's also a tricky balance there too, in that there's studies showing that if you give them a bit too much detail um to the person who got reported, like, hey, we're banning or suspending you because you demonstrated really offensive language in this in this chat, it can sometimes have the opposite effect of...
00:45:39
Speaker
encouraging people or giving them insight and into how to get around that system. So yeah, um some it might work to stop someone from repeating that bad behavior, but it might also inadvertently encourage them to to do more of it.
00:45:54
Speaker
Kind of going back to what you saying a little earlier, right? And kind of changing up where we're going is Games like Among Us blew up with popularity out of nowhere right after it was kind of long forgotten. And they kind of were in a bad spot because they're like, oh, no, we need to do something and we're already live.
00:46:11
Speaker
When the game is already live and is back, already kind of in the hands of everybody, right? Is it too late? How can they go about getting started when a crawl, walk, run approach really isn't acceptable?
00:46:25
Speaker
It's harder when you're already live, um but possible, i think with commitment and new systems and analyzing where the abuse is happening and why it's happening. um I think one thing I'll say here too is you can't just bolt on a moderation tool and hope for the best. You need some kind of reassessment or reset into that system.
00:46:47
Speaker
Um, There are ways of doing it depending on what kind of abuse you're seeing. I know that Rainbow Six Siege added um real-time muting and reporting tools to try to just make it easier for users to raise bad behavior when it happened.
00:47:03
Speaker
I can talk about opt-in gates as well. So, sorry, I'm just trying to tie all of these things together in my head. um In a personal experience, I don't know if this is too related.
00:47:14
Speaker
i remember first getting a VR headset and jumping into VR ah chat to see how ridiculous that was and was immediately thrown into open proximity chat with no way of um muting it or turning it off or like controlling who could I could hear or who could like um interact with me.
00:47:38
Speaker
And just using that as an example is ah good example of assumptions developers can sometimes make about the experience their players want to to have. And that's where a lot of abuse can come from. Just assuming that you can throw...
00:47:54
Speaker
someone new coming into your game into like an open voice chat or open text chat or global chat environment and hope that nothing bad will happen. And when it does happen, um having controls that don't really allow the user to to um opt in or opt out of those experiences.
00:48:10
Speaker
So um going back to yeah games that are already live, um adding in more levers and controls over those experiences is I think something that has helped a lot.
00:48:23
Speaker
i I have notes here about, like, there's a lot of things I wanted to talk about. Going back to, yeah, when players log in, immediately face open voice chat or toxic global chat. It can feel super overwhelming.
00:48:34
Speaker
and that Accessibility by design. Yeah, accessibility through safety. And then I talk a little bit about um the ADL, the Anti-Defamation League notes, games with open interaction systems without clear player controls can exacerbate harm, especially for marginalized players.
00:48:53
Speaker
um And they stress a lot that being able to tailor one's experience is often is as important as having moderation systems. So the examples I have we here were Roblox has, it requires your ID for using spatial voice.
Player Control and Content Creation
00:49:07
Speaker
Fortnite lets players opt out entirely of voice chat. um Destiny 2 gives you really, really granular privacy settings. um These types of granular controls is what helps players of all kinds have more control over the experience they want to have in the game. So this is shifting the the conversation a bit away from what reactive controls can we put in place to sort of stop bad behavior after it happens and shifting it more into what what controls can we give to the players and what power can we give to the players to tailor the experience that they want to have.
00:49:44
Speaker
Why is there no, um and maybe we're getting there, right? But why is there no standard suite of, hey, these are the accessibility options that should be available in your game? And clearly cost will be an aspect of that, right? But that can be something that with bare bones, you can mitigate to minimal work. That's a good question. I don't know that I have the best answer. I know that there are companies or or groups out there like the...
00:50:10
Speaker
um they did They just changed their name actually, the Fair Play Alliance, but I don't know what they're called now. there's There's things out there that will talk about and set standards of things that you should consider and implement into your game to have a safer experience. but I think there's a lot of factors that contribute to why games don't usually adhere to them. It could be anything from trust and safety or safety is not considered a prioritization at the company.
00:50:38
Speaker
Maybe it's, they see it more as an effort or um the way else or the benefits are not worth putting investment into. it could be a resourcing thing. It could be ah lack of understanding their community, where the abuse stems from and how to combat against it.
00:50:55
Speaker
um I think generally it's just an un, I don't want to say that, but I think it it is ah can be a misunderstanding of how to combat abuse or what even causes that abuse in certain games.
00:51:08
Speaker
Some studios just don't even care. I think one of the um competitive online FPSs are like, yeah, we know where our players are toxic and horrible. It's a fucking shooting game and we don't care to do anything about it.
00:51:22
Speaker
I think you can have that, but also have some basic controls in place for someone to minimize that. It's crazy. It's funny. When you first brought this topic up, I thought years ago, me and my buddy were into Uno on the Xbox. And Uno at one point enabled the webcam mode and people's webcams were automatically turned on. And it was like, there was lots of balls that went on. Yeah, also didn't talk about in this podcast is TTP. So there was somewhere, think like why games are important to you, but I talked a bit, I want to talk a bit about...
00:51:57
Speaker
player When you give players the tools to make content, it can be like a really, really wonderful thing. you see amazing creations in Breath of the Wild, Roblox, Minecraft, like player creativity is fucking amazing.
00:52:10
Speaker
um But on the other side of that is TCP or time to penis and that when you give players controls and the ability to create content, the first thing they're typically going to make is a penis or phallic object or something inappropriate.
00:52:25
Speaker
um And I talk about actually the Spore example and Uno was one as well. Like you introduce webcams or picture uploads. What are you going to get?
00:52:36
Speaker
Dix. Dicks. Don't know why it's always Dicks, but there it is. Yeah. Spore is a hilarious story because they, i mean, Spore is a game where you can create majestic creatures.
00:52:47
Speaker
um First thing that players made, Dicks. um And the team actually tried to combat it by introducing a system. I think that would take screenshots, like four screenshots of a player's creation and run it through a detection model before allowing upload.
00:53:04
Speaker
um And it would take screenshots for like four corners of the creation and players pretty quickly found that if you made the creature look safe from those four angles, but inappropriate from the other angles, it would bypass the controls entirely. i think the devs eventually like, we can't proactively stop this stuff from being created. Let's invest more heavily into reactive controls to like make it easier for people to report inappropriate content when it's seen.
00:53:32
Speaker
Um, but it's just a great example of, um, plays being passionate and that can lead to great things and also terrible things. I had no idea sport even had that side of a community to it. That is amazing.
00:53:46
Speaker
I want to go back to when you kind of restarted the sections, you said something I really liked is that you want to give the power back to the player basically to make the decision that
Customization in Gaming Experience
00:53:53
Speaker
they want that. And it's just, it reminds me why I wanted to become kind of the sales engineer for customer support tools. Cause sales would sell stuff we couldn't do. and I was like, I want to protect the customer support team. I want to make sure that what we're selling is the tools we get, the tools we can do.
00:54:08
Speaker
um I want to make sure that like, and it sounds like you want to protect those people too. You want to protect the people that are paying for the game, buying the game, right? You want them to be able to choose what to turn on, what to turn off. and ah And I like that it's kind of that gamer mindset of yours. It's like, I know who should have these options. And it sounds like when you're designing it you're the gamer in you is the one that's kind of helping guide this. And I think that's why it's important to have people who play games, grew up with games, know these experiences to help with this stuff because you've lived it, you've been through it.
00:54:36
Speaker
Yeah, you just reminded me too that when you don't give the appropriate amount of controls or levers to the player, you're sort of making an assumption on how they want to play and experience your game.
00:54:47
Speaker
Maybe someone loves to play a hyper-competitive shooter but just doesn't ever want to be on voice chat. Maybe they only want to have pre-selected phrases um and not hear people shouting on the other end.
00:54:58
Speaker
And by not giving them the controls, you're assuming, hey, this kind of player wants this kind of experience and that's the experience they're going to have. um But by just adding a little bit more customization options or ways that they can choose to opt in or opt out of certain experiences while still being part of that core gameplay experience, think is is equally as important.
00:55:19
Speaker
i have one other last question then. Christina Camilleri, what is the best gaming community or the most memorable gaming community you've been a part of? Jeez, what?
00:55:31
Speaker
Most memorable gaming community? My honest answer is like, I'm not a huge online community person anymore. I like to play single player yeah go back what say The best community.
Reflecting on Gaming's Impact
00:55:49
Speaker
was probably that tight knit group of players I play with in my guild in MapleStory. I don't know that it was the best. It was probably pretty bad, but it was, it opened my eyes to online communities, I suppose, and the ways that we can interact. And, um, I don't know that it was the best.
00:56:10
Speaker
Do you still talk to any of them? Sorry? you still talk to any of them? Um, one or two. Yeah. Not, not very frequently these days, but. Do you think you would be where you are today if you didn't be, if you weren't part of that community?
00:56:24
Speaker
No, it's a really wacky way, but because I was part of that guild that led me to IOC, IOC led me to those security folks that led me to my first conference.
00:56:36
Speaker
If I didn't go to that first conference, I wouldn't have gone to this like recruiting event. If I didn't go to that recruiting event, I wouldn't have gotten my first job. Um, Weirdly, it's also how I ended up in the US.
00:56:48
Speaker
Like I went to go see um and music show that I really loved and in a weird and wonderful way, that's how I decided to move through the US and then pursue my career here. So no, it's i have probably MapleStory to thank for my career and my move to this country um in a weird and roundabout way, but yeah.
00:57:12
Speaker
yeah
Engagements and Connections
00:57:13
Speaker
There we go. And next you can see Christina talking at Gamescom as part of Community Clubhouse. Right? You are still doing that? before Yeah. So we will be able to have links to where you can check her out and sign up for that talk here.
00:57:27
Speaker
k Christina, you have such a cool story and I really appreciate you coming and sharing from the beginning to the end and the middle and everything you had. There were so many cool quotes that I'm excited to pull out of here. Before we do go, is there anyone you want to shout out or just say in general or let us know where we can find you?
00:57:45
Speaker
I don't have any shout outs. My cat Lucy can find me on LinkedIn. or you can email me. I don't know what two people you can share. don't email me. You don't want that. I'm on LinkedIn under my name. There we go. easiest way to reach out.
00:57:59
Speaker
We will link to Christina. i have all information. Again, this is some really cool stuff from Trust and Safety, and didn't even talk about i'd love to eventually have a follow-up where we talk about how you did this from the security side rather than the customer support side and where they overlap because it's such a cool space, the Trust and Safety, and the types of characters that are in there. But Thank you again for coming out today.
00:58:21
Speaker
i hope you have a great rest of your day. Thank you so much, Greg. It was a pleasure of being on here.