Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
4. Game Plan
511 Plays9 months ago
Dr. Ellie Bartels, the co-director of RAND’s Center for Gaming, joins Andrew for a conversation on war game design. Ellie and Andrew break down their approaches to building and running games. What are the basic elements of a wargame? What kinds of choices do game architects have to make? And what are the implications of these choices?
Key Points
- When first building a wargame designers should hone in on the type of wargame that they need: whether it should be pedagogical, experiential, or analytical, what key themes or questions should it explore, and what is sought from the players in the game.
- Different styles of wargames yield different types of insights, so wargame design is derived from intended outcomes. If a game is meant to explore a problem set then it can be designed to be less structured, often leaning towards a seminar style game. Games that are meant to test hypotheses will be more structured and analytically rigorous.
- Wargames come in many different forms. They can be board game based, take place one line, or exist mainly on a powerpoint. The medium of gameplay is less important than the overall structure and world-building of the game itself.
- As wargaming undergoes a renaissance, Ellie walks us through three areas of promise: (1) discussions on fundamental research design and what good versus bad wargame design looks like; (2) examinations of, and evolutions in, how to better measure human decision making; and (3) the arrival of new technological tools, like generative AI, that can enables wargame designers to scale wargame design, potentially making it better, cheaper, and and faster.
Ellie’s Recommendations for Engaging with Wargames
- The Connections Wargaming Conference
- Ellie Bartel’s “Building Better Games for Policy Analysis,” RAND (2020)
- Robert Burks, Jeff Appleget, and Fred Cameron’s The Craft of Wargaming: A Detailed Planning Guide for Defense Planners and Analysts
A reading list to accompany this podcast series can be found at: https://brsl.berkeley.edu/podcasts/
Recommended
Transcript
Exploring Games as Tools for Expression
00:00:05
Speaker
You know, the whole point of games is that there are a way to get at the squishiness that's going on inside people's heads. The question then becomes, how do we elicit that from people? How do we pull it out of their head and get it into something that you and I as observers can see?
Welcome to the Risk Calculus Podcast
00:00:22
Speaker
Hello and welcome to the Risk Calculus, a podcast from UC Berkeley's Risk and Security Lab. I'm Andrew Reddy, the lab's founder and your host for this series on wargaming. In the last episode, we discussed some of the past applications of wargaming methods. Today, we're going to be talking about game design and what it takes to build and run a wargaming one of the world's leading experts on game design. Specifically, we ask, what are the basic elements of a wargame? What kinds of choices do game architects have to make? What are the implications of these choices?
Meet Dr. Ellie Bartels: Wargaming Expert
00:00:53
Speaker
My guest today is Dr. Ellie Bartels, the co-director of Rand Center for Gaming and a policy researcher focused on defense issues. Welcome, Ellie. As we get started, just given your background in the field, I'm wondering if we could speak through kind of what, from your perspective, are the basic elements of a war game.
What Are the Key Elements of Wargaming?
00:01:10
Speaker
Sure. Yeah. So I tend to think of a war game as having sort of three constituent parts, but maybe let me back up quickly and start by saying what I define as a war game, because there's lots of different technology out in the field and can be kind of confusingly overlapping. So when I talk about a game, I talk about an event in which human players are asked to make decisions in a contested environment and then experience the consequences of those decisions.
00:01:33
Speaker
And so what I think of as the three elements of a game's design are really embedded within that definition. So on one hand, you have the actors who are making decisions, and in particular, the human beings who you're actually asking to take on those roles. And so that's really important because it distinguishes games from things like workshops or seminars in which we have experts come together and offer their opinions about what might happen and questions that are maybe similar to the types of questions we ask of a war game.
00:02:00
Speaker
But we're not asking them to take on the type of role that we do when we're playing a policy game. The second piece is really that contested environment. And so in a classic war game, we think about a battle or a war. But we actually use games in a lot broader space. And so I like to think about contestation. That can be bureaucratic friction between different elements of the same government. That can be between non-defense spaces between the residents of a city and a city planner and the real estate developers.
00:02:30
Speaker
anytime that you've got a policy space where there are different issues that people feel have different values and different preferences on, that's really encapsulated in the environment of the game. And then the third piece is really the living with the consequences of their decisions. And so that's really the roles and the process of adjudication that we have that determine
00:02:51
Speaker
what actions people can take and what impact those actions have on the other players and on the game environment. And so when we put those pieces together, we end up with a system that has, on one hand, the actors who have a particular set of preferences, goals, objectives. We have the environment that they're trying to shape and move that presents the policy problem of interest. And then we have the roles adjudication that govern how those actors can influence one another and the policy space itself.
00:03:19
Speaker
Perfect. That's great.
Factors Shaping Game Design
00:03:21
Speaker
And so obviously, as you're just kind of thinking about game design, there's all sorts of decisions that you have to make, right? Regarding medium and the format of the game and, you know, how you want to have education occur. I'm just wondering if you can kind of speak to the variety of different choices that you have to make as you're kind of making those game design decisions.
00:03:39
Speaker
Yeah, so the first place I start with is really trying to think about what type of game am I running? And so there's a couple of sort of big framing ideas I use when I think about that. So the first is, is this a game for research and analysis or is this a game for communication and education?
00:03:55
Speaker
And the big distinction there is whether this is a game where I'm hoping to learn information from the players. I'm trying to pull information out of them to be able to generate new understanding. Or is this a game that I'm trying to communicate my understanding of a problem to the players through the experience of going through the game?
00:04:13
Speaker
is this a game where I have a clear understanding of how this problem works and what I think the sort of important parts of the system are that's going to lead me towards a game that is probably more structured, maybe a little bit more of that kind of classic rigid rule set versus do I have a game where I'm interested in trying to pull information out of the players and there I want more flexibility because in part what I'm trying to do is say I don't understand the full dynamics of this system
00:04:42
Speaker
well enough to think I'm able to completely represent it in a game. So I want to leave space in my game design for players to be able to tell me their understanding of the system and be able to impact that themselves. And so that tends to lead to looser designs where there's more space for that player input.
Designing Games for Different Audiences
00:04:57
Speaker
The next big decision that I'm going to make is really about who the audience of the game is and what the type of problem I'm interested in exploring is. And my work, I'm mostly working on the research and analysis side of the spectrum. And so in that space, I really tend to think about games as either being about really trying to explore the structure of the policy problem or being about experimenting with different potential solutions. And then I think a little bit about who the audience for the game is. Is this really about the game design team?
00:05:28
Speaker
and maybe, you know, sort of the immediate people who are involved in the game directly as players or indirectly as like the sponsors and observers of the game? Or is this a game where I'm hoping to be able to take the data I'm collecting from the game and persuade people who weren't in the game themselves that the evidence I've collected is sort of valuable for decision making purposes?
00:05:48
Speaker
Again, that's sort of a question that has to do with the maturity of the research problem, right? Am I in that early kind of soaking and poking stage of research, or have I actually kind of coalesced in having strong hypotheses that I'm in the process of doing testing on?
00:06:02
Speaker
And so what that sort of sets up is a sort of classic two by two kind of framework where we can think about problems, games where we're interested in exploring a problem space for internal audiences. And I tend to think of those as system exploration. I'm trying to figure out what the component parts of a policy problem are and how they interact with each other. I can think about innovation games where I'm interested in exploring what potential solutions might be. So it's almost like a brainstorming activity.
00:06:28
Speaker
I can think about games that are about exploring alternative conditions. This is like A-B testing. And Andrew, I think their work fits very neatly in this category. What we're interested in exploring, you know, if I have policy context one or policy context two, how is decision making different under those conditions?
00:06:45
Speaker
And then the last category, and this is probably the thing that games are best known for in defense context, though, actually, I don't think that the majority of games that we run is really games for evaluation. And so this would be
Types of Wargames
00:06:56
Speaker
like a course of action or concept analysis type games in which we're almost prototyping or road testing an idea. We have a potential solution and we want to have run it through the game and see how well it performs.
00:07:10
Speaker
We don't want to say that that's been going to validate that that solution is definitely going to work. But it's a really great way of telling us what the second order consequences that are really lousy that we haven't anticipated of our potential idea are. And so that's sort of the fourth main sort of chunk.
00:07:26
Speaker
Those distinctions are important because I'm going to design the game differently depending on which of those quadrants I'm existing in. If I'm in that systems exploration point, I really want to have a big open space in which players are able to tell me how they think about the structure of the game. And so that's a game that's likely to have pretty loose rules. That might be what's called a seminar style game where people are sitting around the table and most of the gameplay happens just through verbal conversation.
00:07:51
Speaker
And so in that sort of game, rather than having like a set of rules about what plays you can and can't make and what the consequences are, it would be more discussive. I would have a player who's representing the US and a player that's representing Russia, and the US player would say, hey, I'm thinking about doing X, Y, and Z. And the Russia player would say, well, in response, I would do A, B, and Z, right? So almost the entirety of the rules are just that discussive practice back and forth.
00:08:16
Speaker
because we don't know enough to be able to set up more rigid rules. We're really trying to understand what the space of action is that's available, what types of consequences there might be from the other actors in the system. We can contrast that. When we get into those policy prototyping type games, those evaluatory type games, those we tend to be much more interested in being able to play out the consequences of decisions in a way that lines up with our other types of analysis. That's where I might pull on technical system data,
00:08:45
Speaker
I might pull on historical operations to be able to say, here's the types of effects that I would expect to see based on the military interactions or the social interactions or political interactions. And so you'd see similar types of choices in all of these quadrants where when you're speaking about sort of how you're selecting your players, how you're constructing your environment and what sorts of rules you are or are not putting into place, those are really going to fall in line with those sort of big buckets from my perspective.
00:09:14
Speaker
That's great. Yeah, I really appreciate the point that you have to be really intentional about what your game is for and letting that kind of drive your game design decisions, because I think that's where that's probably where the most kind of games that go wrong kind of start going wrong, right? Where you've got a game, this doesn't match actually what you're trying to use it for.
00:09:31
Speaker
And we often have this problem where people have seen one game and so they think that their game should look like the game they've already seen. I think that that's natural, right? Most methodologies aren't as formal in the sense of the way the thing looks.
00:09:46
Speaker
Um, like games can look so different from each other. You can have a room with a bunch of people sitting around with sheets of paper in front of them and a PowerPoint on the screen. And you'd think you just stumbled into a meeting. Like that can be a game or we can have a game that's got people with boards and counters, or we can have games that have computers. I think it's really easy to fixate on those, that sort of medium, the format of the game, and then kind of miss some of these sort of more foundational questions about what's really driving the research questions and the research design that follows out of it.
00:10:16
Speaker
Yeah, I mean, I know you've obviously designed games for a variety of different government agencies over your career. I'm just wondering kind of, have you found the different types of game, wargame sponsors or are more or less comfortable with particular types of wargame designs? It's a good question. I think there does tend to be a little bit of, you know, what, what you've seen previously in your career is kind of what you expect. That's not consistent across government though. So just to give an example,
00:10:43
Speaker
There's lots of games that are interested in interagency processes right how does the whole of US government get itself aligned around a singular policy issue that might be novel or particularly problematic at the moment because there's big differences between how different agents think about it.
00:11:00
Speaker
Those games tend to be seminar style game and that makes sense, right? They tend to be in that system exploration place. Players come in with really different understandings of the problem and what you're doing in that game is ironing out those differences. But then when you try to put more game mechanisms on top of that, sometimes that can be jarring for people who are used to that sort of seminar style format.
00:11:20
Speaker
I think the converse side of it is that often when you're dealing with uniform military officers who are very used to operational games that are looking at either sort of tactical level or operational level war fighting decisions in the kinetic space. There's some sort of standard.
00:11:37
Speaker
we use to present that information. So we have maps that have hex cords overlaid on top of them, counters that we use to indicate different types of military floors, more rigid rules about how we're adjudicating the kinetic interactions between military systems. And if you then transfer them into using a GAM, for example, acquisition
Influence of Past Games and Visuals in Gaming
00:11:56
Speaker
policy, and all of a sudden there's none of that sort of infrastructure that they're used to, I think that can be jarring as well.
00:12:04
Speaker
I think what's important, though, is that's not just that they're sort of latched on to the visuals of it or the structure of it. They're also latched on to the use case for game. Right. I think a lot of times, whatever the first game we see is, is what we think of as what games look like, but also what we think games are for. And so part of why I find the sort of framework helpful is because it sets up the diversity of what we can actually use games for so we can illustrate some of that breadth.
00:12:31
Speaker
to people who might have only seen one piece of the pie so far. And Ellie, what I want to kind of move the conversation to now is thinking about kind of what distinguishes a good game from the RAM perspective from
What Makes a Game Good or Bad?
00:12:53
Speaker
a bad game. And when you're kind of brought in to consult in a wargaming project, how do you look to make a game better?
00:12:59
Speaker
Yeah, it's a great question. So I think the first thing, and this isn't probably going to be a surprise to those who listen to me in the first part, sort of lay out the differences I see between different types of games, is that I think it's really important that we design a game specifically to the question that we're interested in getting after. And I think often, you know, when you read through the gaming literature, there's lots of talk about the importance of the game purpose and the objectives in shaping
00:13:24
Speaker
How you do game design and that's true but i think that can be a little amorphous for people sometimes the way i like to think about it is what is the information that you are trying to generate out of the game and because games are fundamentally about human decisions that piece of information is.
00:13:41
Speaker
almost always either the specific decision that somebody made or the process by which they made that decision or something about how people reacted to the outcomes of the decision. And different games will touch on all three of those and some will only be kind of really interested in digging in on one or two of those. But really what you want is a game design that's going to highlight those decision points that you're interested in
00:14:06
Speaker
both so that players can sort of think about them consciously, but also so that you can capture the data that you need about it. And so the way I often think about game design is that game design should be helping you capture that data. Jeff Applegate got a really good book called The Craft of Wargaming that does a really nice job of walking through some of how you can think about decomposing this in order to be able to try to get down to it.
00:14:30
Speaker
And I think one of the things that this focus on decision-making does is it helps keep us from trying to have a game that's answering too many questions, right? I think it's really tempting to just add, oh, you just collect data on this one more thing and this one more thing and this one more thing. And the next thing you know, you've got a data collection list that's so long you can't possibly get through all of it.
00:14:49
Speaker
And so I think really reframing from sort of the general point about, you know, the purpose and objectives should drive game design to what is the decisions that you want the game to highlight so that you can generate information about them from the game. I think that that sort of next us down a little bit. And I think it also can help highlight when we've got objectives that are in tension with one another, right? If you're really interested in understanding
00:15:15
Speaker
interagency decision making and also super interested in understanding tactical details about how X, Y, and Z system is going to perform. You've got a really clear disconnect there between the kind of level of analysis. The last thing I would say, and this is a really common pitfall, is that people want a game because they want a game, not because a game is suitable to the research question that they're asking.
00:15:38
Speaker
And so attention to, you know, is this a question that's really about human decision-making or is this a question that is about other aspects of policymaking and can probably be better tackled using other types of tools? For example, if we've got really good real world observational data about a field. Use that. Don't use a facsimile when we've got the real thing. If we're interested in studying the far future and there's no data available on it, like then go have a conversation about games.
00:16:07
Speaker
If you're caring about the specifics of technical performance, that's not really about the human, that's about the machine, like go run range tests and figure out whether it actually does what you think it does. Perfect. So now, Ali, I'd like to move us under a conversation around what kind of games you're building now and how kind of new methods, new technologies are helping us to kind of do wargaming better to kind of carry out this
00:16:33
Speaker
analysis of the inferences that we can glean from these gaming environments better in the future.
Innovations and Technologies in Wargaming
00:16:39
Speaker
Yeah, so I think I tend to think about sort of three big areas where I think we're seeing a lot of really exciting work right now. So I think one of them is really in kind of fundamental research design. You know, gaming has really, I think, suffered from a little bit of a tendency to reinvent the wheel.
00:16:56
Speaker
There are a few kind of canonical texts but a lot of them are on the older side. We've just gotten some new really exciting texts within the last couple of years, but a lot of them are you know 2030 years old, and so there hasn't been what we think of as kind of a natural sort of building on on the field and the way we would expect in an academic discipline and so
00:17:15
Speaker
I think having more of those conversations has really helped have conversations about sort of what is the intellectual framework around what we're doing, what does good design look like and bad design look like in a way that is more comparable to other types of research methodology. And let's just have that conversation on a level playing field where we can start to say, you know, here's how we're different from statistics or how we're different from a workshop or how we're different from all these other tools that we use.
00:17:43
Speaker
The second piece that I think is a place where we're seeing some growth and some change is this question around measurement of human decision making, right? So the whole point of games is that there are a way to get at the squishiness that's going on inside people's heads. The question then becomes, how do we elicit that from people? How do we pull it out of their head and get it into something that you and I as observers can see?
00:18:07
Speaker
And this is a place where I think the field is really kind of not made a lot of progress. If you go back and you look at the historical games from the 50s and 60s that I know John and Reed talked about in earlier, earlier episodes, you know, when I go back as a modern game designer, one of the things that's always really shocking is that.
00:18:24
Speaker
the state of data capture, the state of sort of the measurement tools that people are using haven't moved that much. And so this is a place where I think there's been a really rich conversation between practitioner game designers and social scientists, not just political scientists where, you know,
00:18:40
Speaker
me, Andrew come from, but also kind of the broader discipline, econ and psych about how do we do a better job at getting at some of these measurement problems so that we've got a better idea of what is actually going on inside people's heads. This is so telling when we run games about like escalation management, right? The number of times where the group came to one decision and then you pull individual people and the average decision that an individual would have made is different than the
00:19:09
Speaker
would have made, right? We get such strong signs of what's called process loss in the form of literature. There's lots of ways of getting at measuring that and figuring out whether that's a helpful representation of group actual real world group decision making, or whether that's a bias or laboratory effect. And we're really just starting to get into spaces where we're we're playing with that more. And Andrew, obviously, your work's really important in trying to think about some of that laboratory effect piece of it.
00:19:33
Speaker
And the third piece, and I think this is the one that most people sort of would have thought of that I would start with, but is really sort of improving the state of some of the tools that we use to do modeling of basically causal inference, right? But we think about it as outcome modeling or adjudication, right?
00:19:51
Speaker
And this is a place where there's a lot of interest in using some of the new tools that are coming online, whether we're thinking of large language models or agent based models or AI magic from the sky that's going to fix everything. There's a lot of trying to incorporate those types of tools in games to improve the underlying models to make the game scale better, faster, cheaper.
00:20:13
Speaker
I think that's important and interesting work. I think one of the places though where I keep seeing places where the state of the sort of computational modeling isn't actually particularly well suited to the types of questions that I think make for really interesting games. So I think a lot of when we're gaming is when we don't understand the structure of the problem.
00:20:35
Speaker
That is the sort of thing that a lot of these tools are also not that good, right? Exactly. So in some ways, we're leaning right into that black box that a lot of AI or generative models tend to have problems with. You know, we don't have, you know, games are not that replicable in the normal sense. And so we don't have the training data that you might expect to have at a surface level when you're thinking about, you know, games in the commercial sense, like Starcraft. And so often I'll get the question of, well, we can teach AI to play Starcraft. Why can't we teach AI to play your game?
00:21:05
Speaker
It's like, well, this is actually like a really fundamentally different type of game structure, even though the name is the same. And so that's a place where there's a lot of interest. And I think it's important that we sort of keep our eye on emerging tech and we keep playing around with it so that as things become relevant to us, we can start incorporating them in. But I do think that this is a place where sort of the technological promise of computers has sort of the promise is more than the actuality.
00:21:32
Speaker
What's sort of funny again, looking back at the historical record is that's always been true. You know, computers were used to support games in the 50s and 60s. Yeah. And there's all sorts of entertaining whining about the fact that the computer can't actually sort of keep up with the bookkeeping in the way that you might want it to. So this is this is definitely a past keeps repeating itself kind of thing.
00:21:51
Speaker
That's great. That's a great list of homework for all of us in the community to kind of be keeping track of the last question here. Well, while I have you from the practitioner side, I think, I think perhaps for some of the students in particular who might be listening to today's podcast, they might be interested in how do you, how do you become the next, you know, the next Ellie?
How to Start a Career in Wargaming
00:22:10
Speaker
Now, how do you become a war gamer or a war game designer? Can I, if you were giving advice to, to younger folks interested in the field, how would you guide them?
00:22:18
Speaker
Yeah. I mean, I think the first thing I would say is that if you look at the careers of most of us, they're really, really idiosyncratic. We've got all sorts of different disciplinary backgrounds. We came into games lots of different ways. And so there's not any one path that is both a blessing and that like any road in is not the wrong road in, but it also makes it really sort of frustrating as somebody who's trying to break into the field that there's not a stronger kind of clear road in.
00:22:44
Speaker
There are some sort of nascent folks who are starting to get more interested doing education around this space, but still a lot of it is really kind of on the job training more than it really is any type of formalized education. You can't go out and get a degree in policy gaming. And so I do think it tends to be a little idiosyncratic.
00:23:04
Speaker
The historical way that most people have come in is that they came in through hobby gaming. And so I think when you talk to older members of the field in particular, that will be one of the really strong kind of pushes that you'll get to explore that space.
00:23:19
Speaker
and there's people who find that a really valuable way in. Sebastian Bay at Georgetown is a great example of somebody who's been a really strong advocate for the power of commercial games both to help sort of as building blocks for games but also as a way to train people up.
00:23:34
Speaker
You know, obviously, from the way I'm talking, I come from kind of a different perspective. And so I'm approaching this much more as a research design approach. And so most of the people who will sound like me tend to have come in through a social science background rather than, say, an engineering background. And we tend to approach the problem more from almost like a structural perspective. Right. We're interested in questions about data collection and how it compares to other methodologies.
00:24:02
Speaker
I'll say there's also a third camp that isn't so much a road in, but is definitely a personality that you're going to encounter as you talk to people, which is that there's people who sort of treat games as a modeling and simulation approach, right? And so that's more of the like operations researcher type field. And so those people tend to come in with more of a background in military operations in particular, this doesn't tend to apply so much into the non-military ops domains.
00:24:28
Speaker
And so what you end up with is sort of three loose tribes. There's sort of that first tribe that sees games very much as an art form. It's all about the narrative of the game. A lot of the constructs is about sort of how you are engaging people through the game.
00:24:43
Speaker
You have social scientists who are all about kind of measurement and research question and design. And then you've got more of the ops researcher types who tend to be more about sort of how the game is an extension of a model almost. And so it's much more about kind of how we're representing those core mechanics and how you're kind of building out that world.
00:25:03
Speaker
And so you'll get all of those types of people. And so you end up with really wide range of backgrounds. And so I think a lot of it is sort of when you have opportunities like this podcast to hear different people coming from different backgrounds, kind of seeing which ones resonate with you, checking out the type of of work that they're doing, both in terms of subsequently, but also where are they working? I think that often ends up being that sort of feeling the elephant type feel, which is not the answer that any person trying to get into a field really wants to hear. But unfortunately, is where we are.
00:25:33
Speaker
Yes, indeed. Give me an internship, right? Yes, very good. And just in closing, do you have any quick recommendations for those that want to dig deeper in terms of a favorite book about the topic or a conference that you think folks should be attending?
Resources for Aspiring Wargamers
00:25:47
Speaker
Yeah. So I do, I mentioned Jeff Applegate's book early on in this. I think craft award gaming is, is a really great approach. It sort of compliments some of the other books that are out there. You know, if folks are sort of interested in hearing more how I think about these problems, if you go to the Rand website, there's a bunch of my publications, building better games for policy analysis is where a lot of the frameworks I've been talking to come from. So if you want to look up in more detail or have questions about that, that might be where I would point you.
00:26:15
Speaker
You know, in terms of opportunities to engage with the community, on the policy side, you know, there's the Connections Wargaming conferences that I think are sort of an important hub, but there's also sort of more informal communities that move back and forth across the policy and non-policy spaces.
00:26:31
Speaker
And so that includes, for those of you who are coming from political science background, there's a group of us who do gaming work pretty regularly at the International Studies Association's annual conference. There's also more informal groups. So there's more and more universities that have groups of interested students who are looking at gaming. And so if your university doesn't have that, that's where we have to the network of the folks who are on the show and asking us for help linking into one of those networks might be useful as well.
00:27:00
Speaker
Perfect. And Ellie, just thank you so much for joining us. Really appreciate you taking the time. We'll include the links to your suggestions in our show notes. We'll also post a list on the BRSL website. We're almost at the end of our series. For the final episode, I'll be joined by my colleague from across the Bay, Dr. Jackie Schneider, the Hoover Institution at Stanford University.
00:27:18
Speaker
We'll be talking about the cutting edge of wargaming research and practice from her perspective, and we'll also tell you how you can play some of those games with us as well. I hope you'll join us for that. Thank you, Dr. Ali Bartels, for joining us.
Closing Remarks and Thanks
00:27:31
Speaker
Thanks to Andrei Anderson and his citrus, our recording studio host, and special thanks to our amazing producer, Indari Mentin, and finally to all of you for tuning in. Until next time, I'm Andrew Ready, and you've been listening to The Risk calculus.
00:27:50
Speaker
Oh, sorry. We'll restart. I'm using acronyms again that I should use. Yeah, I was going to say, do we want at some point for me to define what an FFRDC is and what the hell an FFRDC is? No, no, it's good. Okay, cool. No, good. Okay.