Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Checklist Manifesto on The PriceWriters Podcast Epiode 52
14 Plays4 days ago
Checklists are not bureaucracy. They are cognitive safety nets. We explore The Checklist Manifesto, and how checklists reduce avoidable errors, free up judgement rather than replace it, and how teams can use them to move faster with fewer mistakes. Simple tools, big impact.
Recommended
Transcript
Introduction to Price Writers Podcast
00:00:00
Speaker
I actually love a list.
00:00:06
Speaker
Hello and welcome to the Price Writers Podcast with Jeremy Keating and me, Katrin Townsend. Join us as we explore and discuss the world of insurance related books, offering our insights and recommendations. And I'm really excited to today because I love a list.
00:00:23
Speaker
I don't know about you, how you feel about a big old list. If you have one on your fridge, if you organize all of your lives by list, or if you're the other kind of person who doesn't write a list, or do you write lists only to cross things off just as before you do
The Checklist Manifesto Overview
00:00:40
Speaker
them?
00:00:40
Speaker
I'm not going to spoil it, but I love a list as well. Well, today we're going to go into all of this, Deb, because Jeremy, you've read the Checklist Manifesto, subtitled How to Get Things Right by Atul Gawande.
00:00:56
Speaker
This sounds perfect for a planner like me. Tell me all about it. It is all about how if you want to get things right, you should make lists and then tick them off as you do the steps. I know it's fantastic, isn't it?
00:01:09
Speaker
Why should we write lists? That's the big question at the heart of this book. One of the earliest examples in the book is our test pilots flying the original Flying Fortress. So it's a big four engine plane that the American military ordered.
00:01:25
Speaker
And the very first time they flew it to check it out and say, look, we've trained these pilots in it. Here's our new flying fortress. It crashed, which was not not good actually. Several people died because of it.
00:01:39
Speaker
It was widely considered to be too complicated for a pilot to fly. So it was crushed by an exceedingly experienced pilot. And he forgot a couple of steps in setting the aircraft up that caused it to source aftertake off.
00:01:53
Speaker
Everyone felt for a while, like this is too complicated to fly, but a small group of test pilots was like, well, actually we ought to be able to find these things and they could find them, but it was super important that they never missed a single step now in setting this plane up.
Aviation and Medical Checklist Application
00:02:09
Speaker
And this is how pilots ended up with checklists. And in a modern aircraft today, it will come with printed copies of checklists and they're also available on the pilot screens.
00:02:20
Speaker
This is how airlines run their passenger planes, cargo airlines and so on. They run their planes through jet listing. Why did it come up for the particular author of this book? So Atul Gawande is a surgeon in the USA and he was researching ways to actually cut down on infection rates when doctors put in breathing tubes.
00:02:42
Speaker
So the infection rate actually in hospital being with that was about 11%. So around one in 10 people that's given a breathing line would later develop an infection. And oddly enough, these things should have been quite preventable. So everyone washed, if all the material was washed, if they followed the guidelines, took things out of the packet, didn't leave them lying around, et cetera, et cetera, you shouldn't end up with infection.
00:03:07
Speaker
So he put together a list of steps people should go through in order to put the breathing tube in and not end up with infections. So what you're saying is that these situations occur when this complexity, but not incompetence. People don't get things wrong out of incompetence because, you pilots and surgeons and processing professionals, we're all smart people.
00:03:34
Speaker
And it's not willful incompetence either. We're all trying to do our best. So it's this complexity in the system that these checklists are really trying to some of the issues that come with that complexity.
Challenges and Benefits of Checklists in Complex Fields
00:03:45
Speaker
But it feels to me, I mean, I love a checklist, but I do think even when I'm doing a recipe that I've cooked maybe two or three times before, I'm like, oh, I don't get the gist, don't need that.
00:03:56
Speaker
You kind of have to adopt like a mindset of stupidity to follow these checklists, to follow a process, even when you don't feel you need it. Do you think that in insurance pricing, we have that much discipline?
00:04:12
Speaker
I actually had similar reservations to you in the early part of the book. I was feeling like, wow, do you know what? All models are different. So can things really read across? We tend to want to be quite flexible. So although we are doing kind of the same thing each time, I think teams that run things as if it's a manufacturing process actually have often run into trouble because I don't think it is a manufacturing process.
00:04:34
Speaker
So I had quite similar reservations, but around third of the way through the check, he used an example to deal with how master builders stopped being thing. This was the builder who would go around and check all the work and decide what weight things should be, how big they should be and so on. All the little details on the ground when a building's being big to make sure that it's safe.
00:04:56
Speaker
And actually what it was replaced by, which is something I had no idea about all, is what we would consider to be the checklist that he's talking about. So when you are building a new building somewhere, apparently there are checklists for people on the ground to go through, and then you've done the job properly.
00:05:12
Speaker
And all that people have to do is check the lists. So instead of going around all the buildings and having to check the work, you have to checklist. Now you have to trust people on the ground, do the work.
00:05:24
Speaker
Okay. So there is that element of it, but the idea is that it's about governance and shared responsibility. It's actually about distributing power to people, but still having strong governance to make sure the work is done properly.
00:05:40
Speaker
Okay, so I see how a checklist could help junior analysts to build their confidence and reduce the fear of making mistakes. But can a checklist really cover every eventuality? There are going to be some off events that just you never thought would occur and you have to use a bit of professional judgment to deal with those. So how does a checklist fit into those kinds of situations?
00:06:06
Speaker
It is a really good point actually. Now, that is the difference between a checklist and getting into detailed instructions, regulations. And I agree with you that detail doesn't work in situations like that, but actually, can you step back and think about what is a short process that you could go through in that situation to make sure that the work is done properly?
00:06:29
Speaker
Okay, so we're talking about small set processes, BAU, fairly standard procedure. That is when the checklist really comes into its own.
00:06:40
Speaker
So how does this really work? And does that empowerment actually happen? but I mean, I can see how a checklist could help junior analysts, but do we actually see that it does? I have some elements of me answered yes to that question, but it's not, I think universal.
00:06:58
Speaker
So confusingly, he used an example about how Walmart handled the Katrina event back, um, 2005. Oh, well done. Okay. Bye.
00:07:10
Speaker
So the author says they handled it better than FEMA because they gave a lot of power to people on the ground. So for example, managers could just break into their own store and give out the drugs that were in the pharmacy, for instance, whereas FEMA had lots and lots of steps to go through for people to actually really do anything.
00:07:29
Speaker
Now, I didn't think that was a good example because I don't really feel how the checklist comes into that one, if I'm honest. this But I think he gave a better explanation. So Van Halen, when they used to tour, would insist as in they had a big list of things that the venues had to do.
00:07:47
Speaker
And a lot of them were related to safety because they've been touring with a massive amount of equipment and people. So they really needed all of these steps to be followed. And buried in amongst the stats was that they wanted all of the brown M&Ms removed from bowls of M&Ms that they had around backstage.
00:08:07
Speaker
Now, I've heard about this before as an example of crazy over-the-top pop stardom. That's not why they actually did it. And they kept quiet for years about why they actually did it.
00:08:18
Speaker
So the reason they did it is because they would arrive and be like, if they see any brown M&Ms, They would go around and check all the equipment. So they had people that did it, but everything would have to be checked and rechecked to make sure that the venue had followed the guidelines.
00:08:34
Speaker
And if they found no branding at M&Ms, they were like, people have followed the lists. And actually these were big, massive setups. So there are times when it does work for what would be considered massive.
00:08:47
Speaker
things and it's kind of up to you what level you set this up because your checklist can be held but it can also be a much higher level.
00:08:57
Speaker
But we're not talking about just setting up equipment. So right now let's jump back to talking about insurance pricing.
Effective Checklist Implementation
00:09:07
Speaker
couple of Van Halen references for anyone listening who is old enough to remember that.
00:09:11
Speaker
What kind of situations in insurance do need a checklist and how do we go about creating a really strong checklist for a big organisation?
00:09:23
Speaker
That's a spoiler. Is it? One of the letdowns with this book is that it's got very little on how to actually create checklists. So in fairness is described as the manifesto. So it's about why you should do them. It's not a how do I create a checklist situation.
00:09:40
Speaker
But there are two types of checklists that he describes. So there's read and confirm and there's read and do. So read and confirm is just to read through it and have you done all of the steps thing.
00:09:52
Speaker
Read and do is read through it and do the steps as you go. So it's quite there. He does say that it should be things with sort of five to nine steps. So that's something you can easily go through and easily check. You can imagine a committee can easily look at whether you've done five to nine steps.
00:10:10
Speaker
You ain't going to get them to do more than that. That's not something you can do in a committee. they In fact, nine's pushing it through. Thank you. So it fits on a page. It's super easy to read.
00:10:20
Speaker
Now, one of the very important things here, which comes in with the building example, actually, is that these only really work if they're involving teamwork. So if you give one person a checklist to do and they tick off that they've done it and give it back to you, that is not effective governance.
00:10:38
Speaker
Okay. So you check it together. So have you done this and you work through it and you tick it. So that is affected governance because you know, at least two people have checked these things when you actually hand in your checklist for people to look. And it's only what's essential.
00:10:55
Speaker
So actually I can see times when you would be like, have you reconciled the data? It is essential. If you're giving me a model, I'm going to want to know the data been reconciled.
00:11:08
Speaker
And once in a while, no matter how diligent people are about at this, just like with breathing tubes, they will forget that. They will be busy. They'll be rushed. They will forget that. And then it will be like, the model doesn't work. Why doesn't the model work? Oh, the data wasn't reconciled.
00:11:22
Speaker
So actually i can see this working. Okay. So you think this really could work in insurance, even though we don't quite know what the perfect list looks like. What is the last a totally avoidable mistake that you saw in an insurance pricing team that you think a checklist would have prevented?
00:11:41
Speaker
Oh, well that's an interesting question. it' that Okay. I usually talk about hypotheticals, but you've asked me for a specific case. They imagine a situation where live, some postcode lookup tables haven't been set up properly.
00:11:59
Speaker
and thus every quote is returning the default level, which is the lowest level. Ooh. Not even a mid-level of default. Nope. It's the, the unknown is going to the lowest level. Okay.
00:12:15
Speaker
There are several places where a checklist would have prevented that. Tell me where. So firstly, are the unknown set to a high level? Should the lookup fail?
00:12:27
Speaker
Is it going to return a high price? Yes. so To be honest, I think just that one it is good enough actually. That's a check that ought to be in most people's systems. So is that set up correctly? And to us, that is quite a high level thing. So if something breaks, is a high price going to be returned?
00:12:44
Speaker
So that ought to be included in the testing. This particular case, which went wrong because of the translation between analysis and live actually. So I can't think of other places where it could have been stopped by just doing bit more checking.
00:12:59
Speaker
And it could have been high level. The problem's in the detail, but the checking can be in high level. Wow. Yeah. mean, know that does sound like a big problem where a bit of a checklist would have helped.
00:13:09
Speaker
One reservation that sometimes occurs with checklists is someone will write a checklist for very good reason. And then they think of some more things or someone else adds something to their checklist.
00:13:22
Speaker
You know oh well don't forget this situation or we need it to include this situation or whatever it is. And these checklists then become too big. They've got these like bloated long documents or they're outdated or there's some so steps you will say yes to even though you don't even use that system anymore. anymore How do you stop checklists from becoming outdated and ignored? How do we make checklists actually useful?
Underutilization of Checklists in Industries
00:13:49
Speaker
Yeah, so I'm going to say something that's probably going to upset some people, right? So generally, there are not very many plane crashes in the world. Okay?
00:14:01
Speaker
I think a lot of people deploy mistakes in their prices. Okay. Is flying a plane harder than deploying correct prices? So I'm a bit like you. i was a bit like, can we really do this? Is it going to work really on the ground? Do you know what? They have perfectly good governance inside airplanes that mean that these checklists say about flight nine items.
00:14:23
Speaker
They start with things that are actually useful at advice, like fly the plane. Cause actually apparently people can get so obsessed with a problem that they forget to fly the plane. Yeah. It's happened sometimes you can get so focused on a problem that neither of the pilots flying the plane.
00:14:38
Speaker
Wow. Yeah. Okay. I mean, yeah that does seem like a good instruction, doesn't Yeah. So actually you've got a deployment error, right? If you've had a checklist that you go through for finding the problem with it, that's just a few steps, but remember you're a governance committee. You don't want to be in the detail every single thing.
00:14:58
Speaker
If you've agreed with your specialists, what six to seven items you need to know that they have done. And it's not one person telling you they've been through it to people.
00:15:10
Speaker
Actually, I feel that's surprising effect of governance. Now, interestingly, the conclusion towards the end of the book is that checklists are not adopted enough. People do not like them.
00:15:22
Speaker
So only 10% of hospitals across the board actually adopted these things despite the infection rate dropping from 11 to zero. Oh, he calculated how many human lives would be saved by people actually using checklists. And it's hundreds of that. It's a lot of people.
00:15:38
Speaker
A lot of people are dying because people don't like using checklists. Now pilots use it, fine. They've adopted it. They know that it works, but hospitals, it's interesting. So I think hospitals, more like insurance companies, big bureaucratic operations kind of don't want to use these things.
00:15:54
Speaker
However, he used the example of surgical robots, which have been adopted everywhere on mass at the cost of billions of dollars. And despite a huge hype around them and the cost, they've actually had very little impact on surgery.
00:16:08
Speaker
Because generally the surgery is too complicated for the robot. So actually they're not used that much despite a massive amount of resource having everyone been put into them. And if people had adopted checklist with the same gusto, lots of people would still be alive. So I found that quite stunning actually.
00:16:28
Speaker
Yeah, I mean, that is so great, isn't it? That just following instructions is such a big improvement in very complicated circumstances as well.
Practical Benefits of Simple Checklists
00:16:39
Speaker
But bringing it back to insurance, you've talked a lot about governance committees and also about deployment in particular. Now, some people might not be on a governance committee. Wow, also, you're living the dream.
00:16:52
Speaker
Some people might not be involved in the law of deployments. So for anyone in a pricing role, what is the smallest, most useful checklist that a pricing team could implement tomorrow, but not in a governance deployment space?
00:17:10
Speaker
You know, it's really annoying because he doesn't use this example, but I thought about it. I was like, you could have a checklist before you send an email. Okay. So how many people have sent an email without the attachment?
00:17:22
Speaker
yeah You can't do that anymore on modern email. It says, go on to send it. No, it's not perfect though, is it? Only if you say it touchlin' does it know. It's not genius.
00:17:34
Speaker
But you know, there's other things. Ask yourself, as someone who checks work, what things do I always ask people before I even want to open the spreadsheet and look?
00:17:45
Speaker
What are the things you always ask an analyst? Yeah, I actually have a list of them in my book. in Oh, there. You you? I literally have a webinar. You do. You do. So you've done this. Yeah. For me, it is you always want to check check your 5M. So that's minimum, maximum, mean, missing and most common values. Double check what those are. Do they make sense for the situation?
00:18:11
Speaker
i always do a sense check based on some other information. Some people would call this a reconciliation, but I mean like at the most basic. If you have seen a presentation that says roughly that your product's GWP 200 million, then you know if your premium has that many zeros or not, you know, and you have to get into the the little itty bitty reconciliation there, but just does this number even in the right ballpark? I remember one of my graduates sent me a spreadsheet once and I opened it and I looked at one number
00:18:46
Speaker
And I went, nope, and sent it right back to him because he had suggested that a burn cost for a single Land Rover was 1.2 million. on I know that i'm I'm not a big Land Rover fan necessarily, but what had happened is he'd done frequency times severity times severity to give
Checklists as Experience Replacements
00:19:03
Speaker
you the burn cost instead. So he'd like...
00:19:05
Speaker
massively overshot it which is that one like sense check the union right ballpark that's always something i do and the check to last time if you have ever produced a piece of work like this before just double check what your numbers said last time as a rough estimate i think so often we do feel that this expert judgment that our seniors have is some like magic power some magic experience something that they've gained from all their years of wisdom. And actually it's not.
00:19:38
Speaker
A lot of the time, we're just following a checklist of things that we know have gone wrong in the past and and what would have prevented them. Experience can 100% be learned.
00:19:48
Speaker
Gut feel can be memorized from a checklist. So I am with you on this. More just sense checks on your data. That's the ideal checklist for me.
00:19:59
Speaker
All right, so what I suggest, give your senior analyst that list, okay? And before a spreadsheet gets sent to me as the manager, want one of the senior analysts to sit and ask you if you've done these five things, okay?
00:20:14
Speaker
They might not enjoy doing it, but how much time are you going to save? Yeah. Or yeah even get the juniors to do it as well. Get the juniors to ask each other. Because juniors are going to ask each other We always talk about how important peer review is.
00:20:27
Speaker
Yeah, actually do it, you know? yeah People use it in the building trade. They use it to keep your planes flying. Hospitals don't like using it. And therefore, it's bad.
00:20:38
Speaker
And I can imagine a lot of our teams will grumble about using it. But actually, it's pretty effective. Yeah, it sounds like that you'd get the best of both worlds to bring in one more Van Halen reference.
00:20:53
Speaker
Oof. Yeah. I think I only know Jump, so, and I didn't live to him era. Okay, wow. Okay. So Jeremy, this is like our Inville podcast
Critique of The Checklist Manifesto
00:21:02
Speaker
checklist, right? We always ask each other this one question. Would you recommend the book?
00:21:07
Speaker
No. Oh. Okay. It's way too long. It's such a basic idea. you can cover this in an article. It's quite a long book and every part of it is really about telling you why you should use a checklist.
00:21:22
Speaker
There is a section that's about making checklists, but even that just feels really light to me. I pretty much gave you this summary of what you should do minute ago. So there's way too little information on how to actually make a list, way too much on why you should have them.
00:21:37
Speaker
I think my advice is adopt checklists, but they read the book. Perfect. That sounds exactly the kind of summary I like for a book. Good job. Great.
00:21:48
Speaker
Next time we will be talking about Murky Water. This is a book I went a little bit rogue with. So I think you're going to, again, really enjoy this one. Slightly off topic. So join us next time. In my head, the music started playing.
00:22:03
Speaker
Jump. No, not that one. Our podcast music. Oh, okay.
00:22:20
Speaker
PriceWriters transforms price professionals into the most respected leaders in insurance. Find out more about our capability audits by visiting pricewriters.com.