Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
AI, Dashboards, and Human Decisions: A Conversation with Melanie Tory
322 Plays3 hours ago
In this week’s episode, I talk with Melanie Tory, Professor of the Practice at Northeastern University, about how people actually use dashboards in the real world — and why that use often looks very different from what designers intend. Her research reveals that dashboards frequently serve as a starting point for accessing data rather than tools for answering questions directly, with many users simply exporting data to Excel to do their real analytical work. We also explore her work on AI-enabled healthcare systems designed to help clinicians monitor patient risk in intensive care units, including how to visualize uncertainty in ways that busy medical teams can process quickly. And we close with a look at her emerging research on how people are beginning to use generative AI tools for data visualization tasks. It's a thought-provoking conversation about the gap between the tools we build and the ways people actually work with data.
Subscribe to the PolicyViz Podcast wherever you get your podcasts.
Keywords: data visualization, dashboards, dashboard design, dashboard usability, data analysis workflows, Tableau dashboards, Power BI dashboards, human data interaction, Melanie Tory, data communication, dashboard research, analytics tools, business intelligence dashboards, data storytelling, data workflows, PolicyViz Podcast
Become a patron of the PolicyViz Podcast for as little as a buck a month
Visit Melanie's webpage at Northeastern University
Follow me on Instagram, LinkedIn, Substack, Twitter, Website, YouTube
Email: jon@policyviz.com
Recommended
Transcript
Introduction to Melanie Torrey and Dashboard Efficacy
00:00:13
Speaker
Welcome back to the Policy Viz Podcast. I'm your host, John Schwabisch. On this week's episode of the show, I am joined by Melanie Torrey from Northeastern University. We talk about dashboards. Now, hopefully you've seen my post on dashboards.
00:00:29
Speaker
I am not a big fan these days. I think the way we are... Consuming media, mostly on our phones, means that dashboards and data tools are becoming less and less useful to reach those broader audiences.
00:00:43
Speaker
Different from internal dashboards, which have a specific use case and specific audience, but I reached out to Melanie because she has a really interesting paper on how people who are using dashboards internally for their work actually use them, or in many cases, don't use them. So we first spent a lot of time talking about dashboards and this particular study and Melanie's perspectives on dashboard use and creators. And then we've transitioned to some of her work, more recent research on AI, LLMs in different contexts. And I think you're going to find it really interesting. She has some interesting early findings and different perspectives on AI and data visualization specifically, and in the particular use case of the one paper that we spend a bit of time talking about.
Relevance of Dashboards in Modern Media
00:01:27
Speaker
Really interesting conversation. I think you're really going to enjoy it. If you're a dashboard user dashboard creator, if you're thinking about how AI is going to impact your life or your healthcare, or as a data visualization practitioner, this is the episode for you. So here we go. Here's my conversation with Melanie Torrey and you only get it here on the policy viz podcast.
00:01:49
Speaker
Well, hello, professor. Hello. Thanks for having me on the show. but oh It's great to have you. Thanks for coming on. It's been a while since I saw you last. I don't even know, did I even see you at at Viz?
00:02:02
Speaker
I was at Viz. I think I might have seen you there, but yeah I saw a lot of people there. A lot of people. lot of people. And at one point, a lot of people in a very small bar, which which was super entertaining. But yeah, I spent a lot of time. i think that was the West Coast Viz Party. I think the organizers were trying desperately to get rid of their drink tickets so they could go home or back to the hotel. so That's pretty funny. I stayed for a while, but the whole thing, I was pretty tired.
Research Focus: Dashboard Usage and Interaction
00:02:34
Speaker
ah um
00:02:36
Speaker
Okay, so you've got a lot of work going on There's one particular paper of yours that's now ah you know two three years old that I i definitely want to talk about about dashboards. um But why don't we start with introductions for folks who are not familiar with you or not familiar with your work? So you know where are you and and you know what does your work tend to focus on?
00:02:57
Speaker
Yeah, sure. So I'm a professor of the practice at Northeastern University. and i'm based at the Rue Institute, which is Northeastern's campus in Portland, Maine.
00:03:08
Speaker
Northeastern actually has a bunch of different campuses all over the place. So this is one of them. And we're a little bit unusual, I suppose you might say, in that we focus on three things. One is research. So I lead a research team here, but we also have graduate education and we have an embedded entrepreneurship accelerator. So that makes us a little bit different as a university.
00:03:33
Speaker
And part of our mission is to work really closely with industry and foster a bit of economic activity in the state of Maine to help grow the tech economy here.
00:03:45
Speaker
So that means that my group does sort of pure research in data visualization and human data interaction. But we also do a lot of applied work with company partners, nonprofits, and so on to try to do a little bit more applied and practical things than you might get in a normal university.
00:04:05
Speaker
And does most of that work with the business communities that tend to be in Maine and New England, or is it just everywhere? Could be anywhere, but we've mostly focused, yeah, Maine and New England. Yeah, yeah. Okay, that makes sense. That makes sense. um Okay, so I reached out initially because I've had this question in my head for a while about whether dashboards are worth our time to create. um And I found your study with Lynn Bartram and others, Finding Their Data Voice, Practices and Challenges of Dashboard Users. um
00:04:41
Speaker
And then I recently posted a blog post on my thoughts about, you know, why, especially now in our current media environment, why I don't think they're they're very useful. So um I want to dive into this, but I want to sort of maybe ask you first to give folks a summary of the work so they know kind of where we're coming from.
Design Challenges and User Needs in Dashboards
00:05:02
Speaker
Sure, so this particular study was an interview study we did with people who are users of dashboards. So these are, and and this was a particular dashboards that are internal to organizations, so this wasn't looking at really at public facing dashboards, which might be a little different, but we were curious to know how are people actually using dashboards in practice?
00:05:29
Speaker
And is that actual use aligned with the way that the visualization community and the visualization tool developer community is thinking that people are using dashboards? yeah I was working at Tableau Software at the time, so we were a dashboard building company. And we had a lot of contact with analysts who build dashboards because they were our direct users.
00:05:55
Speaker
But we actually didn't have much insight into the people who were ultimately using the dashboards because those were in some ways a hop away, right? There are our customers' customers, if you will.
00:06:08
Speaker
And so we wanted to know, you know, Are the dashboards that they get meeting their needs? Is it, or is there more we could be doing to support these folks in working with data?
00:06:20
Speaker
Right. Okay. And so this was primarily a qualitative study and you talked to, what was it about, like 20, 24 people who are in a variety of sectors.
00:06:32
Speaker
Right. It was 20-ish people and rough the majority were dashboard end users, I think, maybe seven of the 20 were analysts who were building dashboards because we also wanted the perspective of that what did analysts think that they could be doing to help their users be more effective.
00:06:55
Speaker
right And so, yeah, it was across variety of sectors, across a variety of tools. We didn't limit it to Tableau. We were interested in dashboards of any kind. dashboards, whether they're Power BI or Tableau or custom built, largely all kind of function the same from the end user's point of view anyway, so it didn't matter. yeah And yeah, we were curious to know what works, what doesn't, and are they being used the way we think they are? so I want to give you a chance to talk about what works and what doesn't, what the bottom line was, but I do find it
00:07:29
Speaker
striking that Tableau, but I'm guessing this is similar for lots of dashboarding companies and you know beyond Tableau and Power BI, that like the even though the user is two steps away, that that's still not a focus issue.
00:07:46
Speaker
of the I mean, i'm I'm guessing like the marketing team sort of focus on it, but like, I just find that fascinating that like, the like people still have to click, like, does the checkbox work? Does the drop down work?
00:07:59
Speaker
Like, I'm not sure I have a real question here for you. But like, I just find it fascinating that that was not a focus of the of the company. Yeah, I'm with you on that. It was surprising to me too. yeah but in some ways it makes sense because the people who are deeply using the product are the analysts using the desktop product to build the dashboards.
00:08:19
Speaker
And we can kind of assume that they know what their end users needs
Case Studies: Telecom and Dashboard Design Issues
00:08:23
Speaker
are. Right. And so the they're also a harder population to reach, right? Because we have all of the people who were in the database are all the analysts and the data admins and so on and right not the end users. Right, right, right. Yeah, that that totally makes sense. and they're And they're the ones that are paying for the product. So I get it. It's just, yeah, anyway. Okay, so what's the bottom line here? What did what did you all find?
00:08:51
Speaker
Well, what was super interesting was we kind of found that dashboards weren't being used the way that everybody in the biz community thought they were. Right? We all kind of thought you the user will go to the dashboard and answer their questions. And the dashboard you know the dashboard designer, that's their role, is to design the dashboard so that all the user's questions can be answered. and so We, you know, the model prevailing mental model was user goes to the dashboard, answers their question, all is done.
00:09:26
Speaker
but what we really found out is that it was almost like the dashboard was the portal to the data. Like that was the place that the user started. They go there, get their data, and then they do whatever they need to do with it.
00:09:41
Speaker
And there's some questions they can just answer there. That's great. But there were a whole bunch of other things that people wanted to do that The dashboard was just the starting place. I mean, so do you think that there's a fundamental difference between these folks who are using these for their internal work purposes and a more public dashboard? And I'm just going to pick one.
00:10:08
Speaker
just because it gives us a grounding. So the OECD had this better life index. It was a custom build. It was a beautiful thing. you know It's really cool, but just to give us like something to think about. So that was on a website where you could you know move sliders and you can pick and you can filter and you know sort of your your basic thing. But that was developed for the public to use. Do you think the folks in your study that their behavior is fundamentally different than someone going to the OECD website?
00:10:39
Speaker
I think only in that the folks in our study were kind of maybe more purpose-driven, like they had to use this data to get part of their job done. Whereas that might be true for some people going to that OVC dashboard, but a lot of people might be going there just out of interest. yeah And it's it's not of critical importance to something they need to deliver on.
00:11:05
Speaker
right That's probably the major difference. That makes sense. Yeah. So these folks who have a task, they need to answer a question or a set of questions.
00:11:15
Speaker
they're primarily using these dashboards as as an entryway to actually just get to the data. Do you, did you find that it, like, what was the reason that that's what people were doing was because the dashboards weren't designed well, was because,
00:11:33
Speaker
you know I remember there was one person that was like doing gym membership stuff, so I'm guessing that person isn't like you know like a computer scientist or like a PhD in stat. No shade on people working in gyms, but you know you know not a know deep data person. um Yeah,
00:11:53
Speaker
Yeah, I think that was true of most of the people in our stuff, right? These are folks who, you know, data's got to be part of their job, but it's not their day-to-day life. It's not everything they do in their job. And for the most part, they're not data experts.
00:12:09
Speaker
We had a couple of folks who had been data experts and then moved their way into leadership roles. So they did have that expertise, but no time. But most of the people were average folks who'd you know, weren't data experts, but had to use the data in some way to answer questions, to build reports for their managers, et cetera, things like this.
00:12:33
Speaker
And yet they're still going to the dashboards for the most part and downloading the data and exploring it or making the graph or the slide that they need to make. Yeah, that was a super common theme. We called it the data dump, actually. Yeah. And even some of our participants called it, you know, taking a data dump. You go to the dashboard because that's where you get your data. yeah dump the data out into Excel or some other spreadsheet, and then you can answer your questions with it.
00:13:03
Speaker
We even had some analysts tell us that they had built dashboards that were
Adapting Dashboards for Diverse User Needs
00:13:09
Speaker
essentially data dump interfaces. And they were almost embarrassed to tell us that this is what they had used their tool like Power BI or Tableau to produce because that's not the intent of those tools, but it, it served a need for their users, right? Their users have a really simple way to go and do what was essentially a database query, but without having to know any SQL, get their data and then it's flexible and they can do whatever they want with it.
00:13:39
Speaker
Sure. Right. So they're, so they're using it as a way to make the data more, familiar feeling and looking for the end user, not necessarily for the purpose of exploration.
00:13:53
Speaker
Yeah, it was a real mix of things, right? Like there were a lot of questions that people had that the dashboard was designed well to answer using all the visuals that you would expect in a dashboard. And and those scenarios are great.
00:14:08
Speaker
But then there were also questions, analytical questions people had that the dashboard wasn't designed well. to be able to meet. So that would be one reason that they would have to go through awkward workarounds or dump out the data to try to answer those questions.
00:14:26
Speaker
And there were a whole lot of use cases that didn't involve analysis, but rather sort of communication where, for example, they would have someone would have to get the the numbers for the current month for their sales, let's say.
00:14:43
Speaker
and repackage that in some way for leadership or for distribution throughout the organization. and that might mean you know rolling it up to a higher level of detail because leadership doesn't want to see all those detailed numbers. They just want the big picture. It might mean sort of spinning a story around those numbers. right like I don't just want to tell you what the number is this month. I want to tell you my take on yeah why it is that way. What was going on? Like, what's the context?
00:15:15
Speaker
I want to repackage it maybe, make it look prettier or make it look different for my audience. So so they were doing a lot of this kind of repackaging and communication of the data in addition to just answering questions with it.
00:15:29
Speaker
So I want to dig in on just sort of these ah two of these groups um on the dashboards that were well-designed and then on the communication side. Because I think a lot of people would say,
00:15:41
Speaker
a poorly designed dashboard is therefore a bad
AI in Healthcare: The HEART Project
00:15:44
Speaker
dashboard. um But it sounds like in in your study, at least, even the well-designed dashboards didn't quite meet the user's needs. So do you think that is just not understanding your end user? Is that just dashboards don't quite do what we think they are supposed to do? Like, what do you think is going on with that group?
00:16:09
Speaker
I think it's kind of both of those things depending on the use case. So I'll give you one example of a dashboard that I think was not well designed for the purpose that the end user was using it for.
00:16:22
Speaker
So this one ah this one user worked at a telecom company and had to get monthly numbers, kind of the use case I was talking about before. And so the dashboard that she was using was designed to give all of the numbers for a given month. So she could go in, set a bunch of filters, set it to like, I want to see the numbers for January. And she's got all the numbers there she needs.
00:16:46
Speaker
But what she really needed to do for her report to leadership was to compare the current month to last month and show the difference. and This particular dashboard would only show you the snapshot of one month at a time.
00:17:01
Speaker
so She had to go through this really super awkward workflow of loading up January, set that filter, write down all the numbers either in a notebook or in a separate spreadsheet manually. wow Change the filter to February, write down all those numbers, and then compute the difference.
00:17:20
Speaker
And that's a ah really simple thing that the dashboard could have been designed to make that direct comparison, saving her tons and tons of work had the dashboard designer known that it would be used for that comparative use case. So that, I think, is a case of failure on the dashboard designer's part.
00:17:42
Speaker
at least to understand who are all of the users of this book dashboard and what are all of the ways of using it. Right. and And do you think that also applies to the users who needed to create some sort of other product, more of the narrative storytelling briefing, books, slide deck. Like, I guess if person A is the dashboard designer and they know that, you know, person, person's B through Q have to make PowerPoint slides out of the dashboard. I feel like you would build that dashboard in a way to facilitate that end use, but also like maybe not.
00:18:21
Speaker
I think that that's true to some extent that, that maybe they could have been able to do that a little bit more. But I also think that this is a place where our current dashboard tools just fail to support the kinds of flexible use that the end users would really like to have.
00:18:39
Speaker
right? Like, wouldn't it be nice if you could go into your dashboard and as an end user without having to go into some kind of deep edit mode and understand all of the inner workings of the tool that built the dashboard, could you just like change the encoding of a bar chart into a pie chart because that's how you want to see it?
00:19:02
Speaker
yeah Or could you change the color scheme to make it what you think is beautiful for your end users? Or could you filter out parts or do roll-ups? Like there's a lot of, or add annotations, add story around it. Like if they had more of this flexible sort of mash it up functionality to them, then these folks would have been able to do a lot of more of that work in the dashboarding tool without having to sort of fail out and go back to a more manual process in Excel or PowerPoint or whatever else.
00:19:39
Speaker
Right. So the last question on this paper I wanted to ask you is a little beyond the study because your study was focusing on internal use cases. But did the work lead you to have more thoughts about dashboards for external purposes? So we could go back to the OECD or just like
Exploring AI's Role in Data Visualization
00:19:58
Speaker
Tableau Public, right, or or whatever. like Did it give you any thoughts or insight on how you think people are using dashboards sort of generally speaking?
00:20:09
Speaker
I think it probably speaks to dashboard use in general, even though we studied the more narrow population. My suspicion is that dashboards are useful for a fixed set of ah functions that they They support, right? They're good at distributing information to a wide audience. They're good at being a sort of source of truth, right? They document the state of the data at the current time or perhaps through past history, right?
00:20:45
Speaker
They're good at those sort of circulation of information functions. They're not necessarily great tools for data exploration, certainly not for answering novel questions that the designer didn't think of. And that designer is probably not going to think of all of the possible questions that someone might want to answer with their dashboard. Right. And so it's just, it's sort of a, it's a property of the way dashboards work.
00:21:16
Speaker
Yeah. They're are a little bit inflexible and fixed to a, a small set of things that they do well and a bunch of other things that they just don't do or don't do well.
00:21:27
Speaker
Yeah. i mean, one of the points that I made it in my post was that, you know, more than half of people now, especially globally, but even in the United States are are primarily using their mobile phone for their internet access. And I just not sure that dashboards,
00:21:46
Speaker
I might even say i'm not sure. Dashboards just don't work on mobile phones, right? Like that's just not, they're just too small to do all the filtering and searching and dropdowns, stuff like that. And um I wonder if you think, and we'll talk about AI in a second, because you know we've got to talk about AI these days, but I wonder if you think that the way in which consumption
Conclusion and Further Engagement
00:22:08
Speaker
of data and media is changing changes how we should be thinking about creating dashboards in this public for this public use?
00:22:18
Speaker
Yeah, I, like I said, I think dashboards serve their purpose. no But we shouldn't assume that they're going to meet all needs of people with data like people are way more flexible and interesting questions that we can't anticipate.
00:22:35
Speaker
And a dashboard can't handle that. Right. Yeah. um Okay. um Really interesting. i'm sure there's a lot of dashboard creators who are yelling at us right now and shaking their fists. So we'll we'll hear from them later on. um I want to turn to some of the the newer work that you're doing. um You've got kind of like two very different papers I wanted to ask you about. One is the heart interface, which I think has...
00:23:07
Speaker
An abbreviation. Yes. Healthcare enabled by AI in real time. So this is going to get us to a number of different things at at one point. So why don't we start there? Who is that project with and and what is that project focusing on?
00:23:20
Speaker
Yeah, that's a really big collaborative project I've been working on since I started here at Northeastern. It's a collaboration between us here at the Rue Institute and a a local hospital, Maine Health, as well as a ah healthcare care technology company, Nihon Code and Digital Health Solutions.
00:23:41
Speaker
And this project started with the idea that we might be able to enhance patient care in intensive care settings, particularly the cardiothoracic ICU.
00:23:53
Speaker
where all patients go to recover after they've had open heart surgery. And the idea was, could, is it possible that an AI would be able to help the care team better care for these patients by predicting ahead of time, if they were going downhill and heading for some adverse outcome, could we predict that ahead of time before the humans could have e let the humans know,
00:24:23
Speaker
And then maybe they might be able to intervene earlier and and save some of these patients from pretty nasty outcomes. right And so this project was all about, yeah I'm not an AI developer at all. I couldn't build this kind of predictive AI. But fortunately, I have great colleagues who can. And so the idea was to build an AI predictor, build an interface around that. That was my team's job that could then be ultimately deployed in the ICU and tested through a clinical trial, which we haven't got to yet. But that's ultimately the goal to see if humans plus an ai could have more effective outcomes than humans alone.
00:25:06
Speaker
Interesting. And so this is doing predictive analytics on the real time, i guess, healthcare care or health information from the patient. Exactly. So it's, it's being fed the patient's health record, any vital signs, data that's collected, all that stuff you see hooked up to, you know, heart rate monitors and so on in hospitals, all of that information is being fed in, in real time. and used to make predictions about a range of possible negative outcomes like heart failure, kidney injury, and so on.
00:25:42
Speaker
And is there a viz component to this? Because because the way you've described it sounds like there's a big predictive analytics challenge. There's a big AI sort of data ingestion problem. There's obviously like communicate it to the healthcare care workers, but like, is there a viz part to it too? That's like trying to pull it all together?
00:26:01
Speaker
Yeah, because that bit about communicating it to the healthcare care workers is exactly where the biz comes in. Yeah. So what we ultimately wind up with is a data set that is changing over time of all of the patients in the ICU, their current predicted risk scores for a range of like nine or 10 different possible negative outcomes and uncertainty information.
00:26:28
Speaker
about the best likely outcomes, because what can happen is that, you know, the algorithm might be predicting that it's highly likely that they're going to go into a septic shock, but maybe there's missing values in the patient record, like lab tests that haven't come back yet.
00:26:46
Speaker
And we know that the the result of those lab tests could affect the score up or down. And so that leaves us with a range of risk scores that have to be communicated as along with the risk score themselves. So it actually becomes a pretty big data problem. It's just that there happens to be an AI in the loop generating all this data.
00:27:07
Speaker
Right. um So the uncertainty piece has got to be pretty important here. Yeah. And communicating uncertainty to even healthcare workers has got to be pretty a pretty hard nut to crack. So how are you, at least it sounds like kind of early-ish stages, but how are you thinking about communicating uncertainty metrics of uncertainty to, I mean, I'm not even say non-data people, because I think healthcare workers are are pretty steeped in data all the time, but they're also really busy. um yeah Sort of like understand, at a glance, understand this like, you know, median number, but also that it's plus or minus, you know, blah, blah, blah percent.
00:27:51
Speaker
Yeah, that was exactly the challenge that we tackled in this particular paper is how to convey those the risk scores and their uncertainty values around them.
00:28:01
Speaker
And one of the first things that we learned in talking to our collaborators at the hospital was that they didn't really want to see a raw number score.
00:28:12
Speaker
They like to see the number, but what was more important to them was to classify it into sort of threshold categories, right? They think in terms of patients are high risk, medium risk, or low risk. Oh,
00:28:26
Speaker
They, and they think about this across a bunch of metrics already that they monitor in the hospital and they even give them colors. Right. So that's like red is always high risk. Yeah. Right. Green is low risk. We got away from the red, green color scale and our interface, but they even think about these categories as colors, right? They talk about red alarms, right? An alarm rings and it's a bad one about something dire. Like that's a red alarm. Yeah, yeah.
00:28:55
Speaker
Code red versus code white versus code brown. Yep. I remember those. Yeah. Yeah. I forget where we were going with this question. I think you were asking about we conveyed uncertainty. Uncertainty. Yeah. Yeah. So for their purposes, it's not so much plus or minus one standard v deviation. it's is this high, medium, low, like places them in a bucket.
00:29:18
Speaker
Yeah, exactly. And so what that got us thinking about is it's not when we convey uncertainty, it's not so much, you know, plus or minus five points. What matters is, does the the window of uncertainty potentially cross the threshold into the higher or lower category?
00:29:39
Speaker
right So what would be quite worrisome is if a low risk patient could actually be a medium risk patient because we're missing data about that person. So that was kind of the critical thing that we wanted to convey. And so we kind of simplified down the way that we were conveying uncertainty to focus on those boundary crossings.
00:30:05
Speaker
And what we ended up doing in our interface was creating these sort of pills. For each patient, we would show a pill for each possible negative outcome that they might be at risk for filtering out all the ones that aren't a problem for that patient.
00:30:20
Speaker
So like, let's say they're at risk of renal failure, we have a little pill with the name renal failure on it, and it's colored based on their risk category. And then we would append to the end of that pill, a little blip of color of the neighboring category if the uncertainty could possibly cross the boundary. ah So that's how we made a sort of like super duper simplified representation of uncertainty for this. Interesting.
00:30:49
Speaker
Interesting. um You might not be able to answer this question, but it is a question that everybody's talking about, which is, are all of our jobs at risk because of AI? And the way you describe this project is interesting because...
00:31:05
Speaker
It actually sounds more like a tool for health care workers to use and sort of summarizing a whole bunch of health data. It's not replacing anybody, but like these sorts of tools, do you think that maybe not far enough along in the project to actually talk about this yet, but I'm curious whether you think this sort of project, this sort of tool,
00:31:27
Speaker
will facilitate job loss or job gain? Because I could see it going kind of, or nothing or or neither, but I can see it going in all sorts of different directions. Yeah, we were pretty deliberate about this when we were conceptualizing this project, that this was never intended to be a tool that would replace healthcare care workers or their decisions. we We were even really careful that, you know, it's not even making direct care recommendations, it's not even suggesting actions that they can take.
00:32:00
Speaker
It's meant just as another information source. It can help them think about what's going on with their patients so that they can hopefully make better decisions.
00:32:11
Speaker
And that was a deliberate choice on our part to keep the health care workers in charge. Right. Like to us, they are the decision makers here. yeah They're going to be in charge. Our job is to inform them with more information.
00:32:26
Speaker
right I don't know about you, but if I'm in the hospital after cardiac surgery, I want the experienced physician character. I don't want an AI making decisions. No, right. That's right. That's right. Yeah. The person who has to cut my chest open, that's the person I want making decisions. Yes. 100%. You have one more project that is work in progress, but just wanted to let you mention it has a very cool title, Vibe Modeling. um yeah and And that one is also on AI, right? And and practitioners?
00:32:55
Speaker
It is. So I guess a theme of my work for a while has been getting out of my academic bubble and trying to understand how people are using data in the world and how people are using tools in the world and what we as tool developers or researchers could be doing to make their lives easier.
00:33:14
Speaker
And since everybody is now starting to use AI for everything, yeah the natural question for me was, well, how are people using things like large language models, generative ai for data visualization work? Yeah.
00:33:30
Speaker
and Is that working? Is it effective? Or what needs to change to actually make it work? Because we know how air prone these kind of models are. played with them myself.
00:33:41
Speaker
They do kind of bad jobs some of the time. yeah But we know people are using them. so We just simply wanted to know, you how are people using Gen AI for data viz work? And what are all the problems that are going on? So this was another interview study, kind of similar to the dashboard user study, just trying to understand current practice and challenges.
00:34:03
Speaker
And do you, I know it's early, but like, what is your level of concern with doing an academic study, especially a qualitative study, which tends to take time, and then how fast the AI models are changing?
00:34:17
Speaker
It's true. it is a big problem. yeah we We do want to get this work out fairly quickly because of that reason. right But I still think even if the AI models change and get better, it's still interesting to see where in the Viz process people are using these models and what kind of recommendations we could make or what kind of training programs we could create to help people use them in safer, more effective ways.
00:34:46
Speaker
For sure. Yeah. Awesome. Melanie, thanks so much for for coming on the show. Before i let you go, if people have more questions about any of these projects or, you know, if they're working on risk scores and hospitals in their area, what's the best way to get in touch?
00:35:02
Speaker
Oh, they can absolutely reach out to me. I'll get you my email. It's on my webpage too. Can get you the, uh, the link to our our human data and interaction group webpage and there's also a contact form on there if people want to reach out and get in touch.
00:35:17
Speaker
So lots of ways. Awesome. Awesome. Terrific. Thanks for coming on the show. It was great to see you and good luck with all these and I very much appreciate that dashboard paper. It was great to see you. Thanks so much.
00:35:28
Speaker
Thanks so much for having me. It's been fun. Thanks for tuning in everybody. Hope you enjoyed that episode. I hope you'll check out Melanie's website. I hope you'll check out that paper. If you have a moment, I'll put a link in the show notes. I really did find it interesting. You should also read my blog post on dashboard use in a changing media consumption environment. i of course, obviously think that that's an interesting write-up as well. And of course, I'd be remiss if I didn't please ask you to take a moment to rate or review this show wherever you get your podcasts. If you are on YouTube to subscribe and like, I would appreciate it. Helps me keep the momentum to bring this show to you each and every other week. So until next time, this has been the Policy of His Podcast. Thanks so much for listening.