Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
D3 creator Mike Bostock visits the show
248 Plays5 years ago
Mike Bostock is the creator of D3, a popular open-source JavaScript library for visualizing data and Observable, an open-source notebook for exploring data and thinking with code. Previously, Mike was a graphics editor for The New York Times, where he helped produce a...
The post Episode #175: Mike Bostock appeared first on PolicyViz.
Recommended
Transcript
Introduction and Episode Overview
00:00:11
Speaker
Welcome back to the Policy Viz Podcast. I'm your host, John Schwabish. I hope everyone is well and safe and healthy in these strange times. So I'm looking forward to this week's episode of the Policy Viz Podcast because I am reposting one of the recent Data and Urban Digital Discussion conversations I had with Mike Bostock, the founder and creator of the D3 JavaScript Library.
00:00:35
Speaker
I myself am not a JavaScript programmer, but there were a lot of people on this conversation that had questions about how to use D3, about some of the ongoing evolution and modifications about D3, and Mike spends a bit of time talking about his new project, Observable, which is, I think has found an audience out there of users who are interested in sharing and creating open source code.
00:01:02
Speaker
So I hope you enjoy this week's episode. Before we get into that conversation, just a couple of quick notes. First off, if you haven't seen, I have recently posted my data visualization catalog.
Data Visualization Catalog and Community Engagement
00:01:14
Speaker
I've been collecting data visualizations for the last couple of years. I have more than 700 images in the catalog right now. I have used those for myself for inspiration and for techniques that I might like to try for other data. And I also use it, of course, for teaching in my classes and my workshops.
00:01:31
Speaker
And I asked my Twitter network to see if they would like to take a look at that catalog and there was an overwhelming response in the affirmative to that. So I tooled around with the Google Data Studio tool to create an interactive table where you can see both the images and the URLs and my notes. So that's up on my website if you would like to take a look. I have placed it in a new part of the site dedicated to my forthcoming book, Better Data Visualizations, that will hopefully be out this fall.
00:02:00
Speaker
So I'll leave notes and links to that tool on the show notes page, and I'll also link over to all the other data at Urban Digital Discussions that we've been having over the last couple of months. Most of those, if not all of those, have been recorded and posted to the Urban Institute's YouTube page, so you can go in and take a look at those.
00:02:19
Speaker
But in this week's episode, I've taken that video chat and I've turned it into a podcast. I think it does work as an audio, so hopefully you will enjoy it and be able to learn some things from it. So here is this week's episode of the PolicyViz podcast, my discussion with Mike Bostock.
Pre-Discussion Banter
00:02:39
Speaker
Good afternoon and good evening or good morning, everybody. Come on in.
00:02:47
Speaker
Grab a cocktail if you're on the East Coast. I'm not quite there yet. I still have 29 minutes before I pour the first cocktail of the day. I should have brought a cocktail and I'm on the West Coast. I wasn't told. I did have to go to the liquor store today. They've changed the liquor store thing. You have to stand outside and give them your order and they will go get it for you. So I got the, I said, give me the biggest bottle of bourbon and the biggest bottle of rye and just, and the biggest bottle of vodka. And you don't just, you know, many bottles.
00:03:18
Speaker
All right, so we'll just let folks settle in for a minute and then we'll get started. And also while we're waiting here for everybody to settle in, I'll just share next week's lineup.
Upcoming Sessions Announcement
00:03:29
Speaker
So next week for those who are interested, three sessions next week.
00:03:34
Speaker
Tuesday at noon with Mike Correll, Jessica Holman, and Robert Casara, academic uncertainty discussion. Wednesday at 11 with Michael Brenner from Data for Change. And then Thomas Mock Thursday at three about R. So we'll turn back to tools. So just quickly, we're working on getting that up.
00:03:54
Speaker
All right, I think we're ready to roll.
Observable: Simplifying Interactive Visualizations
00:03:57
Speaker
Mike, thanks a lot for tuning in today. It's good to see you. My pleasure. So again, I don't have an agenda or a list or anything. Good to see you too.
00:04:07
Speaker
Um, maybe let's see, I don't know. Do you want to talk about what you're working on right now? Maybe that's when we can just start there. And, and just for folks who haven't done this before. Feel free to put your questions in the chat box and then you can just unmute yourselves and you all just build up a queue and you can chat with Mike yourself and we'll just take it like that. So I don't know, Mike, you want to start by just, you know, giving us the lay of land, what you're working on. Sure. Well, for the last few years, uh, I've been working on building this company observable.
00:04:37
Speaker
Um, and it really started out, um, you know, coming, coming out of the work that I was doing for D three. And, you know, one of the problems that keep, uh, coming back to was essentially like how difficult it is to build interactive visualizations. Um, and, you know, seeing a lot of the people learning D three and struggling to figure out how to put the APIs together or to, um, you know, really do creative stuff with all the examples that existed there.
00:05:06
Speaker
And ultimately I kind of came to the conclusion that the biggest issue wasn't visualization per se, it was code. It was sort of the difficulty that people run into just writing interactive programs, building interfaces. And I think part of that also comes to, you know, how much I enjoy writing code or I guess like the joy that I get from code, the flexibility, the power that it provides,
00:05:34
Speaker
It always felt like there was this compromise between, you know, tools that were based in code that could let you do whatever you want and tools that were sort of point and click and were easier to use but were correspondingly, you know, much more limited in what you create. And I always, you know, I like the creative aspect of it. I don't want to build a, you know, kind of a patronizing tool that makes all of the decisions for you and just, uh,
00:06:02
Speaker
I want things to be as easy to use as possible, but I don't want to take away people's creativity and freedom and whatever they create with it. And so that's the problem that I wanted to tackle is basically how do I let people have that expressiveness and that power that comes from code, but at the same time trying to make it more accessible. And we did have a lot of, I think success in the D3 community with the examples where people would sort of share those examples on blocks or elsewhere on the internet.
00:06:32
Speaker
Um, and in a sense, like observable was trying to kind of take that to the next level, right? Thinking about how can we sort of eliminate all of the technical obstacles to getting people to start tinkering with these examples so they don't have to, you know, get cloned in the terminal and install a bunch of stuff and set up a local web server, um, and have sort of a whole ID set of locally. They can just get started in their browser. Um, but more than that, I think it's about thinking about.
00:07:01
Speaker
you know, the abstraction, the programming language. There's a lot of complexity, I think, in sort of traditional programming languages where you have mutable state and you have event listeners and you're sort of mutating things and remembering these side effects. And, you know, I've seen a lot of people struggle just getting that stuff working when they're trying to build an interactive visualization. And, you know, I would prefer,
00:07:30
Speaker
people spend time thinking about, you know, is the visualization communicating? And, you know, am I learning something interesting? Um, am I able to, you know, express or explain whatever it is that I'm trying to do and hopefully spend less time with sort of the machinery, the technical aspects. And, and so that's, you know, essentially what we try to do with observable is make reactivity sort of a language level feature rather than just a library feature. And by doing that,
00:08:00
Speaker
It simplifies a lot of the code that you would need to write to make an interactive interface. And I think the most exciting thing for me is it also has this kind of social effect where it becomes easier for you to import code that other people use and to compose it and to do interesting things. So, I mean, you can fork somebody's notebook, of course, but
00:08:21
Speaker
You can also import the cells directly into your notebook and then you can even rewire them. So like a lot of the D3 examples, you know, you don't have to fork them in order to get your data into it. I mean, that's a, that's a valid strategy, but in addition to that, you can just import them and replace the data. And I think it's, it's really exciting to have kind of new ways of composing together code. That's a bit more flexible and repurpose. And, you know, ultimately our goal is to try to help the community learn from each other.
The Future of Programming and Visualization
00:08:50
Speaker
and share their techniques and to bring more people in that are able to do visualization effectively.
00:08:57
Speaker
Right. I'll just ask a couple of questions until people start asking you questions, because I'm just the moderator. So I know Raul is out there, so I'm sure he's going to have questions. But in your view, what is the future of programming? I mean, so observable, you're trying to be at that edge of how people are going to work together. But five years from now, do you have a vision of how people will work together on code that will be fundamentally different than how we're doing it now? Yeah, I don't think there's going to be sort of a grand unification.
00:09:26
Speaker
I think that there's, or like one ring to roll them all or anything like that. I mean, I think there's, there's so many different, um, applications and use cases and needs that there's always going to be sort of different languages, different ways of development, um, you know, different technologies, different styles, all that stuff. So I don't think we're trying to sort of, uh, replace all of the different ways that people use programming right now. I think in a way how I view observable is more like,
00:09:54
Speaker
replacing Google docs or replacing, you know, like Dropbox paper and stuff like that, or medium. Like basically like if code is this, is this language that enables us not just to do analysis, but also to communicate more effectively. Like we need that within our communication medium, right? Like we need that within our standard tools for working together with other people for communicating, for presenting data, all that stuff. Um, and you know, I certainly write a lot of,
00:10:23
Speaker
uh, blog posts and articles that involve code and having this like medium that lets me write pros and include visuals and make them interactive at the drop of a hat. I mean, that's, that's what I love about it. So I think of it more as focused around communication and collaboration rather than sort of general purpose programming. Um, although it certainly does have fuzzy lines and you can kind of extend it into a lot of different areas.
00:10:50
Speaker
Um, in fact, what I've been working on this week is, you know, learning serverless computing, like the AWS stuff and building more sort of stateful, uh, persistent, really like fully fledged web applications, essentially within an observable notebook. Um, and there was this great, uh, interactive Chris Zubeck skis, I think is his name, uh, who made, uh, you know, can you identify Ukraine on a map? Like there was this, um, case where it was like,
00:11:19
Speaker
Mary Louise Kelly, the NPR reporter, I think was doing an interview with Mike Pompeo and he questioned her ability to identify Ukraine on the map. And of course famously like she had no problem identifying Ukraine on the map. And then the question was like, how good is the general public at actually doing that? And so this reporter made an interactive on observable that
00:11:45
Speaker
when you clicked on the map, it would record the location where you clicked, posting it up to AWS and storing it as a table so that you could then aggregate all of those results and see where people were clicking. So let's see. So Robert, Kasar has a question for you about video. So Robert, if you wanted to just unmute yourself, go for it.
00:12:06
Speaker
Sure, yeah. Hey, Mike. Hey, Robert. So I saw you mention, so I have to say, I haven't really done much myself in observable, but I keep getting, and use it, and it's really awesome. And I saw you mention, I think it was, maybe I'm making this up, but I hope, I hope I'm not, that you said something about wanting to integrate video into observable. And so I was curious what your plans were there. Like, is it just gonna be full on video, the traditional way, or do you have any ideas that go beyond
00:12:36
Speaker
Basically YouTube. Yeah. So we're actually, um, basically spinning up our dev rel team now. And one of the big initiatives is going to be a series of videos, teaching people, um, both observable and D three and visualization and a variety of other things. Um, so we're just getting started with that now. I don't know if it's going to be like directly integrated into the platform per se. Like, I don't think we have any immediate plans to build like a MOOC or anything like that, but then observable.
00:13:02
Speaker
But we are looking at other formats for teaching people observable. You know, we've written a lot of notebooks already. And, you know, that's nice because it's within the product. You can start tinkering with it. But, you know, different people have different styles of learning. And we want to sort of make it available to people who like those different styles as appropriate.
00:13:25
Speaker
I was just wondering if it's more like Khan Academy where there's more sort of like integration between video and the content. So you have like chapters and maybe like you have to do a thing before you can go to the next step. I mean, this is just me making stuff up here, but I was just curious if you had thought about that kind of thing.
Tools and Integration in Media
00:13:44
Speaker
I don't think so, not yet. I mean, one of the
00:13:47
Speaker
strengths of observable is that it's code. So you can kind of do whatever you want. So I mean, you could theoretically build that potentially a fair amount of work. But I think, you know, we're, we're open to ideas. I think we're just getting started right now with the video content and making sort of interactive sessions could be could be interesting. Yeah.
00:14:09
Speaker
So I'll again put a call out for people to ask questions. I have another question for you. And this is, it's going to sound like a, it might sound like an accusatory question, but it's not. So what I've heard is that the New York Times is moving to using data wrapper for all of their graphics.
00:14:31
Speaker
And I know they also have an internal tool for a lot of the journalists to make, you know, more static things that they need that they need quickly. And I'm just curious. And again, I don't, I'm not trying to accuse the times or data or anybody doing anything, whatever. I'm just curious about your perspective on the model of moving away from
00:14:52
Speaker
using, you know, pure JavaScript or using observable and moving toward more towards a tool where it has more of this drop and drag type, you know, flicking approach. Well, I would imagine that the New York Times is going to retain like a whole variety of tools at its disposal for doing interactives. Um, although I'm sure that there's a lot of, uh, daily graphics and stuff like that, that will be covered well by data wrapper.
00:15:19
Speaker
Um, so I think for us, um, you know, I don't think we're really directly competing with data wrapper because it is more of a development environment rather than sort of a point and click construction of visualizations. Um, and also I think we're also not really, um, just making a visualization that you can drop into a larger article. Like there's a whole variety of things that you can build and observable that are, um, you know, either dashboards or sort of explorable explanations.
00:15:50
Speaker
um, or sort of real analysis that you're doing in observable, um, rather than just sort of dropping in some data and then assigning some visual coatings and dropping out an SVG. Um, so it's really like I was saying earlier, sort of more about replacing, you know, how people communicate and collaborate within, well, for the, I mean, for the business side of things, cause I see the other, the chat question there, you know, it's about like, how, how do teams of people, you know, within enterprise,
00:16:19
Speaker
that are all within these data driven organizations dating to make decisions and get insights out of their data. Like how do they work together around that data? How do they communicate with each other? How do they collaborate on the analysis? And for doing that, you really need the flexibility of code. So there's a lot of stuff that you can do in point and click interfaces, but ultimately at some point, you know, whatever you're doing probably hasn't been done before. If it's giving you a competitive advantage, like that's kind of the,
00:16:48
Speaker
The definition of doing science is that you're doing something that's new, right? Like you're asking questions that have not been asked before, or at least have not been answered before. And so you need the creativity of code. And so it's really about trying to bring code into the standard ways that people communicate, write stuff down, share their results. Yeah. I don't know if you want to talk to Martin's question about the business model for Observable.
00:17:16
Speaker
Well, I think it's really just about collaboration. So for us, we have, you know, two kind of audiences, I think, and the idea is to have some synergy between those. So we have people that are, you know, using observable out on the public, you know, the open source, like myself, like doing D3 examples and just sharing techniques, sharing our love for visualization or whatever else, just making fun, pretty things. And for those people, you know, they're welcome to use it for free.
00:17:47
Speaker
Uh, and then we also have sort of a more enterprise focused product where when people are collaborating around private data or private analysis and they want to work together, they can sign up for a team with an observable and they just pay a subscription fee to go along with that. Yeah. Um, great. Um, so we have two questions in the queue. I'll let David, I don't know if David, if you want to, um, unmute yourself, um, you can, oh yeah. Yeah.
00:18:12
Speaker
Yeah, hey, Mike, thanks for doing this. This is really interesting. And I've only dipped my toes into observable, but really exciting to get my feet wet. An exciting thing happening in the R community right now is R Markdown is kind of taking off. It's kind of a similar notebook technology. You're probably familiar with it, along with many other listeners here. But they've started to integrate Bash, Python, SQL, other languages seamlessly within the notebook.
00:18:42
Speaker
And a big dream of mine would be to do something similar within observable, where maybe I'm, you know, alternating between R or Python before dipping into, you know, D3 and kind of this integrated context.
JavaScript Focus and API Integration
00:18:54
Speaker
I don't know if that's on your radar, on the horizon, or I'd be curious to know. Yeah, I think that's a pretty common question because, of course, there are a lot of different languages that people use, like we were discussing earlier. And I think for us, though,
00:19:10
Speaker
JavaScript has so much strength, I think, in it being the sort of de facto or lingua franca of the web. You know, it's what every browser runs. And therefore, you know, that's why we put so much emphasis, you know, on JavaScript being in the language. And in fact, even why observable extends the JavaScript language.
00:19:32
Speaker
Um, so when you're writing stuff in observable, you're not actually writing vanilla JavaScript. It's an extension of the language that we've, uh, you know, added these reactive features to it, this data flow to it, um, to simplify how you do, you know, interactive state. Um, so I think we don't have any immediate plans for supporting other languages, but I think what our long-term vision is that we want to be able to integrate with other systems or pipelines that people would use.
00:20:01
Speaker
So you might use R or Python to do some data analysis. And then we want to have an easy way to bring that into observable either to just do the presentation or to do sort of further like downstream analysis from there. So we have like file attachments is a feature that we launched within the last year. And I think we'd also like to see sort of a programmatic API for that so that you can easily sort of upload new data into your notebooks or to sort of
00:20:29
Speaker
automate the display or like a dashboard or something like that. And so I think that there's always going to be multiple languages that people use. Now, there are some interesting possibilities around WebAssembly, you know, Emscripten and other tools that let you take languages and compile them into something that can run fast in the browser. So there's been a project at Mozilla
00:20:56
Speaker
Iodide was a web-based notebook that had Pyodide, which is basically the ability to run pandas, Python within the browser JavaScript. So we might see some things like that, but we do have a sort of a very strong focus on JavaScript because we're integrating in at the language level. And I think another aspect for us is having a single language is also good for collaboration.
00:21:24
Speaker
the notebooks are all written in the same language. It's easier for you to sort of import and reuse techniques and other notebooks that you could find on observable. So that's part of it too.
00:21:37
Speaker
Before Stuart Thompson, who, you know, I'm sure he's just like chomping at the bit, before we get to Stuart's question, Martin followed up with me, and he was, just to follow up on this, on this, what you were just talking about, do you use, Martin wants to know, I think it's a good question, do you use JavaScript for everything, or do you also use R, Python, other languages for both, I guess, data wrangling and viz? I mean, for me,
00:22:02
Speaker
I mean, yeah, I use JavaScript for pretty much everything. But of course, I'm not really doing visualization professionally these days. You know, I'm working on building observable and I'm making examples and we're doing some analysis internally, of course, at observable. And for that, yeah, we use JavaScript sort of all the way down.
00:22:22
Speaker
Right. Um, all right. So let's see, uh, Martin, I hope that was, uh, what you were looking for. I don't know. Um, so Stuart, uh, Thompson is here. He has a question for you. I don't know if he's. Yeah. Hey, thanks for this mic. Yeah. I'm just kind of wondering like you and your role as like a data visualization person and like, do you miss having like that as your main responsibility or do you not? I mean, I see still make stuff, but yeah, I'm just kind of curious about that.
00:22:48
Speaker
Oh yeah, absolutely. I miss it every day. Um, I mean, it was a, it was a dream job. And I think it's not just like, you know, getting to work on visualization that has a large audience and they can sort of resonate with the general public. Um, but it's also just, you know, the people that I got to work with at the New York times were fantastic. Um, and I think, you know, all, all reporters, all journalists like really have a passion for, for learning and trying to absorb as much information as they can around them.
00:23:17
Speaker
And they're also good at sort of explaining that to other people, like digesting it and sharing it. And that makes them for very interesting people. Like there's a lot that you can learn from and there's a lot of like good stories. So, so sure. I miss that. But I think one of the things that I enjoy now about the work that I do is also this sense of, you know, trying to build, um,
00:23:43
Speaker
you know, infrastructure or tools that will have lasting value and will change sort of the set of people that can do this stuff that can have sort of that deeper understanding of what's going on around them, the quantitative understanding or the ability to explain it. And so I do, you know, I enjoy building new technologies and I enjoy making them available to people. And it's hard to do that when you have sort of deadlines of publishing graphics.
00:24:13
Speaker
So, you know, I'd love to do both. I always like having the mix of sort of practical applications like building graphics together with working on sort of the more abstract development of tooling. Like if you just do tooling for too long, then you kind of, you lose your concrete foundation. And so I think what we've tried to do to mitigate that with observable is have a very strong focus on learning from our users, like building out sort of the user research group,
00:24:42
Speaker
and doing surveys and analytics and, you know, obviously just spending a lot of time talking to people, like helping people out when they ask questions in the forum or on Twitter or Slack or wherever and trying to learn from those frustrations and figure out how we can make the product easier for people to use. Cool. Thanks. Great. Thanks, Stuart. I'll put another shout out there for folks to put questions in the chat window.
00:25:12
Speaker
Mike, I was going to ask, outside of the big news organizations like The Times, The Post, The Guardian, do you have, are there other media organizations that you think are doing really phenomenal or just good data visits that people should check out? Well, I think Jon Bird-Murdock of the Financial Times is doing some pretty great coronavirus visualizations on a regular basis.
00:25:39
Speaker
And of course he's well known for popularizing that bar chart race for better or for worse. Yeah. Um, yeah. Um, do you have any other ones? Yeah. I mean, he's, he's great. Um, you might not have any, I was just curious.
00:25:58
Speaker
Yeah, I don't know. I feel like I've been trying to avoid some of the news lately. I don't know. Yeah. Yeah. Yeah. Um, well, since, uh, there's, there's, there aren't any other questions rolling in. So let me ask another one. Um, are you in the camp that you think everyone should learn to code? Um, I mean, I don't know if I'm like an absolutist or like more of a pragmatist. I think that it certainly is a valuable skill.
00:26:26
Speaker
Um, and I think that if we can make it approachable, like if we can actually make it accessible to people, then like that's really what matters. So it's sort of a function of how valuable is it combined with sort of how difficult is it for people to learn? You know, if you have something that is extremely difficult for people to learn and telling everybody that they need to learn it, like is just not practical because there's a lot of stuff, like they may not get the value out of it that you're expecting.
00:26:54
Speaker
So it's important to accommodate that. So in a sense, like, I think there is potential value that is not being realized today because it is too hard for people to learn. And so my focus is on trying to make it more accessible.
00:27:07
Speaker
Um, rather than just, you know, telling people that they should do this impossible thing. Right. Is it that, is it the act of actually building something that's important for code or is it being able to think in a particular way that it's language is ultimately what it is. Right. So it's really about configuration versus composition. Um, and it doesn't have to be written. It's just that written language or just the concept of a language.
00:27:34
Speaker
is inherently going to favor creative composition over something that is really just configuring something that's been built by somebody else. So you can do plenty of creative things like an illustrator or sketch or other sort of drawing tools. And I think that's an example of creativity. I think for computers, so it's both the compositional approach
00:28:03
Speaker
And of course the fact that when you're expressing language in a way that a computer can understand, you're creating a solution that can basically be scaled up arbitrarily, right? Like can be automated and you essentially solve a problem once and now you can solve that problem as many times as you want for no additional cost. Right. Right. Um, let's see. So let's see, uh, two other questions. One from Martin again. Um, are there any major developments for D three on the roadmap?
D3 Library Updates
00:28:30
Speaker
I mean, yeah, there's D3 6.0, which I've been working on for for quite some time, but which hasn't quite made it out the door yet, although it has made it out in sort of piecemeal fashion. So if you recall for D3 4.0, the big effort there was to break D3 up from a monolithic library into about 30 different modules that people could use independently.
00:28:49
Speaker
And I think that's had some pretty good traction now. I mean, I think there's been a lot of modernization and the JavaScript tool chain since then. So it's easier for people to sort of pick and choose which parts they want to use. So for example, D3 array 2.0 was released a while back. And I think that has some really great improvements to it, both for the JavaScript language in terms of support for iterables, but also like the new D3 group, D3 rollup as a replacement for D3 nest, I think are fantastic.
00:29:17
Speaker
Uh, if I can toot my own horn. Um, and then there's, uh, like D3 Delaunay, which is based on Vladimir Agafunkin's, uh, Delaunay, which is a really fast implementation of Delaunay triangulation. And then we implemented a Voronoi tessellation on top of that. And so that's going to replace D3 Voronoi. And then there's been a bunch of other improvements like to D3 scale, a bunch of new scale types, um, trying to simplify a little bit how event handling works in D3 selection.
00:29:44
Speaker
So some of it hasn't been done yet, which is why we don't have sort of a pre-release available for D3.6.0, but hopefully we will get a chance to do that within the next few months. And in the meantime, you know, people can check out sort of individual modules and I would particularly recommend D3 array 2.0. Okay. There's a lot of people who are like twirling, you know, sort of,
00:30:10
Speaker
Rubbing their hands together right now getting ready. That's great. So Julia has a question for you about helping folks make DataViz. So Julia, if you want to unmute yourself, you can just ask. Oh, there you are.
00:30:26
Speaker
Yeah, I was wondering what your challenges are and how you resolve them at Observable around helping people to make useful and readable data visualizations. Because I figure, you know, you give somebody a tool, like it's one thing to use the tool, but it's another entire skill set making data visualizations that are useful and that achieve what the person wants to achieve. Yeah, absolutely.
00:30:54
Speaker
I don't think we're doing enough there. I think part of it, the challenge is that observable is more low level focused, right? Like we're focused on sort of how people write code and how they put together interactive interfaces. And we're not very opinionated about what those interfaces are. Like we're not Vega light, for example, like you can use Vega light within observable, but you can use any other tool that you want to make your visualization so that observable as well.
00:31:24
Speaker
And so there's not really an opportunity at the platform level to guide people in the creation of visualizations because it's just a lower level abstraction. I think we're guiding people to make simpler programs and sort of developing the good idioms for representing interactive programs within observable and making them more broadly accessible. But I think ultimately the improvement to visualization practice comes from people helping other people.
00:31:54
Speaker
and from discovering and sharing techniques, sort of through publishing notebooks and through collaborating with other people in the platform. So it's not the platform itself, it's the community of people that are using it and how they work together with each other. And we're trying to make improvements there as well, for example. One of the things that we're working on that should be available any day now is sort of improving our commenting system, just so that people can talk to each other and comment and ask questions about notebooks.
00:32:22
Speaker
Um, I think when we started out with observable, you know, we, we came from an open source background and we're familiar with the maintenance burden and some of the emotional costs that comes from dealing with random people on the internet every day. And therefore we designed observable to be very sort of private focused. Like you can send a suggestion to somebody to, to, here's like a code change to your notebook, but you're not allowed to comment directly on other people's notebooks. And,
00:32:50
Speaker
I think ultimately that was too much airing too much on the side of not letting people talk to each other. And so we're trying to strike the right balance there to allow the community to flourish and collaborate with each other. And so hopefully you'll see some improvements soon. And I think also maybe I would add to that is that we want to seed some of the content unobservable to teach people how to use both the visualization tools
00:33:20
Speaker
but also the visualization practice. And we've seen some great stuff, like Jeff Hare at UW, for example, has been publishing a bunch of notebooks for his visualization classes that are being shared and observable. And I would love to see more of that. So I think a combination of examples that people can use, but also sort of critical essays, tutorials, guidance, that sort of stuff is what I would love to see from the community.
Community Collaboration in Observable
00:33:46
Speaker
Cool. Thank you.
00:33:47
Speaker
So another just call for other questions. On that line of you were just talking about Mike, have you ever considered or talked to anyone about coming up with a or developing a D3 JavaScript observable type of environment like the makeover Monday or the tidy Tuesday? I don't know thing. It's a safe program, but I don't know what you would call it. It's a thing.
00:34:13
Speaker
Yeah, I'd love to do some of that. I think, you know, as I mentioned, like our dev role team is just getting started. And I think we maybe are starting to have some of the bandwidth where we could support things like that. And I think having a regular cadence could be nice for building the community. Um, I mean, it's been a challenge trying to, to do everything that we're trying to do with the product. Um, and also like do stuff that's just sort of, um, supporting the community.
00:34:40
Speaker
Um, like it was, you're trying to build a platform at the same time, but I mean, I think we're, the team's grown a lot. Um, and so I think we're, we're hoping we can start to take on some of those additional responsibilities. Yeah. No, it's certainly a big lift. Um, yeah, I'm sure it's a big lift.
00:34:56
Speaker
We don't need to stretch this out if other folks don't have questions, but I'll ask one more and then if we're done, we're done and we'll get cocktails early. I was wondering, I've been in an ongoing discussion and I've done one of these sessions on teaching DataViz to kids.
00:35:11
Speaker
And I'm curious, have you thought about or had experience. I mean, I know you have kids of your own, but I'm just curious. Have you thought about or tried to do teach code or teach JavaScript to kids and how do you approach that as an is a different than any, you know, teaching anybody else how to code. I mean, I only have like n equals one sample size on that because I've only tried to start teaching my seven year old daughter how to code in observable
00:35:36
Speaker
And she was super excited about it. I mean basically you just put some emojis on there and some sliders and Anything like you know colorful and moving was enough to like anyway. She was she was really excited about it, and it is I Don't know it's fun. I mean I'm always trying to you know teach people D3 and Like each time you kind of go lower down the stack It's like it's harder and harder because you're having to
00:36:02
Speaker
sort of really surface all of those things that you've internalized and assumed and essentially like forgotten about you now process them on a purely subconscious level. And so explaining like kind of basic things about, you know, what is a primitive type, you know, like what is a variable, um, is, is challenging if you don't have experience doing that. Um, I would love to do more of that. I think we are planning on, um, doing some,
00:36:30
Speaker
you know, introductions to JavaScript on observable because I think, you know, it's, it's a great platform for learning because it is so focused on letting people start tinkering as soon as possible. Um, and I don't know if we'll go like down to the seven year old level necessarily, but I think, um, ultimately I do want, you know, to make JavaScript and programming more accessible to people, not just visualization. Um, and I would like us to start exploring more stuff like that.
00:37:00
Speaker
Right. Um, so I think we have maybe, I mean, we have about 20 minutes, but there is one question and maybe this'll, this'll wrap us up. Um, from Sarah who wants to know if someone wanted to get started into contributing to D3, how should they get started? That's a good question. I think D3 is not really that, uh, friendly for other contributors. And I think part of it is just because, um, there's a desire to keep it small or comparatively.
00:37:29
Speaker
And also there's a big cost whenever like something is broken, you know, because then you have to go in and fix it and everybody's mad at you. So I think honestly, the best way for people to contribute is really through examples and tutorials, um, where you're not necessarily making a new feature, but maybe you're sort of advocating for a new feature. Um, and you are sort of demonstrating how to do that with the existing API. Um, or maybe you're sort of building your own sort of library or an extension.
00:37:59
Speaker
I mean, I think D3 is very modular. So there's been a lot of success, I think, for other modules that add to this sort of core set of D3, you know, like D3 legend and D3 annotation, for example.
00:38:14
Speaker
and just a variety of things like there. So I think that's easier because you won't be sort of gated on me approving your PR being merged into the library. But I think there's still plenty of value that you can bring to the community by either teaching other people how to use the D3 API in some new way or advocating for some sort of improvement to the library.
00:38:39
Speaker
Right, right, right. Sarah, I hope that was helpful. Ankur has a question for you on reusable charts. Yes, specifically, there's been a whole tutorial that Mike had written how to build reusable charts. Have there been any new updates or recommendations that you would suggest? I mean, no, not really. I think I tend to be fairly agnostic about how people use
00:39:08
Speaker
how people kind of assemble D3 into sort of applications or frameworks. So D3 is not a framework. It's a collection of sort of related tools, like the low level modular library. And I think there's a lot of different ways that you can apply those tools in whatever application you're building. So for instance, you know, if you're using react to build your react application, you know, there's a pretty well established pattern of building react components.
00:39:36
Speaker
And that's probably what I would recommend using if you're using React. If you're using Vue or if you're using Svelte or if you're using Angular, there's different ways of building components in those systems. And I want D3 to be agnostic and to work well with all of those systems. So I don't really have a stance or an approach that I would recommend over the others. It would depend on the other frameworks or development style that you prefer. Thanks, Mike.
00:40:02
Speaker
A pleasure. Let's see, Ben. I don't know if Ben, you want to unmute? Ben has a question for you on data literacy. Hey, yeah, Mike. So this is just a general question, but the Towel Center for Data Journalism has done extensive work on algorithmic accountability and then also the convergence of platforms and publishers. I'm just wondering if you have
00:40:28
Speaker
you know, any, you know, I guess, tips of the trade, you know, for data journalists who are thinking about data vis and kind of ethical responsibilities, you know, whether it's in the newsroom or thinking about audiences, just more broadly, I guess. I think that covers a lot of ground. So let's see, what would I say? I think one of the one of my
00:40:56
Speaker
sort of takeaways from working at the New York Times is just like how important it is to surface all of the assumptions that go into the design of a visualization. And that means the importance of the annotation layer and any sort of keys or legends or guides that sort of explain what the visualization means. The problem as a designer is that you've internalized sort of what your visualization means because you've been working with the data, you understand what the data means,
00:41:25
Speaker
and you've wrote the code that assigned those visual encodings. And it's very easy, surprisingly easy to forget to explicitly say, you know, what the X and Y axis means or what the color means or what size means and stuff like that.
Ethics in Data Visualization
00:41:40
Speaker
And one of the sort of rules of thumb that I found good working on published graphics is like, you know, if you were to just explain your visualization verbally to somebody else,
00:41:54
Speaker
Like, would you have something interesting to say or not? Like, if there's nothing that you as the author of the visualization can say is like an interesting takeaway from the graphic, then that graphic probably shouldn't be published. Because it's really your responsibility as the editor, as the author of that graphic.
00:42:10
Speaker
To have sort of extracted at least a couple sort of interesting insights and you're then presenting them in a visual way because That might be a more effective way of communicating it and there may be other things that The reader can sort of pull out of the graphic that you don't include in that initial set But if there's really nothing interesting that you can say out of the graphic then you probably haven't done quite enough work yourself to sort of pull out what's interesting and the
00:42:34
Speaker
Great thing about having an explicit annotation layer is that it's essentially forcing you to do that work upfront and to highlight at least a couple of things that are are good in that graphic. I mean, sure. Um, yeah, that was that was interesting. Um,
00:42:52
Speaker
I was going to ask, since your time at the Times, I've observed this anecdotally, but I'm wondering whether you share it, that there's been a move away from interactivity for sort of the everyday graph and whether you think that's a good thing or a bad thing, or just a thing. I mean, sure, there was a little bit of hubbub.
00:43:11
Speaker
Gregor Eich wrote a great article on, you know, in defense of interactive graphics. Uh, so I recommend everyone read that if you're interested in the topic. I mean, I think it's like any bit of new technology where things can be overused at the beginning and then, uh, you know, the pendulum swings back a little bit and then people find sort of more refined ways to apply that technology. So I don't think it's really a question of like, is interactivity good or bad? Like it can definitely be misused and it can be used well.
00:43:39
Speaker
And I think you have to understand sort of what the value is that the interactivity is providing and also the costs that are associated with making a graphic interactive because there are costs. There's cost to any sort of design trade off. So I think in particular that I think the main risk that you have with interactivity is that it can lead to a potential sort of abrogation of responsibility where like as the editor, you know, you really should be
00:44:09
Speaker
digesting and refining and presenting this information that sort of front loads the most important thing for the reader. And if you have interactivity, there's kind of a temptation where like you can show everything to everyone and like you don't have to make that decision because the readers can find it. And the problem is like it's work for the readers to find it. Like you're just shifting that responsibility from you as the editor to the reader. And so if there's nothing sort of immediately of value to the reader,
00:44:38
Speaker
then the fact that it's interactive doesn't matter because people aren't going to do that work for you and they're not going to get anything interesting out of it. So I think you have to look at it as kind of a complementary tool or ability where you want to have a good static graphic that communicates something. And then in addition to that, you can have something that lets the reader pull out something that's more of a personalized insight
00:45:06
Speaker
Or more detailed stuff if they're willing to do the work to engage with the graphic. Yeah, yeah I mean there's kind of there's a different thing though. I mean About animation right these days and I don't know exactly how I feel about that where I think a lot of the things that get people's attention on social media are things that are animated and
00:45:29
Speaker
And is that really a better visualization or is it just something that's where it's able to capture people's attention more effectively? I don't know. Well, so is it so I guess the question is, is it better for that particular platform? Because I feel like you could create a static graph or an interactive graph, you know, you could you could create static interactive for the New York Times site, but you might create even something different for the for for Twitter. I'm thinking of like the
00:45:57
Speaker
When the Times did those little racing swimming animated gifts for the Olympics, must've been 2016 or something like that. Like those little animations didn't show up on the New York Times website, they were only on Twitter. Yeah, I don't know. I don't know. I don't know. So Sarah's a good question related to this is your thoughts on the scrolly telling.
00:46:24
Speaker
Well, I've written an article some years ago about how to scroll that tries to provide more specific guidance on sort of what good scrolling behavior should be and how you can sort of still do things on scroll, but not frustrate the user who may have expectations on how the page should react to scrolling. So I think
00:46:49
Speaker
Like any other technique, it should be used judiciously. Um, I think the main advantage that you get with scrolling over clicking is that it's sort of the most common and accessible affordance. Um, like it's in particular, like for, for journalism, you know, you're, you're telling a story usually at the very least, like there should be one sort of one dimensional path or narrative through the article or through the story that you're trying to tell.
00:47:18
Speaker
and scrolling typically is also one-dimensional, and therefore that's an advantage. It's similar to just reading static text, but now you can have other things that are happening alongside of it that lets you do something potentially more interesting. I don't tend to use it very much unobservable, but I don't know. I don't have any sort of explicit guidelines as to when it's appropriate to use it or when it's not. I just kind of know
00:47:47
Speaker
When it's done poorly, it can be frustrating. I'm taking advantage for a second. But when it comes to scrolly telling and any sort of other interactivity, in particular with observable, how are you and how are you thinking about needs for accessibility for people?
00:48:07
Speaker
Scrolling telling is a good example because there are people when you have these two different things that are moving not in sequence or one is showing on top of another one that can trigger, you know, different types of migraines and reactions and things like that. So I'm just curious how you're thinking about vision issues or cognitive disabilities or any of that.
Advancements in Accessibility for Visualizations
00:48:27
Speaker
I mean, I think that's a challenge for the visualization space as a whole.
00:48:33
Speaker
I mean, it turns out that for technical reasons, coincidentally, you can't actually implement scrollytelling within Observable. There's probably a way around it, but generally speaking, because of the way the code gets run in the sandboxed iframe, it's much harder to do typical on-scroll fancy effects. And so I haven't really seen it done on Observable, although it may be possible if you're willing to jump through a few hoops. There's certainly some things that you can do with intersection observers. But I think the other...
00:49:03
Speaker
primary thing that where I've seen people make some progress in terms of making visualizations more accessible are sort of color deficiency issues and trying to both come up with good defaults for colors, both like for ordinal or categorical colors and for sort of continuous or sequential color schemes. And then like tools that let you sort of either try to evaluate contrast levels or simulate different types of color deficiencies. So
00:49:33
Speaker
I think early on, you know, we added Cynthia Brewer's color schemes to D3 as a D3 scale chromatic and we also eliminated the category 20 color schemes.
00:49:47
Speaker
Which was somewhat controversial, but I think was the right call because they were essentially paired color schemes which tended to create sort of a false association between the light variant and the dark variant of the same hue. They could have a light blue and a dark blue that would seem to be related when, of course, they're categorical and they probably weren't.
00:50:05
Speaker
Um, and also it's just difficult to distinguish 20 categorical colors. And so we just got rid of those and we switched to 10. And I think that's how hopefully encourage people to switch to using like an other category. If they have more than 10 categories that they want to include in their visualization. Um, and I've seen some sort of interesting, uh, new work on like perceptual color spaces. Um, some of which has been done in observable.
00:50:35
Speaker
So that looks like a Jay-Z, A-Z, B-Z color scheme. Basically, they're just designed to give you distance in color space equates to perceptual distance. So that if two color coordinates are similar, you're likely to perceive those as similar colors. If they're very different, you perceive them as different colors. And so you can start to quantify
00:51:04
Speaker
in this color space, how different two colors are and therefore measure like contrast ratios. And one of the interesting things is some of the color metrics that we use, for example, there's a WCAG contrast ratio, um, uses, uh, I think it's, well, I don't remember off the top of my head, but it's definitely not one of these like perceptual color schemes. And I don't think they even use like linear light color schemes. So it's just like RGB values.
00:51:33
Speaker
Um, and so it's pretty heavily distorted and there's some criticism that they should be using sort of a more, uh, perceptually based color scheme or color space for doing this analysis. On the other hand, um, they're pretty rudimentary tools for evaluating, um, sort of what contrast ratios are. So I don't know if it, if that's more of a pedantic argument or if it would make a practical improvement. Right. Right.
00:51:59
Speaker
I think we have one more from folks. Ivan has a question for you on using observable for hiring. So Ivan, if you're there. Hey there. Thank you for your amazing tools. I love them both. Probably you have much more. The question is, I know a lot of companies you'd have for hiring people. And so I'm curious to know
00:52:27
Speaker
Do they use observable for that? Well, I don't know for certain, but it's definitely, I think, one of our hopes is that, you know, people can kind of build up a visualization portfolio and do that more easily and observable than they could somewhere else. And then use that, you know, in their CV or their resume to sort of give demonstration of someone's abilities.
00:52:50
Speaker
I mean certainly like in our own hiring of course like we look to see you know what observable notebooks has somebody written and we do like Some pair programming or interviewing stuff that are based in within observable notebooks But I don't know if we've quite reached the mainstreamness yet to be Competing with github as the sort of like de facto portfolio, but you know maybe eventually I
00:53:15
Speaker
Maybe. Yeah, one day, one day. So one more question for Martin, which I think is a good way to wrap up, which is, what's your favorite observable notebook? I refuse to answer this. It would not be fair. No comment on that one. I think, I mean, I have been really impressed, I think, by
00:53:38
Speaker
the work that's been done on like analyzing the coronavirus or COVID-19 data recently.
COVID-19 Data Visualization and Community Analysis
00:53:44
Speaker
I mean, there's just been literally hundreds of notebooks that people have published that are looking at the data and visualizing it in different ways and sort of creating sort of more localized views, you know, like somebody wants. Like New York Times does a great graphic for sort of the US, but like if you're in another part of the world and you have data, like you can just fork that and replace it and you create a sort of more localized view as well.
00:54:07
Speaker
So that's been cool. And I've definitely learned from some of the other more analysis focused things, like, you know, people looking at creating simulations of the coronavirus, you know, like these SIR and SEIR epidemiological models. But of course, I think I would caveat that with
00:54:30
Speaker
it can be challenging and dangerous to potentially try to draw too much from the simulations that people create if you don't have a background in epidemiology. So they can be sort of intellectually interesting, but I would be careful about trying to draw too many conclusions from them. Right, right. Cool. It was great. Thanks, man. Thank you. My pleasure. Thanks for all the questions.
00:54:55
Speaker
Yeah, everybody, that was great. Let me just quickly share next week, if you're interested, if you're interested in chatting next week. On Tuesday, Michael Correll, Jessica Holman, Robert Casara from noon to one Eastern time, Michael Brenner for Data for Change on Wednesday from 11 to 12 Eastern time. And on Thursday, Tom Mock, who runs the Tidy Tuesday, our project and works at RStudio, will be here from three to four on Thursday. And that's all I've got for now. That's probably enough.
00:55:25
Speaker
So thanks, Mike. Thanks everybody for tuning in. Go enjoy your evening cocktails. And yeah, talk to you all soon. Thanks again, Mike. That was great. Appreciate it. Thank you.
00:55:39
Speaker
And thanks to everyone for tuning into this week's episode. I hope you enjoyed it. I hope you learned a bit about D3 and about Observable. If you are interested in supporting the show, please share it with your friends and family and neighbors, or you can support it financially. Go over to my Patreon page where I have various levels of support where you can help me for the cost of web hosting, podcast hosting, and all the tools and tricks that I need to bring this show to you every other week.
00:56:09
Speaker
So I hope you enjoyed this week's episode of the show. Until next time, this has been the Policy Vis Podcast. Thanks so much for listening.