Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Colby Hawker, Product Manager, Natural Language AI, Google, on the changing economics of AI
135 Plays1 year ago
This week on Responsible AI from the AI Forum, Colby Hawker, a natural language product manager at Google, joins Alex Alben and Patrick Ip to discuss the impact of AI on society and the economy.
Colby shares insights on the evolving approach to privacy in the current era compared to Web 2.0, highlighting AI's role as a collaborator rather than a replacement for human work. We explore privacy concerns surrounding AI models and delve into its potential applications in sectors like customer service, healthcare, and education.
Colby also addresses emerging challenges in AI, such as code generation and personalized medicine. Through their conversation, they emphasize AI's ability to enhance productivity and knowledge augmentation across industries, offering a glimpse into the transformative power of AI in today's world.
For more discussion, news, and thinking about responsible AI, visit our website, The AI Forum.
Recommended
Transcript
Introduction to the AI Forum Podcast
00:00:10
Speaker
Hi, I'm Alex Albin, and we're in another episode of the AI Forum podcast. We invite experts in the field of AI and relevant subspecialties to chat about the impact of artificial intelligence and related
00:00:27
Speaker
technology on society and the economy and technology. And we're really happy to have a special guest today.
Meet Colby Hawker: AI and Privacy Expert
00:00:36
Speaker
I will have Patrick Ip introduce him. Patrick. Thank you, Alex. Yeah, it's a special day. I'm glad to have a kind of an old friend, Colby Hawker, and I have been the friends at Google near
00:00:48
Speaker
nearly a decade. Colby has been at Google for the last 10 years, where he's currently a natural language product manager for both Palm and Gemini. He also teaches at Stanford Continuing Studies, where he teaches a course in making critical bets. He went to undergrad at BYU, and also has a master's from Cornell and a master's in engineering. And he also regularly publishes and writes for Google, writing on NLP and large language models.
00:01:15
Speaker
So Colby, you know, thanks so much for joining us today. I'll give you a little chance to share anything more in your background. Yeah, pleasure is mine. I think you captured most of the key points there. So excited to talk a little bit more. Yeah, I know. And kind of just team things up here. One of the first questions we wanted to kind of go into was looking at the difference between kind of web 2.0 and LMS and what you think they how they different in their approach to privacy.
00:01:45
Speaker
Yeah, I think it's a really great question, first of all. So Web 2.0, we can think of it as the era of social media and content creation, mass content creation. I'm a user. Go to Facebook or Instagram. I upload my photos. I create my posts. And I am basically sharing a little bit about me, which is my cost.
00:02:12
Speaker
in order to derive some value, which is being able to connect with my friends, being able to share more about my life and learn a little bit more about theirs and stay connected. In this era, the primary currency has been advertising.
00:02:30
Speaker
And with advertising, what's really required is some piece of personal information, whether it's an identifier about the person, so you can target them with ads or something about the browser, but you need some sort of personal information for relevant advertising. And for generative AI, for this era that we're in,
00:02:58
Speaker
these AI models, they don't need personally identifiable information. They take a massive amount of information from the web in aggregate, and they're using this to learn patterns and make predictions. And that's at the primary core of what AI does, it's making predictions. And so what's interesting here is the approach to privacy is quite different because you're able to
00:03:28
Speaker
de-identify a lot of the data that's used in LLMs and it opens the door to new privacy techniques like differential privacy, which to the audience here, this is basically adding random noise to the data that's being ingested so that the model itself cannot go back and completely understand
00:03:55
Speaker
you know, identifying information about the data. And so I do think the approach is
Data Use and Privacy in AI
00:03:59
Speaker
quite different. Now, the trade-off is also a little bit different as well, because when you're going in, you know, from a consumer standpoint, when you're going giving some of your information to, say, social media sites, you know who you're giving your information to, right?
00:04:18
Speaker
But for AI models that are now sharing a massive amount of information that they're taking or scraping from the web, you don't always know when your information is being used. And so there is a little bit of trade-off here. And that's why I thought it was a really interesting question. Kobe, this is a great area to start our discussion regarding the business model behind the technology.
00:04:47
Speaker
And I really like the observation that, in a way, the natural language model is more anonymous, at least starting out for the user and the participant. And I think that's important for users to understand. Like many new technologies,
00:05:06
Speaker
There seems to be a lot of fear around the use of the technology with regard to personal information. I'll give you one use case, but I think you can probably give us a few more. When attorneys are using
00:05:22
Speaker
chatgbt or bard or these other systems and they upload documents. Their fear is that the personal information of their clients is going to be absorbed by the model and somehow reused and somehow identified. Could you speak to that scenario a little bit? Yeah. And so this is the other, so there's a couple of pillars here. One is, you know, the data that's being used in training these massive models and the other is,
00:05:51
Speaker
actually using these models to make predictions or derive some sort of value. And so the vendors who are hosting these models are processing this information. And how that data is used, how it's stored, of course, really matters. And this comes down to business model and every company has their own approach to
00:06:17
Speaker
to making trade-offs with, you know, privacy. And I will say that, you know, most hyperscalers that I know of, and when I say hyperscalers, I mean, you know, these big tech companies typically follow very rigorous protocols, take privacy very seriously. But every company does have a different approach on are they retaining the information that's processed?
00:06:42
Speaker
Or is the processing stateless? Is it a stateless API, meaning the information is not recorded on the servers after the information is processed? And so that's very important, I think, for users to understand, and especially for companies or enterprises who are using these large language models or AI services to understand what exactly is happening with that data.
00:07:11
Speaker
Well, is the Google model a stateless API? Yeah. So it depends on the service, but we do focus a lot on stateless services. OK. And what are the other privacy concerns that you have to answer frequently? Yeah, so there's quite a lot in the range in the realm of privacy, because privacy is a collection of data, the storing of data, the processing of data.
00:07:40
Speaker
And so these concerns are very different at a consumer level. So someone who is maybe using Bard versus someone who is using large foundation models, like with OpenAI's chat GBT or, you know, some of Google's models. And so for the enterprise, it's exactly like you stated, Alex, you know, a lot of
00:08:07
Speaker
A lot of these large language models cater to big investment banks or other companies that are processing very sensitive information. And so the concern that comes up is, where's my data being stored? So does it need to be stored in Europe? Does it need to be stored in Australia? Do they require customer managed encryption keys? What level of encryption?
00:08:36
Speaker
Who can access this information? So when there's debugging or troubleshooting that needs to take place, who can access this? And so really, I think the commonality between consumer and enterprise use cases is transparency, knowing where this information is going. And maybe the ability to
00:08:59
Speaker
correct or delete the information? Is that offered? Exactly that. And the ability to go back and request that the information is wiped or deleted. And that's something that also comes up quite a bit. I was going to ask maybe like a more sociological question. I think in Web 2.0, we've grown accustomed to trading our privacy for access to services like Google or Facebook.
00:09:24
Speaker
And it seems in, you know, there's the web 3.0 era, but we might actually be taking kind of a detour to this more AI, LLM type of business model where there's actual subscription money being transferred. It's not your identifiable information. How do you think consumer psychology will change there? Where if there's a paywall now for every service, maybe like basic services, you know, the future of Google search, maybe in order to access it, you have to pay for it.
00:09:55
Speaker
Do you have any thoughts on how the general public should think about these kind of new emerging business models and kind of this new web? What's kind of your guidance and thoughts there? Yeah, I think it's anyone's guess at this point of how this will be received in the future. And I think a lot of what shapes this is how do journalists
00:10:22
Speaker
you know, talk about developments in AI. How do they talk
LLMs in Production and Economic Impact
00:10:26
Speaker
about issues in AI? And that will help dictate, you know, what are consumers willing to give up, whether that's monetary or maybe it's some level of privacy. When you look at the overall business world, Colby, what do you think the, you know, most obvious applications are going to be for organizational improvement, say,
00:10:48
Speaker
whether it's manufacturing or professional services. And are you seeing those things occurring already with the use of these large foundational models? Yeah, it was really interesting. So rewind a year and a half. I started as a PM lead for large language models before the era of chat GVT. And at the time, we were exploring different use cases with different customers.
00:11:17
Speaker
And when everything started accelerating, when OpenAI released chat GPT and then we subsequently released Palm and every other company released their own model, a lot of organizations were in a hurry to find that killer use case, that killer application. And the interesting thing was like all of last year, so all of 2023, very, very few organizations had managed
00:11:46
Speaker
large language models in production. There were a lot of toy use cases, there were a lot of experimental use cases, but it wasn't until the last, I would say four to six months, so very talented last year, where we started seeing real use cases with significant economic value. In terms of the sectors where we've kind of seen
00:12:12
Speaker
where I've seen the most disruption is one, professional services, as you mentioned, being able to process and understand information at scale, whether that be documents, you look at insurance companies, financial institutions, taking very laborious tasks and being able to automate this, having with large language models being able to interpret, you know,
00:12:39
Speaker
and generalize information so well, this has become much more effective than using traditional AI models that we had two, three, four years ago. And so companies that were otherwise hesitant are now putting these models in production. So professional services, customer service is an obvious one. You look at
00:13:03
Speaker
you know, Klarna, Klarna is a Swedish FinTech company. And over the last week, they made a press release that they actually were able to cut down 700 customer service employees because their customer service chatbot was working so well. And in terms of bottom line, they're forecasting $40 million in net new profitability.
00:13:32
Speaker
for 2024, which is pretty astounding. And so now is where more and more economic value is being realized. And then you look at life sciences, healthcare, and finance. Definitely. That's where we've seen a lot of traction and a lot of applications going live. We're speaking with Colby Hawker of Google on the AI Forum podcast. Patrick, did you have a question in another area?
00:14:01
Speaker
Yeah, well, I think one of the things, and I think for the general public too, is this kind of fear that people are losing their jobs and there's some displacement.
AI as a Workforce Collaborator
00:14:10
Speaker
What do you think is generative AI a net positive for society? How do people kind of reshift their job functions in this kind of new world?
00:14:24
Speaker
I think a very large misconception is that AI is going to completely replace certain types of jobs, which is not necessarily correct. I think most types of AI, including generative AI,
00:14:42
Speaker
is going to serve as a collaborator to human work. And so what this is going to do is it's going to automate a lot of the monotonous or laborious tasks. And then the people working in those jobs are going to be freed up to do more creative work and being able to upscale in the same field, really.
00:15:02
Speaker
And so I think it is a net positive impact. It's just going to be a shift and it will take some getting used to. And I do have friends who are working in fields where they're worried about more automation with AI and what are they gonna do with the longevity and security of their career. But I think most are starting to realize this is actually a great opportunity because they can simply make a pivot and do more interesting work.
00:15:33
Speaker
And I don't know if you noticed that as well from folks that you've talked to, but that seems to be, you know, that the picture is coming in a little bit sharper now. Whereas even a couple of years ago, I think there was a little bit more fear. We're definitely seeing that in the legal tech industry with respect to management of documents and document production, which tasks that used to take 10 hours can be done in five minutes. And that does free up attorneys. And I think it was going to,
00:16:03
Speaker
change, as you say, the role of the attorney in the firm. Another legal area would be this concern about intellectual property rights and plagiarism. I think the public has a conception that the large language models are simply storing every photo and every painting known to mankind in some huge library. Maybe you can
00:16:30
Speaker
educate us a little bit on how these models actually work.
Addressing AI Plagiarism Concerns
00:16:34
Speaker
Because I think that might allay some of the fears when you hear that they are making copies or plagiarizing. Yeah, I think it's a good point. And I think a lot of the fear comes from just the lack of education on how AI actually works. Where are your assets being stored? What is AI actually doing with with your information? And so
00:17:00
Speaker
At the heart, any AI model is just making predictions. And if you've ever played the game, you know, Mad Libs before, you know, this is this is a fun game where, you know, basically your, your, your, you know,
00:17:16
Speaker
shouting out words that are top of mind to you, right? And that's the way AI works is, you know, what word comes next in some given pattern. And so it'll take a prompt and a prompt could be, you know, summarize this information for me. And because of all of the information that I digested, it's able to understand, you know, what is the what is the prompt actually asking? And so therefore, I need to take this information that was given to me and
00:17:46
Speaker
and summarize it. And so it's going to do one word after the other and simply output these what we call tokens and start forming coherent sentences that are in line with the instruction of this given prompt. And so in simplified terms, I mean, this is glorified statistics. This is a prediction model. And once information is processed, once it's done making this prediction,
00:18:14
Speaker
There's no need for this information for this AI to be able to continue to do its job. Now, the difference here is some organizations may suggest that they are actually going to use that data for further training.
00:18:30
Speaker
but they have the obligation to disclose this. And so I think diving a little bit deeper and understanding how this works and what's being done with the asset and what need AI has for those assets or when does it not need it anymore I think is important to learn. But I think with this trend of multimodality, and what I mean by that is the ability for
00:18:59
Speaker
for these generative models to take multiple forms of input, whether this is text, this is imagery, this is videos, and then comprehend all this all at once. I think this raises some fears as well because now
00:19:18
Speaker
the assets or the multimedia assets that you're uploading are richer and may reveal more information. And so I think it's not surprising that there's a little bit more angst behind this, but understanding where the information is stored, how long is it stored for, is also very imperative as well. Interesting. When you talk about, let's say, the future of AI,
00:19:48
Speaker
over the next few years, do you think that it's gonna continue to be an integral part of tech companies? Is it just such a productivity enhancer that it's gonna lift our entire sector and economy? Yeah, I think so. I think we're actually just scratching the surface of where I is going and I use the analogy kind of coming into focus.
AI's Future in Privacy and Information Retrieval
00:20:15
Speaker
It was very opaque even
00:20:17
Speaker
12, 16 months ago, but it's becoming clearer of what is the future of AI at least looking the next two to four years out? Where could this go? And so we're starting to see some of those golden applications like a reference earlier, but I think
00:20:37
Speaker
There's a lot of progress in a couple of areas. One is safeguards and privacy. So removing fear, helping organizations get to production by removing some of those stumbling blocks and then allowing
00:20:56
Speaker
users and users of AI to feel more at ease, you know, collaborating with AI and using it in their daily lives. So I think that's one area. And two is, you know, specifically speaking to the progression of language models, the ability to enhance the quality of information that's being returned. And LLMs, as we all know,
00:21:24
Speaker
Large models of any sort can hallucinate. They can get information wrong. But there's lots of new approaches and techniques that are emerging. One of them being RAG, which is Retrieval Augmented Generation, basically connecting an LLM, a large language model, to a database or a ground truth information. And being able to have these large models
00:21:52
Speaker
back-check against themselves and cite where this information is coming from. And that's kind of the holy grail. And that's not true just for language, but also for imagery and videos and other types of large models as well. And so I think the future is still very bright. You're removing stumbling blocks. You're enhancing the quality of information that's coming to you, which is a big asset. And I personally think one of the
00:22:22
Speaker
the biggest values of AI is building on previous information. With large models that have digested the world's information and piggybacking off of Web 2.0, which is all this content that's been generated on the internet, all these Reddit forums and social media posts, these models learn how people interact with each other and also learn about
00:22:48
Speaker
human progression research articles. And it's really cool because you can go back in, you know, you have medical researchers, you have a lot of scientists who are now using large language models to build on previous research where they don't have to go and sift through all these previous papers. They can augment it, they can use it to augment their own creativity. And so this augmentation of current human knowledge
00:23:17
Speaker
is a big deal. And so, more forward-looking, I think that's only going to get better. I think for a lot of the Gen AI products, you know, we hear about advancements in productivity or medical. We had a guest on, you know, a couple weeks ago who talked about the advancements in the not-for-profit space, some of these kind of unsolved problems that now with these new tools, you know, we can actually have the bandwidth to go solve them.
00:23:40
Speaker
Do you have any perspective on maybe things that you feel like Gen AI, it's more difficult for them to solve or for them to touch? Like what are kind of the last milestones or last things that you think Gen AI will get to? Yeah, that's a really good question. There's a lot of use cases that are still
00:24:03
Speaker
very nascent and where generative AI doesn't quite get right. I think one of those areas where there's potential for, and I keep coming back to killer applications, is around code generation. Code generation is getting so close, you can automate a lot of functions, you can automate very basic website creation, so you can ask it to create
00:24:32
Speaker
HTML and some basic JavaScript code for you. But where it's lacking is creating some of these larger components to web applications or more sophisticated products, take an entire GitHub repo and be able to automate a lot of those coding processes. I think we're not quite there yet.
00:24:57
Speaker
because it doesn't always get the code right. And I think we'll eventually get there. So I think that's one area. And I think the other area is personalized medicine, so healthcare, actually a lot of areas of healthcare. And right now, data quality, and I'm speaking more broadly here. I'm not speaking on behalf of Google by any means.
00:25:19
Speaker
Um, broadly speaking, you know, there, there are still a lot of obstacles, um, in healthcare and in terms of, you know, the reliability of, of, of information, you know, can you, can you rely on it? And how much can it understand, um, from medical knowledge? Now you have med palm, you have a lot of these advancements, um, in, in, we're, we're going somewhere, um, but it's still very nascent. And so I think we're still, you know, sometime away from, from having that killer application.
00:25:51
Speaker
We're talking with Colby Hawker of Google and Patrick and I have been working a little bit in the education field and I know that you're involved at Stanford as I have been on the Stanford Law and Tech Advisory Board.
00:26:07
Speaker
education and advancements. Do you see some net positives happening already in that field aside from students generating their term papers with the various applicable tools? Yeah, so I think the commercial applications, we still are scratching the surface in education and I wish there would be
00:26:30
Speaker
more investment in that space specifically, AI education, because I think there's tremendous impact.
Personalized Education with AI
00:26:36
Speaker
The way that students learn is fundamentally evolving. It's changing. And with AI, you now have the capability of catering knowledge to individual needs, individual learning styles and preferences,
00:26:54
Speaker
And I think that's amazing. And even for me, when I am either preparing to teach a course, I teach multiple courses, or when I'm just trying to catch up on latest trends of generative AI or large language models, there's papers going out every day. I need to stay on top of this information, but I don't have time to go and read through every paper
00:27:23
Speaker
I want to be able to augment this information for my use case, for what I'm using this for. And so a bad use case is taking a bunch of information in an educational context and pawning it off as your own work or doing it without any sort of oversight. I would never suggest that Stanford students would do that. I would never suggest that.
00:27:49
Speaker
But it has the ability to augment your own understanding and comprehend quicker and be able to take that information and incorporate it into your own work. And I think that's a really cool capability. Patrick, we probably have time for one or two more questions with Colby.
00:28:13
Speaker
Yeah, we a swath of our audience are students. And I'm curious if you had to go back to school knowing everything that you know today, would you change what you learned or what would you kind of focus on given today's evolving landscape? Yeah, it's an interesting question. And I, I honestly haven't given it much thought. And I think, I think if I were to go back, I think it's always important in any, again, any educational context to learn
00:28:43
Speaker
the building blocks of your field. Because even when I was in graduate school, they would tell us from an engineering perspective, what you're learning today is likely going to be outdated two or three years from now. And so we want to teach you how to learn better and equip you with the right tools to accelerate your learning five or six years down the road. And so being able to take apart, what is static information today?
Document AI: Automating Business Processes
00:29:14
Speaker
versus what are the dynamic tools that are likely going to, in some shape or form, going to exist or live on in some evolved form tomorrow or five years from now, how do you learn those tools? And then how do you change the way you learn to be able to, again, comprehend and synthesize information quicker? And so I think learning the right tools, learning how to do better research quicker, I think are some of the skills that I'd go back and try to
00:29:44
Speaker
do a little refresher on. Colby, one of the specific projects I think you worked on are identity cards using AI. Can you talk a little bit about that, Document AI? Yeah, so several years ago, I worked on a couple of projects in Document, we call it Document AI, which is a solution for businesses to be able to better understand their documents. And these documents are
00:30:15
Speaker
you know, contain very ambiguous information to computers because you have a document and then you have potentially different images or charts in these documents and you have unstructured text that could be anywhere. And so what we were doing is basically, you know, training machines to be able to understand
00:30:38
Speaker
you know, patterns in a lot of these types of documents where this information was likely to be found, whether these were identification cards like passports or driver's licenses or something else. And you have to teach these models to be able to interpret spatial information coming from the document, linguistic cues from the documents.
00:31:06
Speaker
and other visual cues on the document as well. And so it was a very interesting project and ultimately the goal was to help with automation and help companies, big banks, government organizations to process information quicker.
00:31:22
Speaker
I mean, maybe when you think about it, one of the great achievements already of these language models is they've helped us interpret unstructured data. We have so much, the world is a world of unstructured data and actually just bringing structure to it enables you to utilize all of the wealth of information and intelligence there in a way that's accessible to anybody who has access.
00:31:51
Speaker
Exactly, and for the last several years, I've worked on language models and three years ago, four years ago, the goal of natural language models was to understand ambiguous information and make sense of it because it's really hard to do. And even now with generative capabilities, so now it's understanding these documents and now generating something new
00:32:19
Speaker
It's still very relevant because oftentimes some of the organizations that we work with will simply generate a structured view of that document, whether it's a table and extracting key piece of information, you know, name of people that are represented on the document, different dates, different locations. And then this structured view can basically be uploaded to some database.
00:32:46
Speaker
and shared with the broader organization or other workers who are involved in various capacities. Right. Well, on that note, I want to thank you for your time and speaking with the AI Forum Podcast. Patrick, did you have any closing thoughts? No, probably this is great. Thank you so much for sharing your perspective and hopefully we'll have you on again. Absolutely. We really appreciate it. Thank you. Thank you.