Transcript
Speaker: Hey, everybody.
Speaker: Welcome to Grounded Truth, a podcast where we gather some of the world's most influential data scientists, machine learning practitioners, and innovation leaders for conversations on the most relevant topics in AI today.
Speaker: I'm your host, John Singleton, co-founder and head of success here at Watchful, the machine teaching platform for data-centric AI.
Speaker: You can try Watchful for free at www.watchful.io.
Speaker: And please, if you like this podcast or any of our content, like, subscribe, follow us on Apple, Spotify, YouTube, or anywhere you find podcasts today.
Speaker: Joining us today is my co-founder and fearless leader here at Watchful, CEO, Cheyenne Mahante.
Speaker: How's it going, man?
Speaker: Not too bad.
Speaker: How are you?
Speaker: I'm doing great.
Speaker: The seasons are changing.
Speaker: We just had Halloween.
Speaker: Did not get a single trick or treater, which I don't know quite how I feel about that, but I feel like there's like an adult phase in your life.
Speaker: You're supposed to accept these things, but it's been fun.
Speaker: Today, I'm really excited to talk about something that I had a very, very small contribution with, and I think I can officially add to my CV ML researcher.
Speaker: Congrats.
Speaker: Yeah.
Speaker: Thank you.
Speaker: I know.
Speaker: I like patting myself on the back.
Speaker: But I'm really excited to talk about our most recent blog post that you published titled, Decoding LLM Uncertainties for Better Predictability.
Speaker: Quite a mouthful.
Speaker: Why don't you start off by saying, what does that mean?
Speaker: And kind of what sparked all this?
Speaker: Yeah.
Speaker: So what sparked it, let me just start there.
Speaker: We've been hard at work integrating a lot of like Gen.ai platforms.
Speaker: oriented features into Watchful, obviously.
Speaker: And one of the things that we noticed is that as we were iterating on various prompts, prompt templates, various ways that our users want to integrate prompts into our product, there just wasn't really a whole lot of actionable feedback that we could give them to improve results.
Speaker: And this came, you know, not just because prompting is hard, but almost more importantly,
Speaker: we couldn't possibly be arbiters of every possible use case.
Speaker: And the way that you go about improving prompts seems to right now be very much around rules of thumb, like use capital letters to get the model to do what you want or- It likes new lines.
Speaker: Yeah, it likes new lines.
Speaker: It's all very soft and in the realm of magic and arcane.
Speaker: Yeah, yeah.
Speaker: So, you know, what we have noticed, though, is that some of these rules of thumb do work, which means that there is a method to the madness.
Speaker: It's not all random, right?
Speaker: There is a set of things that can be done that will improve results.
Speaker: But how do you even measure those results?
Speaker: How do you know if it's actually an improvement or not?
Speaker: And more than that, if there is a method to the madness, can we quantify it?
Speaker: That sort of set us down this journey.
Speaker: And we did this first release a while ago, which our last podcast was on, which is around kind of like the prompt heat map thing.
Speaker: Can we define token level importance for- Yeah, exactly.
Speaker: Can we assess where the model likely spent its attention, so to speak, where it paid the most attention to our prompt without having direct access to the model?
Speaker: So all of this research that we're doing is in the vein of
Speaker: I have these models where I only have access to it via an API for the most part, you know, unless you're all in on open source, but a lot of the world isn't right now in a world where I only have access to the model via an API.
Speaker: What can I do to quantify my actions so far?
Speaker: What can I do to quantify how good my prompt is at addressing a particular task?
Speaker: How do I quantify how good my model is at addressing the task?
Speaker: How do I quantify what I need to do next to improve my prompt and so on?
Speaker: So that's like how we started.
Speaker: In the process of doing this, we've just sort of been in some ways like meandering through like the breadth of work that can be done here.
Speaker: And one of the things that we landed on was measures of uncertainty.
Speaker: where essentially what we noticed is that since you can write a prompt and get varied outputs depending on not just like your hyperparameters, your temperature and so on of the given API call, but even more than that, even if I keep the temperature the same, as long as the temperature is not like zero, or whatever the minimum value is for the model that you're using,
Speaker: I would get varied responses.
Speaker: And so there is a space of responses that would come back.
Speaker: And what I'd wanna know is how consistent are these responses?
Speaker: And so I basically wanted to be able to quantify
Speaker: How certain is my model about what it's trying to say?
Speaker: And generally speaking, I try not to anthropomorphize these models, but oftentimes- It's hard not to.
Speaker: Yeah, it's easier to explain the intuition behind these things when you put almost like a human face over it.
Speaker: But that's essentially what it is.
Speaker: But it's, I mean, which makes sense because it's just an echo of human communication and data that we have created, right?
Speaker: It makes sense to kind of, at least it feels natural to reason about this thing, you know, kind of with those same constraints.
Speaker: Yeah, exactly.
Speaker: So that's sort of the impetus.
Speaker: You know, we just wanted to find various ways to quantify what the model is doing.
Speaker: reason about what it might do in the future.
Speaker: And one such thing is just figuring out how big is the space from which it is sampling responses.
Speaker: And the smaller that space, the more concentrated it is likely, the more certain the model is and therefore likely the more, uh, consistent the outputs will be, uh, given various inputs.
Speaker: So that, that was sort of how we landed on this.
Speaker: And why do I care?
Speaker: Why do we care about model uncertainty?
Speaker: Why is this important?
Speaker: Yeah, so there are a couple of things.
Speaker: One is, I think right now, a lot of the interface that is most in vogue for Gen.ai applications seems to be chat.
Speaker: So one such reason to care is like, you want to make sure that your model isn't going to say something harmful.
Speaker: There are lots of ways to sort of go about quantifying and estimating that.
Speaker: But one way is,
Speaker: How uncertain is your model generally when it's being prompted?
Speaker: And what shapes of prompts will spike uncertainty?
Speaker: Which ones cause it to go all the way down?
Speaker: Does that go in line with human expectations for how this model should act?
Speaker: That's one very obvious reason.
Speaker: The other semi-obvious, but a little bit longer term, is that these models...
Speaker: are currently being integrated with existing APIs.
Speaker: And I think in the future, they will be even more so integrated, meaning at some point, chat interfaces may go away or not be as prevalent, and these models will end up getting embedded inside of workflows.
Speaker: So in that world, the model will have to be able to produce outputs that can then be consumed by downstream system.
Speaker: which means that it needs to adhere to some sort of spec in some capacity.
Speaker: In the natural language case, adhering to a spec might just be like, you want the model to always be helpful and harmless.
Speaker: In the case of integrating with an API, it's like, I always wanted to produce JSON of this particular structure.
Speaker: I always wanted to produce strings in this particular way.
Speaker: Being able to guarantee that or at least quantify how likely it is that the model will produce the thing that you intended to is very important.
Speaker: And then finally, the amalgamation of all of this is just as someone who's building AI systems, you want to have confidence in the system that you're building.
Speaker: You want to know that you understand the space within which it operates.
Speaker: And you want to know that there are boundaries around it, some of which could be hard in the form of
Speaker: literal like guidelines or guardrails or whatever you want to call them.
Speaker: Others are perhaps that you tuned the model and now you have high confidence that it's always going to output something in the space that you intended to.
Speaker: So we just wanted to bring rigor to this instead of just having a whole bunch of like code guardrails where it's like if this happens then
Speaker: Like, don't send this response.
Speaker: Try and reprompt in this way.
Speaker: We wanted to add rigor to the way that we quantify these things.
Speaker: How do I quantify the likelihood that the model will produce something that I want?
Speaker: And that's, you know, sort of why you should care.
Speaker: So I'm hearing kind of like maybe to give a summation, I'm hearing two main themes.
Speaker: One is predictability.
Speaker: I need to be able to trust the output of these systems and the shape of what they're outputting for, like whether it's workflow reasons or like even trust and safety.
Speaker: And I think they kind of, it's also the second kind of theme.
Speaker: There's a pretty strong Venn diagram overlap here is just observability.
Speaker: We need to be able to just as
Speaker: practitioners who are building AI systems, we need to be able to observe what they are doing and hopefully in a predictable manner.
Speaker: And I think that's very topical because this week, just a few days ago, as of this recording, the Biden administration released a sweeping executive order.
Speaker: And I have it pulled up over here with the second bullet point literally being
Speaker: developed standards, tools, and tests to help ensure that AI systems are safe, secure, and trustworthy, which are just a lot of words for saying we need to be able to observe these systems and apply logic and rules to those outputs and be able to trust them in some way, which is something that is sorely missing in the kind of current state of the art and state of the ecosystem today.
Speaker: Yeah, 100%.
Speaker: So,
Speaker: without commenting on the merits of an executive order to define these things.
Speaker: the intention of being able to interpret and explain what these models are doing is good.
Speaker: We have this giant ocean of metrics that can be used to quantify supervised systems very well.
Speaker: You've got ROC, you've got precision recall by nature of that accuracy, F scores, and so on and so forth.
Speaker: You've got all of these metrics that
Speaker: frankly, like data scientists and ML practitioners can look at and have a very good understanding of what their system is doing.
Speaker: Like how is it performing on a given task?
Speaker: And the important thing is that these are task specific metrics.
Speaker: Now in the Gen AI world,
Speaker: There are metrics around model comparisons and things like that, which are largely benchmarking.
Speaker: Benchmarks, yeah.
Speaker: Yeah, just benchmarking across these generic NLP tasks, here's how these various models perform.
Speaker: A, they're not fully comprehensive, and B, more importantly, they're not task-specific.
Speaker: Being able to compare two models relative to a specific, potentially business-specific and business use case-specific task is, frankly, the more important and valuable thing than just being able to
Speaker: broadly wave your arm and be like, oh yeah, this 17 billion parameter model outperforms this 6 billion parameter model.
Speaker: That doesn't really help all that much.
Speaker: So part of the goal of this sort of train of research that we're doing is to focus on task specificity, where we're completely task agnostic.
Speaker: We look at the prompt, the outputs, anything that we can perturb or sort of like investigate interstitially there.
Speaker: And we produce metrics at the other end.
Speaker: So to that point, uncertainty is one such metric.
Speaker: that you might want to use.
Speaker: It's certainly not the be all end all by any search of the imagination, but it is an important part and something that could be added to a much larger group of metrics that could be very task specific.
Speaker: The phrase I keep using to at least try to describe this is we're going from a largely deterministic programming paradigm where we have systems that we can rely on.
Speaker: And we have, as you said, these ocean of metrics that have been developed over years that we can assess these systems and moving to a fundamentally non-deterministic programming paradigm.
Speaker: And we simply just are in the process of developing these tools.
Speaker: And hopefully this research is kind of directionally going along that and bolstering the
Speaker: the kind of state of the art and building those tools.
Speaker: But ultimately they're just lacking.
Speaker: I think as I helped do some of the background research of what is state of the art for doing quantitative prompt assessments is effectively having an ideal or gold standard prompt and then doing a cosine similarity or cosine distance and showing things that are out of band.
Speaker: So clearly we need a better tool chest.
Speaker: Yeah, there's that.
Speaker: And then there's obviously like opening eyes evals library where it's like you take a prompt, you take an output and then you have a human rate it like how good was this output?
Speaker: And then you have model rate it.
Speaker: And it turns out that.
Speaker: All of the GPT models seem to be very complimentary about the responses in general.
Speaker: And you sort of measure the distance between the human rating versus the machine rating.
Speaker: But again, that doesn't really have a lot of rigor to it.
Speaker: So our goal was to make this such that we didn't have to rely so heavily on
Speaker: model outputs to be the source of insight for the metric itself.
Speaker: We wanted to calculate the metric independent of the model and instead make it just like a core mathematical function that can be run across any model.
Speaker: So I think that's a good understanding of the why.
Speaker: Let's move on to the what.
Speaker: What are these uncertainty measures?
Speaker: Yeah.
Speaker: So I'll start just like with the basics, right?
Speaker: So let's say you make an API call to one of these models, a completion call, and you say, okay, here's my prompt.
Speaker: I want you to complete the rest of the response.
Speaker: You will get back text.
Speaker: So you give it text, you get back text.
Speaker: For a lot of these models, you are able to not just get back the text to the response, but also get back the other tokens that could have been sampled in each token position.
Speaker: These are, at least in OpenAI's API, it's called log props.
Speaker: So you'll get like the top N tokens, usually like you can configure at.
Speaker: So I think at maximum on OpenAI, it's like five tokens per position.
Speaker: And it'll give you the probability associated with each of those tokens.
Speaker: So a very common approach is you take all those probabilities and you calculate something called entropy, where essentially the entropy is maximized.
Speaker: There's maximal entropy.
Speaker: when all of those probabilities are really close together.
Speaker: The intuition for this is that the model's not really sure which token it's going to pick.
Speaker: So it spreads its probabilities evenly across a set of tokens for a given position.
Speaker: So that's one option.
Speaker: The other extreme is that entropy is minimized, meaning the model is super certain.
Speaker: So it's the exact opposite.
Speaker: One of the tokens has a very, very high probability and the others have very, very low probabilities.
Speaker: So you'd imagine that you'd be able to trivially calculate the model's overall uncertainty.
Speaker: by just seeing how entropy varies over the course of producing this response.
Speaker: And that's been something that people have looked at for a while.
Speaker: You can calculate perplexity, which is like very closely related to entropy to actually quantify, you know, what sample size of tokens am I actually pulling from?
Speaker: So there's a bunch of things that you can do there.
Speaker: What we realized though, is that this measure of uncertainty
Speaker: is confounded by the fact that oftentimes the tokens that are being sampled from, so let's say that you have a token position that has maximal entropy, five equally probable tokens and the model just picked one.
Speaker: It's possible that those five tokens mean exactly the same thing.
Speaker: And it doesn't actually change the meaning of the outcome.
Speaker: So is that actually useful information?
Speaker: Are we almost artificially inflating the uncertainty of the model by taking that into account?
Speaker: Or is there something else that we should be calculating?
Speaker: So we discovered that there is actually like two pretty distinct forms of uncertainty.
Speaker: We took a stab at naming them.
Speaker: We called one conceptual uncertainty, which again, I'm going to anthropomorphize the model, but it roughly means how certain was the model about what it was trying to say?
Speaker: And the second is structural uncertainty, which again, in the anthropomorphization is how certain was the model about how to say it.
Speaker: So the way we go about calculating these things is, again, fairly straightforward.
Speaker: So you take these responses, you get the log props.
Speaker: And for structural uncertainty, we just basically calculate entropy over every single token position.
Speaker: And we measure how it moves over the course of the response.
Speaker: And we calculate a metric out of that.
Speaker: So that's structural uncertainty.
Speaker: And the details are in the blog post.
Speaker: Yeah, it's also worth mentioning that we haven't mentioned at all
Speaker: In the blog post, there is a link to an interactive demo.
Speaker: So I know this is a lot just in, you know, if you're listening to this in the podcast or watching on YouTube or like, please go play with it, test some of these things out, you know, see and feel and give us feedback if you'd like it.
Speaker: But you can go play with this today.
Speaker: This is up on the blog post.
Speaker: It'll be linked with this in the description.
Speaker: Yeah, exactly.
Speaker: And in the blog post, we have formalizations for exactly how to calculate these metrics and so on.
Speaker: So
Speaker: I'm going to talk at a very high level about them right now.
Speaker: But that's structural uncertainty.
Speaker: So it's like a direct link to entropy, essentially, and a direct link to the log probes themselves.
Speaker: And we just track its movement over the course of generating a response, and we calculate a metric out of that.
Speaker: The second is conceptual uncertainty.
Speaker: So the intuition for this is...
Speaker: As the model is sampling the various tokens it could be choosing, the question is how many distinct logical branches are there?
Speaker: So as an example, you might have who was the first president of the United States as a prompt?
Speaker: There is an answer.
Speaker: There is an answer, right?
Speaker: There is a correct answer.
Speaker: There is a single answer.
Speaker: And so you could have George Washington be the response.
Speaker: That's it.
Speaker: The model decided to sample George in the first token position.
Speaker: And because it chose George, the next token logically must be Washington.
Speaker: And then it decided to end there.
Speaker: So it gave an end token.
Speaker: The next response could be George Washington was the first president of the United States.
Speaker: So again, it picked George.
Speaker: Therefore, it picked Washington, and then it opted not to end there.
Speaker: And instead it picked was.
Speaker: And because it picked was, it then kept going, the first president of the United States.
Speaker: Each next token is conditioned on the tokens previous to it.
Speaker: So in that world, it doesn't really matter how many additional words you add to this response.
Speaker: It doesn't really change the core concept.
Speaker: It's still talking about George Washington.
Speaker: There is an answer.
Speaker: And the answer that was provided was correct.
Speaker: So that just kind of works.
Speaker: So you would imagine in that world, if you had several of these responses that were laid out next to each other, George Washington, George Washington was the first president of the United States.
Speaker: George Washington was the first president of the United States, sir, from this time to this time, known for this, this, and this.
Speaker: These are all acceptable answers.
Speaker: The core concept is exactly the same.
Speaker: So you'd imagine that conceptual uncertainty would be very, very low.
Speaker: But structural uncertainty might be reasonably high.
Speaker: Not high in the sense that like, oh, these outputs are like significantly very... There's only so many ways to say in a short return...
Speaker: George Washington's the answer.
Speaker: Exactly.
Speaker: There's almost infinite little tales that you could have, but they're just, as a human looking at them from afar, which you can, by the way, see this one in the blog post, you see that they all look and smell awfully similar.
Speaker: Yeah, so that's essentially what we're measuring with conceptual uncertainty.
Speaker: It's how consistent are these responses and how, if I were to look at all the various branches that we could go down, if we were to choose various options, if we decided for whatever reason, we wanted to choose some of the other options that didn't quite make it from the token sampling, but could have been chosen, how varied would those branches actually be relative to the response that was chosen?
Speaker: Uh, and, and that is like a very interesting measure because it then like basically tells us how conceptually aligned were all of these various branches with one another.
Speaker: If selecting various tokens doesn't really change the outcome very much.
Speaker: If it's all talking about the same subject, then the model is conceptually aligned with, you know, whatever it's trying to say might not know exactly how it's, how it wants to say it, but it knows what it wants to say.
Speaker: So that's sort of the idea.
Speaker: And this is my, I was tasked with basically validate these metrics, write some prompts that you feel are logically consistent and validate that these metrics do seem to at least reflect the way that you're perceiving these answers.
Speaker: And it was one very satisfying to the anthropomorphization helps a lot.
Speaker: Anthropomorphization.
Speaker: Man, words are hard, dude.
Speaker: I tell you.
Speaker: They are.
Speaker: Yeah.
Speaker: is useful because you start thinking of, okay, like what is the shape of a conversation or a returned result that would have these properties?
Speaker: And I'm very proud of this one.
Speaker: You are a caveman with limited vocabulary, explain quantum physics.
Speaker: And my goal here was, could I get something that had relatively low conceptual uncertainty?
Speaker: The model, like I assume in the backend, there is
Speaker: roughly an answer of the definition of quantum physics.
Speaker: It's a kind of commonly written about thing in a number of textbooks you'd think it'd be commonly represented in the training corpus.
Speaker: Makes sense.
Speaker: I guarantee you there's nothing that a caveman wrote, quote unquote, or in the style of a caveman to describe quantum physics.
Speaker: And what we found was there was very low conceptual uncertainty.
Speaker: So the model was fairly certain on what
Speaker: concepts it was trying to say.
Speaker: But since it has no representation of what a caveman would even sound like, we had the highest, I believe, structural uncertainty of any prompt that I was able to produce.
Speaker: And I really like his example because to your point, the concept itself was pretty consistent.
Speaker: And what's interesting to me is that it could have chosen to write about any number of things in the quantum physics realm.
Speaker: But it actually chose pretty consistently to talk about things like particles and, you know, just like uncertain states and so on.
Speaker: And by constraining the vocabulary to something that is almost diametrically opposed, like, you know, talking like a caveman to then explain quantum physics.
Speaker: It's almost positive.
Speaker: I'm almost certain there isn't enough volume of that type of text available on the internet.
Speaker: Yeah, I'd say that's a fair assumption.
Speaker: For it to have been conditioned that way.
Speaker: So it was very interesting to see the fan out of the structure.
Speaker: It was choosing very interesting words to use that were clearly based on the conditioning of the prompt.
Speaker: But the amount of structural uncertainty was clearly measured to be high simply because the output structure varied significantly across various responses, despite the fact that the core concept was the same.
Speaker: So yeah, very, very interesting example.
Speaker: And the last one, I'll stop tooting my own horn here, at least that I found was very interesting, was something that I wanted to come up with a question that I as a human would feel was extremely uncertain, like the most uncertain.
Speaker: And so I picked a very unimaginative one.
Speaker: What is the meaning of life?
Speaker: And what we found was there is an answer according to ChatGPT.
Speaker: Like there is an answer to the meaning of life.
Speaker: Spoiler alert, it's pretty much that there is no answer, which is probably a correct answer.
Speaker: But as a human, I would score that just objectively extremely ambiguous because it's almost by definition unique to an individual.
Speaker: There is no correct answer.
Speaker: There's nothing that can ever really be produced that would suffice or that would make everybody happy.
Speaker: Yet what we found, we had extremely low
Speaker: conceptual uncertainty and a very middling structural uncertainty.
Speaker: And so what my intuition was is perhaps, and this obviously needs a lot more research is maybe we can discover things that I guarantee you the model has been trained on that answer or aligned on it, I should say.
Speaker: It's too specific because they know this is going to be something that could inflame
Speaker: people's opinions, if it gives really bad answers or just kind of gives the whole universe.
Speaker: I'm curious to kind of continue testing and playing with these metrics to see, discover these kind of, those disparities between the human assumption, some like subjective ambiguity or something like that, subjective uncertainty compared to like the model-based rigorous uncertainty and where those deltas are large.
Speaker: And I think that's the most interesting part of this.
Speaker: It's like looking at the metrics and looking at the response and then coming up with a conclusion that aligns with human intuition about it.
Speaker: To your point, you would imagine that the most ambiguous prompt that you'd be able to write is something that is philosophically ambiguous.
Speaker: And so you'd expect
Speaker: Lots of different answers.
Speaker: The model could be glib and say, or masquerade as someone being glib on the internet, and say 42.
Speaker: I would imagine that in the pre-training set, there's a significant amount of people asking, what is the meaning of life?
Speaker: And a lot of people answering in a glib way, saying 42.
Speaker: Could have said that, could have been pre-trained on a whole bunch of religious texts, and it could have been spouting any part of that.
Speaker: But instead, it chose this fairly politically correct answer, which is probably the right answer to give from this type of model, which is there is no answer.
Speaker: And it is a philosophically hard question to answer and blah, blah, blah, blah.
Speaker: It's just very interesting because you could look at that and be like,
Speaker: Oh yeah, this breaks my assumptions as a human about what this model would likely do given how much I've played with it so far.
Speaker: And so I can kind of assess and most likely correctly assume that this model has been fine tuned on this answer.
Speaker: So it's an interesting, it's an interesting area for us to sort of like continue researching just like
Speaker: Can we figure out areas in which these models have been fine tuned in a way that perhaps break human assumptions?
Speaker: Yeah.
Speaker: And it might give us a little bit more insight into the areas of limitations that these models introduce.
Speaker: So I think we've covered some good examples.
Speaker: I think we all have a good intuition.
Speaker: And again, go play with this.
Speaker: Don't take our word for it.
Speaker: Don't let us talk at you.
Speaker: Go play with this and test your own assumptions.
Speaker: That was really honestly a fun exercise for me, kind of seeing this reflection and discovering what I felt were interesting patterns.
Speaker: So anybody listening, please go play with it.
Speaker: What's next?
Speaker: Continuing this observability work, diving deeper into uncertainty, where does this go?
Speaker: Yeah, so there are a couple of things.
Speaker: I mentioned earlier that we want to continue producing research that gives observability at a task level because that's frankly what people care about, all things considered.
Speaker: It's just like, I want to use this model for this specific thing.
Speaker: I want to know how good it's going to be at performing this thing.
Speaker: So we're going to continue building metrics and scoring mechanisms around that.
Speaker: The obvious one is just like, here's my prompt, here's the model, and here's the thing I'm trying to accomplish.
Speaker: How close can I get to actually accomplishing that with my given prompt?
Speaker: So that's roughly the direction of what's next.
Speaker: I don't want to leak too much, but we do have a fair amount of insight now, given the work that we've already done into
Speaker: how those types of things can be accomplished.
Speaker: And all of this is kind of building into a fairly cohesive set of metrics and therefore a potentially automatable process, which we're pretty excited about.
Speaker: So more still to come, but all of this work basically builds up into at minimum a workbench where anyone writing prompts can sit down and just say like,
Speaker: Here's how good of a job my prompt is doing at performing this particular task.
Speaker: And is this model even the right model for me to be using to perform this task?
Speaker: Or is there another model that's better?
Speaker: So yeah.
Speaker: So in so many words, getting us away from the art of prompt engineering and a little bit more closer to the science.
Speaker: Yeah, and one thing that I neglected to mention earlier, but it's still very, very interesting, is the topic of, we've been focusing all of this on prompt engineering, but we would be remiss to not mention that prompt engineering and fine-tuning are two sides of the same coin.
Speaker: Prompt engineering is a way that you introduce context to the model so that it performs the task that you want it to.
Speaker: Fine tuning allows you to close the aperture so that the model will more reliably produce the shape of output that you care about.
Speaker: So the question then becomes, when should I stop prompt engineering and when should I start fine tuning?
Speaker: That needs to be a metrics driven process.
Speaker: And one thing that we've realized with some of these uncertainty scores is that you can
Speaker: You can quantify at least some part of that.
Speaker: So for instance, you keep engineering your prompt, you're not able to get structural uncertainty to go down below a certain level, or you're not able to get conceptual uncertainty to go down below a certain level.
Speaker: Maybe it's time for you to start fine tuning, given that.
Speaker: So we're working towards not like prompt engineering is one part of it, but really we're trying to like look at the whole spectrum, starting with prompt engineering, because that's where the most ambiguity is, but providing insight into how the whole system should work.
Speaker: That's a great point.
Speaker: It's definitely an open question.
Speaker: I can certainly tell you amongst our customers that no one has the answer of when do you fine tune versus when do you put more into prompt engineering.
Speaker: So applying literally any quantitative rigor to that, I think will be much appreciated in industry.
Speaker: Uh, well, awesome.
Speaker: I had a lot of fun.
Speaker: This was a, a really fun conversation.
Speaker: I had a ton of fun, honestly, doing some of the research and playing with the demo.
Speaker: Uh, again, everybody listening, please go play with it.
Speaker: Uh, watchful.io slash resources.
Speaker: It's our latest blog post.
Speaker: Uh, just wrapping up here again, my name is John Singleton.
Speaker: I'm co-founder and head of success at Watchful.
Speaker: Check us out at www.watchful.io.
Speaker: Like, subscribe, follow our podcast wherever you get your podcasts.
Speaker: And thanks, Cheyenne.
Speaker: Thanks for joining me.
Speaker: And as always, having such a great conversation about some of the good work we at Watchful are doing in the realm of generative AI.
Speaker: Of course.
Speaker: Thanks for having me.
Speaker: Awesome.
Speaker: Appreciate it.
Speaker: I'll talk and look out for the next podcast soon.
