Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
The AI Memory Hack Every Marketer Needs to Know
239 Plays3 months ago
In this episode of AI-Driven Marketer, Dan Sanchez dives into the common misconceptions about AI, particularly around its so-called "hallucinations" and inaccuracies. Dan argues that many AI mistakes are actually the result of user error rather than faults in the AI itself. He explains the difference between AI's short-term and long-term memory, how they function, and why understanding this distinction is crucial for getting better results from AI. By maximizing the use of AI's short-term memory, marketers can significantly enhance the accuracy and relevance of AI-generated outputs. Dan also shares practical strategies for loading information into AI's short-term memory, offering examples from his own work, including newsletter creation and educational tools for homeschooling.
Key Takeaways:
- User Error vs. AI Error: Many AI inaccuracies are due to improper use or misunderstanding of how AI functions rather than faults in the AI itself.
- AI's Memory Systems: AI has both short-term and long-term memory, with short-term memory being crucial for accurate and relevant outputs.
- Maximizing AI Accuracy: Loading detailed context into AI's short-term memory improves the quality of its outputs, reducing reliance on its less reliable long-term memory.
- Practical Applications: Dan shares examples from his work, including using AI for newsletter creation and grading homeschooling assignments, to illustrate how to effectively leverage AI’s memory systems.
- Future Potential: As AI’s context windows expand, the potential for even more accurate and comprehensive AI-driven analyses will grow, opening up new possibilities for marketers.
Timestamps:
- [00:00] - Introduction: The issue of AI inaccuracies and the root causes.
- [01:30] - Explanation of AI "hallucinations" and why they happen.
- [02:52] - How AI's long-term memory can be unreliable.
- [04:30] - Introduction to AI's short-term memory and context windows.
- [06:45] - Strategies for maximizing short-term memory: prompts, documents, web searches.
- [09:30] - Real-world application: Using AI to create more accurate newsletters.
- [11:10] - Custom GPTs: How they help streamline tasks and improve output quality.
- [13:45] - Dan's homeschooling example: Grading with AI.
- [15:30] - The future of AI memory: Expanding context windows and what it means for marketers.
Recommended
Transcript
Introduction and Common Criticisms of AI
00:00:04
Speaker
Hi, welcome back to the AI Driven Marketer. My name's Dan Sanchez, my friend's Kami Danchez, and I am getting sick and tired of people knocking AI for things that are actually their fault. Now you might have heard it said that AI hallucinates, it gets things wrong, it can't remember dates, it counts incorrectly.
00:00:21
Speaker
All
User Error vs. AI Shortcomings
00:00:22
Speaker
those things are true. It can get things wrong, just like all tools can be misapplied. The problem is, is not AI, it's your understanding of how AI works. So if
Understanding AI's Deceptive Simplicity
00:00:36
Speaker
you want AI to be more accurate, if you want AI to perform better for you, listen to this episode because chances are, if you're not getting what you wanted at AI, it's probably a user error, not an AI error.
00:00:48
Speaker
The funny thing is, AI's become so easy to talk to and it seems so smart that when it makes mistakes and doesn't fact check itself, it just seems like, oh, it's it's wrong, it's an error. But there's a few different things to keep in mind. So follow along with this episode. I'm gonna try to explain something that's somewhat abstract. I don't have any visuals for you, so all yeah you audio listeners, I'm not gonna throw any ah slides or anything into it.
AI Memory: Human vs. Machine
00:01:13
Speaker
But there's two different ways of working with AI. And it has to do with the AIs, what I'm calling memory. And I've heard a few other people use this now, so I'm not the only one using this language, but I find that it's a a helpful metaphor into how it works by comparing it to how a human brain works.
00:01:31
Speaker
So the way we work as humans is that we have short-term memory and long-term memory. It's a well-studied psychological phenomenon that we have, things that have happened recently to us versus things that have happened a long time ago. If you ever watched the movie Inside Out, they try to they had a fun time, Disney Pixar had a fun time visualizing short-term versus long-term memory, right? ah AI has the same thing. It has short-term and long-term memory.
00:01:56
Speaker
and where people go off the rails is when they take a prompt and ask it to do something we relying completely on long-term memory. The problem is long-term memory is faulty. What is long-term memory in AI though?
Long-term Memory in AI
00:02:10
Speaker
You gotta remember these AIs are trained on all the information that's out there on the internet or at least a lot of it, right? All the podcast transcripts and YouTube transcripts and blog articles and books and books and books.
00:02:23
Speaker
and libraries of news articles from the last hundred years, right? It's read all these things. It knows a lot of dates and facts and figures and all kinds of things that have happened. And it's somewhat reliable. like Just like if we had humans that remember their whole lives and the things that have happened, it remembers a lot of things relatively well. The problem is there's always holes in those in those memories, especially since it's reading sometimes conflicting information, it gets things wrong.
00:02:51
Speaker
But that's long-term memory for you. It's great at remembering patterns and leading up to what we can do to make decisions today. But it can be faulty. AI has
The Role of Short-term Memory (Context Window)
00:03:02
Speaker
it, and that's the reason why it hallucinates a lot, is that long-term memory. So if I'm talking about long-term memory, then naturally you expect his talk me to talk about short-term memory. And that's where a lot of the real action happens with AI, is if you can feed ah feed it short-term memory.
00:03:17
Speaker
um Some people call it a context window, and you've probably heard this in AI related to tokens, right? What's a token? A token is a piece of information. It often represents like three quarters of a word, so 125,000 tokens is related to about 100,000-ish words. um That's ah AI's ability to have a short-term memory.
00:03:41
Speaker
right The larger the context window or the amount of tokens it can handle at a time is the amount of short-term memory it can handle at a time, which right now OpenAI is about 125,000 tokens at a time. Google's pushing a million or two two two million tokens, which is really helpful, but I generally just stick to OpenAI because it's still kind of on the edge. It kind of goes back and forth with other models, but I'm i'm using that for now. um As AI gets better at short-term window, we'll get bigger and better and we'll be able to take more into account, which means it'll have to depend less on ah the long-term memory and just focus on short-term. Here's why it matters though. The more you lowered load into the short-term memory,
Enhancing AI Accuracy with Contextual Information
00:04:22
Speaker
the more accurate it becomes. If I ask it to write a blog post about a topic and maybe give it a title, maybe even
00:04:29
Speaker
and out a rough outline, it'll be able to fill it in with long-term memory. But chances are it's still going to be generic, it's not going to be that good. But if I give it a really good podcast transcript and outline the blog post based on what was said in that podcast transcript, and I'm load taking that podcast transcript and I'm loading it into short-term memory and then asking it to go off that,
00:04:48
Speaker
That blog post becomes way more accurate. It becomes way more sustentative, and it has to pull less from long-term memory to fill in the little tiny gaps. It's only filling in small gaps versus large gaps when it comes to writing out the content, which is why it works so much better and is way more accurate.
00:05:07
Speaker
the more information you can load into the short-term memory, the better AI becomes. And as somebody who's engineering now AI processes, I'm always trying to figure out ways I can load and absolutely maximize the short-term memory so that it can produce more accurate, more concise, and just more dependable results.
00:05:28
Speaker
even especially as I'm automating it and can't even check all the things that it's doing, I want to have good results. So there's a bunch of different ways to load information into the short term and I'll just kind of list them all off and then I'll give some practical examples of how I'm overcoming the long term memory limits and maximizing what it can do in the short term memory limits.
00:05:46
Speaker
So the first one is just your prompt, right? Like what you give to it. The more context you give to it, the better the outputs it's going to give to you.
Feeding Information into AI's Context Window
00:05:54
Speaker
So don't write short prompts. Tell me a story. Like give it a long prompt. Tell me a story about a man who met a woman and then they broke up. And I don't know, like get paint a picture for it. Like you don't have to tell it the story, but give it more parameters in order to give it give to you what you are actually looking for.
00:06:12
Speaker
um Just the other day I was working with a client and trying to write an email sequence and I had it, I loaded it up with information on the client and of course it wrote a better sequence. But there's other ways to do it and I'll use that example to kind of like talk about different ways you can load up the short term memory.
00:06:28
Speaker
Another popular way that I love doing it with with chat GPT is putting in a document. Yes, if I have a PDF of something that has some really relevant information, it helps to not only put a prompt in the prompt engine, but to actually upload a document, a PDF with maybe some research or some findings or information about the topic that I'm talking about.
00:06:48
Speaker
um I recently, again, that client, I uploaded a a kind of like a shortened version of a book that client had written in order to educate the model on the methodology the client uses with its with their clients. And it was really helpful because I had it write a whole summary of the document, and then it was going off the document plus its summary in order to inform the outputs that I was asking for later.
00:07:11
Speaker
You can also do it with um CSVs or Excel sheets or different Google sheets now I'm using more and more often since it can now correct connect directly to Google Drive. But there's a number of different document types that it can read, and I use them more and more. I'm just dragging documents in, including them with the prompt for context and information for it to pull from. Again, loading it into that short-term memory.
00:07:33
Speaker
The next one is web search. I talked about this in the last episode. you know They were talking about launching church search GPT, but you can search already in chat GPT. Just tell it to do a search on Bing and it can go and find some context for you and go and search
Real-world Applications of AI Short-term Memory
00:07:48
Speaker
the web for it. If you kind of know what's already searching for ranking on a website, then you can use that those search results in chat GPT to better Freshen up its short-term memory, especially if you're doing some so ah something on That's happened recently like these models are only trained up to a certain date So if you want more accurate information or something that's happened recently in your industry or regarding a person or whatever Have it go and do a search and scrape what's currently going on with that person, right? Tell it what search parameter is to use or what search query to use in order to find something accurate and it can come do a summary dump it into a short-term memory and now it has fresh information to actually work from
00:08:28
Speaker
The next one is using a custom GPT. Now I've talked extensively about custom GPT so I won't go into all the use cases and what they do here, but it's essentially a slightly customized version of chat GPT and the other models like Anthropic have their versions of this thing too.
00:08:43
Speaker
um And you can use the instructions area to load it with information to give it more context so that you don't have to reload it over and over and over again. If you find you're uploading a document to chat GPT, the same one over and over again, you should probably just build a custom GPT and put that in the instructions. Or in this next one, like you can do it custom GPT via the instructions area or via its documents area and you can just have the document there permanently and just reference it every time you do a GPT instead of having to upload it every time.
00:09:12
Speaker
And lastly, with a custom GPT, you can also do some sophisticated things by tapping into an API. It's a little bit more technical, and honestly, I haven't even used it yet. I don't know a lot of people that use it, but I know it's there. For those who know how to deal with APIs like manually, that's available to you.
00:09:30
Speaker
But for the most part, we're using those. We're using prompts. We're using documents. We're using web searches. We're using custom GPTs to preload the instructions and upload documents. Those are the
AI in Homeschooling
00:09:40
Speaker
main five ways that I'm doing it all the time in order to load the short-term memory so that I get back accurate results that I'm pretty pleased with consistently. So here's three projects that I've done recently that I've documented here on the show and how I'm using it to load short-term memory.
00:09:53
Speaker
The first one is this newsletter builder I've made. Now I'm using a program called High Level that has kind of like a marketing automation journey builder. And since it's not a custom GPT and I can't load documents into it, I essentially have to build this super long mega prompt and inject information into a ah prompt that goes to the OpenAI's API to bring back the response.
00:10:17
Speaker
um But I'm doing a ton of different things, like when I'm building this newsletter, I'm putting the trans... I essentially just fill out a form real quick with the title of the podcast and the transcript, and I don't do anything else. It goes and takes a pre-built prompt, injects the transcript into it, injects some information about the show, and then injects the title into it. It comes back with the newsletter content then, and it does that a couple of times to get different sections of the newsletter out.
00:10:41
Speaker
um But I'm depending on its short-term memory by giving it the whole podcast, by giving it the title, and giving it a very, very long, beefy prompt in order to give it very clear step-by-step instructions of how to put it together. Now, it's not doing it multi-prompt. It's just doing it all in one prompt often.
00:10:58
Speaker
um but um'm gi up I'm loading its short-term memory up as much as possible so that it doesn't have to depend on its long-term memory to pull data in order to format the newsletter. The next one I'm gonna talk about is my showrunner, ah which is something you can find at myshowrunner.com. It's a custom GPT that I've built, and you can go find it in the GPT store, but if you go to myshowrunner.com and subscribe to my newsletter, I built this with Susan Diaz, um it'll actually take you to a a landing page afterwards that gives you the full instructions and documents that runs the GPT. So if you ever want to figure out how gb a custom GPT works, that's the best place to start, especially if you have a podcast because it's a it's a podcast pre-production assistant is what it does. It helps you build out your episodes with interview guests. I use it every time I i have an interview on the show now.
00:11:48
Speaker
And that whole GPT is designed to load the short-term memory. And what it does is it proactively asks you questions as the show creator to give it to the things that need it needs in order to actually give you great results. So it's like, hey, who are we interviewing today? You give it a name? Great. You want to paste in that person's LinkedIn profile? Again, we're using a prompt taking the LinkedIn profile data to educate the GPT, put it into short-term memory. It then goes and does a web search, again, bringing accurate and fresh information about the person you're you're interviewing and brings it into short-term memory. So it's like you got the name, you got the LinkedIn profile, you copy and pasted a My Showrunner, and then it does a web search to go and refresh and get or grab anything new that's happened with that person to then educate itself on the guest that you're interviewing. It then goes and asks you like, hey, what angle do you want to take?
00:12:37
Speaker
Now, because this GPT is loaded with information about your show and it now understands the guest, it can actually ask you the question, cool, what are we talking about? And it'll you give it an a general description of what you want to talk to them about, it feeds you back five different angles. And those angles are based on what it knows about everything that's going on in its short-term memory.
00:12:59
Speaker
Right? And it's using its long-term memory to fill in the gaps. And then it goes and write creates titles and then creates a show outline based on everything it nodes. The reason why this particular custom GPT is so popular, I keep getting feedback. People saying they love it. They love working with this one.
00:13:16
Speaker
is because it takes the thinking out of loading the short-term memory. That's why this custom GPT works so freaking well is because it it makes it it doesn't make it doesn't require you to think about what it needs to know. It proactively asks you what it needs to know in order to deliver results.
00:13:31
Speaker
But if you can learn how to do it yourself, you won't have to have AI proactively prompt you. You could just give it everything it needs on the first pass. So in the last one is a really fun one that I'm working on now. um' ah I'm a homeschool dad. I teach my kids history, ah science, and Bible, and they have to write journal entries.
00:13:51
Speaker
in their note about what we learn about each day, about the history and the science. and ah Grading is a pain. so like I often sometimes miss grading, sometimes I just enter it, sometimes I grade it, sometimes I don't. um But what I've done recently is I used AI to come up with a rubric for their read their writing assignments.
00:14:10
Speaker
And what I do now is I take a picture with my open AI app and or a chat GPT app in my phone, it amazingly can read their writing. We're talking about elementary school, middle school, their writings. That's okay. I teach also teach them how to type because that's become a bigger, bigger piece of, I don't know, adult life is typing more than writing, but um It reads their writing and actually analyzes it against the rubric um in order to grade it. and it it is It's remarkable how accurate it is. so What am I doing to load it into the short-term memory? One, there's instructions that I've given on what it's supposed to be due what it's supposed to expect. It's expecting a picture of me from
Future of AI's Context Windows
00:14:48
Speaker
either my son or my daughter's daily journal. I actually don't give it context on what we learn, but based on the image that I give it, it understands the context. This is where that long-term memory comes in handy, because if therere my son's writing about Magellan like circumnavigating the globe, it can go to long-term memory and have a rough idea, a pretty good idea of like what that topic is and what were're what he was learning about, and then it can go ahead and analyze the image.
00:15:12
Speaker
um and then give me back a grade. So I'm using a custom GPT that's loaded with instructions as far as the context of what this image is for and what I wanted to do with it. It has the grading rubric already pre-loaded in there that I co-created with the AI. And then it gets the image from me that has the writing for my son um so that it has the context of what generally what was learned about and his take on it.
00:15:37
Speaker
um So because I'm loading all that stuff into the short term memory, its ability to grade it and contextual contextualize the grading ah is very smooth and delivers wonderful results that speeds up my homeschooling dramatically. And of course, I read the grade and under and I read the summary from AI and hand that over to my son or my daughter to be able to make corrections. um But I'm reading it as I'm passing it along to make sure the AI is not screwing up somewhere. And I can tell you it's it's very accurate and is very good at delivering that output.
00:16:07
Speaker
So, hopefully this lesson has kind of educated you on how to approach AI, the difference between its short-term memory and its long-term memory. The one last thing I'll leave you with is that as its short-term memory, the context window gets larger, that's going to open up so many more use cases for us marketers. I mean, right now, Window is 125,000-ish tokens. It can take in, and that's roughly 100,000 words. It's one long book, right? A normal business book is 60,000 words. So it's a longer, it's like a, it's a pretty long book. It's not the whole Harry Potter series. You have to get to like 2 million context, a context window of 2 million tokens for that.
00:16:47
Speaker
But imagine when we get to the day where it's like 10 million, 20 million, 50 million, 100 million, what's then possible? Well, right now you can't just feed AI your whole CRM, especially if you've been in business for 10 years and there's thousands of records in there. You can't feed it because of the short context window, your whole Slack account, and it and expect it to understand the context of every conversation ever said.
00:17:11
Speaker
because its long-term memory is it's a little fuzzy, like our long-term memory, but its short-term memory is very sharp. So as that context window ah opens up, think about the possibilities of us feeding it the whole database once a day, or our whole organization's Slack window, like Slack conversations once a day, ah all all the social media that's taking place and everybody that's interacted with us on social um over the last couple of years. Well,
Conclusion and Future Optimism
00:17:40
Speaker
Now all of a sudden, AI is going to be much more accurate in its analysis of what's going on, be able to highlight things that we're not paying attention to, be able to find opportunities, find threats, so we can strengthen our organization, so we can manage better.
00:17:56
Speaker
That's the kind of future we're looking I'm looking forward to. As the context window grows, we're going to be able to feed it vast sums of data and actually expect it to be able to do a pretty good analysis based on what we want to know. So right now, the context window is short, but that's going to grow over time. In the meantime, figure out how to maximize the short-term memory. ah Even while it's short, it's pretty good at being able to accomplish most tasks and projects if you were able to feed it with the right information to give it the context it needs.