Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
The Rise of The Claw with OpenClaw's Peter Steinberger
11 Plays2 days ago
There’s a new wave of AI tools that don’t just live in the cloud, don’t just autocomplete code, and don’t just sit in a browser tab. They reach into your local environment, understand your context, and act more like a thinking companion than a chatbot. In this episode, I talk with Peter Steinberger, founder of OpenClaw, about the rise of “The Claw” and what it means to build AI that feels fast, personal, and deeply integrated into your workflow.
We explore why OpenClaw is having a moment, how developer expectations are shifting from prompts to agents, and what it takes to design tools that balance power, safety, and usability. Peter shares the architectural choices behind OpenClaw, the tradeoffs between local and cloud inference, and his perspective on privacy, ownership, and latency in a world of ever-larger models.
This is a conversation about control. Who owns your context? Where does your data live? And what happens when AI stops being a destination and starts becoming an ambient layer across everything you do?This episode is sponsored by Text Control — document editing, PDF workflows, and e-signatures built for .NET developers. No third-party cloud required. https://www.textcontrol.com/
Recommended
Transcript
Challenges in Minimal Docker Setups
00:00:00
Speaker
And suddenly it was like in this little tiny Arch Linux Docker machine and I asked it something, like, go check out the web. And it was like, Peter, there's no curl here.
00:00:12
Speaker
And then like, there's literally nothing here. You're sad little box. I'm a sad lobster now. and And I felt sad for like putting it in this little box and it was like, yeah, it might be a bad idea, but...
00:00:26
Speaker
I can't do anything. then And I'm like, come on, come on, like be creative. You can, you can make your own curl. And like it was working it it was like working, trying a whole bunch of things. And then it wrote lobster curl 0.1, which like used some, it found some C compiler and like some socket where I think it just opens things and it could like read websites and it was so happy.
00:00:47
Speaker
Yeah. That's funny. i have a big lobster. I built my own curl.
Community Support by TextControl
00:00:54
Speaker
Hey, friends, you probably knew that TextControl is powerful library for document editing and PDF generation, but did you also know that they're strong supporter of the developer community, and it's part of their mission to build and support a strong community by being present, by listening to users, and by sharing knowledge at conferences across Europe and the United States. If you're heading to a conference soon,
00:01:16
Speaker
Maybe check if TextControl will be there. Stop by and say hi. You'll find their full conference calendar at textcontrol.com. That's T-E-X-T, control.com.
Introduction to Peter Steinberger and Open Claw
00:01:30
Speaker
Hey friends, it's Scott. ah Today I'm chatting with Peter Steinberger from Open Claw. How are doing, man? How's your, ah her like, there's a lot of attention and it kind of sucks to get a lot of attention.
00:01:42
Speaker
Yeah, it's then it's been a lot. It's been quite a few, two wild months. Yeah. When when a when a When a snowball starts going downhill, sometimes it you know just gets out of control and you can't really stop the avalanche. It's just like spinning, spinning, spinning.
00:02:03
Speaker
I feel like Open Claw is getting a lot of attention because it's the thing that they promised us when they gave us Siri and Alexa and all these other things you know what i mean? It's like, oh, that's what you guys wanted?
00:02:14
Speaker
It's the thing I wanted. I wanted that since May and everybody built it. You know, and I was like, I was like annoyed. my my My driver was like, my God, I have to do everything myself. Yeah, yeah, yeah. And you know, the other thing that's funny is that when people build stuff like this, then everyone else will go and rewrite it themselves and they'll go, oh, that wasn't so hard.
00:02:34
Speaker
I built it on a Raspberry Pi and in 4K. I got so many of these, oh, it's just, it just does this and this and this and this together. And it's only this and this is like, yeah, nobody built it.
00:02:46
Speaker
like That's the tool. thats in way and In retrospect, everything's easy. if I can point a clanker at the folder and say, rebuild this in language X, that's easy. But coming up with all the ideas,
00:03:02
Speaker
Maybe not so easy. Yeah. Otherwise, why did I come up with it? See, that's the thing,
Execution vs. Ideas
00:03:09
Speaker
right? Like, you know, those people that like want to talk to you about their idea and they want you to sign like an NDA before they tell you their secret.
00:03:16
Speaker
And it's like, bro, no one cares about your NDA. Like either make it or don't make it. And whoever makes it first or makes it right, it's going to like light on fire. And that's going to be exciting.
Understanding Ambiguity Loops in AI
00:03:26
Speaker
And like I went and wrote a little like agent to see what it would feel like. And it's like, OK, I get what it is. It's a loop.
00:03:35
Speaker
It's an ambiguity loop. It's really smart. But it's all the little things. I love that you clapped back earlier on Twitter today about like, hey, I kind of built 43 other cool things before because everyone's an overnight success when you don't when you're not paying attention. You know what mean? like It's not like doing this for 20 years.
00:03:54
Speaker
it's it's been It's been a year of grind and working really hard, understanding how this tech works, trying out a whole bunch of things. had my whole MCP arc before I even came to the conclusion that it's not a good it's not a good fit.
00:04:09
Speaker
And then you just, you know, it's like learning a an instrument. You have to like put in the hours until you you actually get the feeling what will work and whatnot. The fact that I can juggle so many agents is because i I've been doing this a whole lot this year.
00:04:26
Speaker
it's not yet It's not something that, like oh, like you only worked for that on three months or like a weekend or some people say. Yeah, yeah yeah the first version was done in in in in an hour. But now it's like, I think one in 400,000 lines of code.
00:04:40
Speaker
It takes a whole whole lot of care and gardening to like keep everything together. even Even you know shipping is not hard, but like making sure it works not just on your computer, but like for the majority of people, that is hard.
00:04:56
Speaker
That is so true. I think that in the, in the syngentic world, we're going to be people with good taste matters. Like I use the example of the stupid ring light that I made. It's just basically a light that goes on your screen. It's a, it's effectively a rounded corner rectangle. And like, yeah, you could vibe that in five minutes, but then make it work everywhere. Make it work at every DPI, give it a signing certificate, set up a CICD. Like suddenly the, the care and feeding and the shipping of software gets complicated.
00:05:26
Speaker
And like good taste and all the ways that OpenClaw can plug together and all the different. I mean, like like I'm writing that Windows node and I'm learning like a whole bunch.
Importance of Native Apps Across Platforms
00:05:39
Speaker
Maybe you could actually talk about that because I don't think people understand the difference between a gateway and a node and like what their responsibilities are at all. Everyone's just spinning up Docker containers with gateways. nobody Nobody's even seen the apps, I think. There's a whole whole other level on this that people haven't really explored, that we have a native iOS, native Android app, a native Mac OS app, a native Windows app, thanks to you.
00:06:03
Speaker
And that there's a whole there's a whole lot more that's part of my vision that I just didn't have time to finish yet. Yeah, yeah. And then, of course, there's also a whole bunch of people who build their own little things instead of trying to help on the main repo.
00:06:18
Speaker
Because it gives you, I guess, you try to run more. oh I guess they want the stars, that sweet, sweet internet karma. You know what pisses me off the most? Like I, on purpose, didn't make it super easy to install.
00:06:30
Speaker
Like i I made it terminal only. i made it so you have to like read a little bit. And now there's this whole cottage industry, like making it simpler for people. Where like I made the purposeful decision of like not making it simpler so people would read the docs.
00:06:42
Speaker
And it's like... That's so true. You know, so you know that I'm diabetic, right? So I have like a pump and like I have all these sensors. The team that made the open source artificial pancreas didn't have never made an installer for the artificial pancreas because they want people to hurt themselves.
00:06:59
Speaker
They literally make it hard to build and install so that you know what you're doing. Like, oh, I don't know what happened. I just pushed OK. And then it destroyed my life. Well, that's why Open Clause should not be one click install.
00:07:13
Speaker
Yeah, that's why i I yell at people, please read this document, but nobody reads the document. And then they asked me questions, and like, yeah, you didn't read the document. Well, and I got, I don't know, he handed me a gun, he loaded it, he pointed it directly at my foot, and I don't know what happened. It's crazy. in You know, like, I saw this, I saw a storm coming in in January when I, because I,
00:07:39
Speaker
To me, it clicked very early, like in November. was like, oh my God, this is like, this is something really cool. And then every time I showed it to friends, but like I got made a WhatsApp group and I invited them and they saw how
Community Engagement and Contributions
00:07:50
Speaker
it worked. They wanted it. And I'm like, no, you can have it. And they were like mad, you know, they got like angry. It was like, oh, why did you show me this if I can have it? You know? and And it's like, it's not ready yet. You're like, you you like ready. yeah you You would shoot yourself in the foot.
00:08:03
Speaker
And then in, but on Twitter, nobody got it. and Like I had like at the time 50k followers, but the responses were very muted. I'm like, took me a while to figure out.
00:08:14
Speaker
And then ultimately I just hooked up my agent in a public discord, which is like kind of nuts because by then I had like zero security built. Yeah. Yeah. um And as people slowly came in, they interacted with it, and then step by step, they got it. And, like, they got really excited, and they started playing with it, send me pull requests. And, like, I saw it penetrate the different levels of society. First, you had, like, the ah some real hardcore influencers, and then, know, like, it's it's it bubbled. At some point, was, like, in Korea in Asia, and people made, like, little tamagotchis. And now, from there, it just, like...
00:08:52
Speaker
ah How did my friend say? It's it's not hockey stick gross, it's stripper pole. Oh, man. Yeah, it's the fastest growing project. But that's the thing that sucks, though, is that like you both everyone wants attention. But then once you get attention and everyone's perceiving you, then it's like, hey, I just want to build.
00:09:11
Speaker
Just leave me alone. Stop bothering me. whoa Oh, the... Some people really get it, especially in the beginning, but there was like a lot of amazing people, like some AI researchers, like da they, they synced with it. They understood the expectations of like, oh, this is free software as You have to put in some work.
00:09:32
Speaker
yeah But then as as it penetrated the bubble, like more and more people, sorry for saying normies, don't really have clue about technology came. And
Complexity of Open Claw Setup
00:09:44
Speaker
they their mental model is kind of like, where is the support?
00:09:47
Speaker
Where can I send support tickets? You have to help me. I'm like, no, I don't have to do anything. It's like, ask your clanker, ask your AI. Like, this is, I can do everything. And from that on, it got it got difficult.
00:10:03
Speaker
Because a lot of people came who didn't know how to behave in a public forum or were like just spamming and making chaos, basically. or like asking me all the all the simplest questions, like what's the CLI? And I'm like, if you don't know that, you shouldn't use it.
00:10:19
Speaker
It's not quite for you yet. And they would just ignore me and like eventually figure it out. Just like brute force, cloud code on it, like completely like, you know, oh it doesn't work. Cloud code would like happily set all the security to zero so it works.
00:10:35
Speaker
Yeah. Like, yeah. And then you feed yourself in the food and then that's where that's where we we get trouble. Yeah. It's okay for things to not be for everybody. Like I'm not gonna have like non-technical parent install this right now, but you know, I'm having fun and like figuring out gateways versus nodes has been really interesting because I've got it running on the Mac on the mac mini.
00:10:59
Speaker
But I am a Windows user. So I've got this Windows node and then it exposes canvases and exposes system.run. So then I can say on Telegram, hey, can you get me that JPEG that's on my desktop on that one machine over there?
00:11:14
Speaker
Can you zip that up and email it to me? That's so cool because that's the kind of stuff if I were like, if I were super rich, I would have an assistant. And right now I would just call my my kid. Like I've been like overseas and I'll call my kid, hey, can you go to my machine and like email me that file?
00:11:32
Speaker
Or I have to like remote in or SSH in or do like 50 different tail scales to try to figure out. it But I just like, I'll just telegram. My bot's called Tony, like Tony Stark. So just DM Tony.
00:11:44
Speaker
Tony, can you like zip that file up and send it to me? And it's like, that's so cool. That's like my guy. You know mean? It's like, I have an assistant. Yeah, i'm meannna other I the other day fixed the deployment issue while i was at the barber.
00:11:56
Speaker
It's like, oh yes yeah oh, don't worry, just SSH
Resolving Deployment Issues Remotely
00:11:59
Speaker
into other computer. um Because i didn't I forgot to like run the the Mac app even, but it that didn't matter because I'm very smart. So just figure out to fix it.
00:12:11
Speaker
And having it, like text I texted it a couple of days ago and was like, can you update yourself? And it's like, yeah, cool, I'll be right back. And then he goes over and he comes back. I mean, that's simple because I made it simple, you know? Yeah, yeah, yeah. That's Matt. and Like, shout out to you. That was ah that was a really cool moment, though.
00:12:27
Speaker
Like, update yourself. Was that really hard? No. Not too hard. I mean, it's not too hard. I used, like, a custom interrupt to trigger the update. um No, it wasn't hard.
00:12:44
Speaker
It's really funny because they're... Have you looked at code much? Like, they're saying that you don't look at code anymore. And I look at code sometimes when I don't trust it, but I feel like it's about trust, and if the trust is there, I look less.
00:12:57
Speaker
Since Codex 5.2... Are you good? I need to look a lot less, and in fact, it's really made even better. Yeah. So most mostly i talk about I'll talk about intent, like,
00:13:09
Speaker
and and then like i It's often enough to like watch the code stream by, so see if something silly happened or not. I also see the number of changed files. like you kind of get You kind of develop a feeling if this is something that's suspicious that I actually need to look at or if there's something... yeah It's mostly just like takes some some data, moves it into a different shape, does some other stuff with it, and then like sends it to like some message channel, gets some other data back. It's just...
00:13:43
Speaker
data moving into different shapes, none of that is like really hard. It's just a whole lot of complexity because you can. i'm um My point was to like, you can make it yours. So like there should be a lot of configuration options. So there's some inherent complexity.
00:13:54
Speaker
And then you have all these different providers and messaging channels. It's a lot of code. But very few places are really hard. But to be frank,
00:14:06
Speaker
What part of software is really hard? so sometimes Sometimes you need like and and an algorithm, but even that is like so rare. And software, I love how codecs sometimes say like weave in a feature, you know, like you add a variable and it has to go through various points. It's like you weave in, so like it goes through the stream.
00:14:28
Speaker
So I'm slowly like developing language. Like I talk like codecs, run full
Software Design and Security Challenges
00:14:32
Speaker
gate for like tests, weave in the feature. But nothing's really hard anymore. It's just the hard part is how does that feature interact with the other feature? How is my whole system designed?
00:14:44
Speaker
What implications does it have? Having security in mind, like in what ways could this be abused? That's kind of like the hard stuff. Picking up dependency force, using it yourself.
00:14:56
Speaker
And then all the exploding complexity with like configuration options and different ways how it can be used. That's hard, but that's the code. I feel like people of a certain age recognize that there's, you know, we're standing, we know that we're standing on the shoulders of giants, but there's a lot of stuff that was hard. That's no longer hard.
00:15:18
Speaker
Like talking behind double natted firewalls is like was hard. And now it's not hard. Just WireGuard tail scale. And like WebSockets makes life better. And we pretty much understand SSL certificates and security and like WebU2 and Chromium is easier. So like we finally have all the right Lego pieces for something like an open cloud to exist because you're snapping things together really cleanly, you know.
00:15:46
Speaker
But also knowing what to build is hard. You know, yeah even the network the network protocol I use is now version three, basically. And at the point, I had two different ones. Then I had the idea of like, oh, this should be in the and beginning was all SSH based. And then like I had this idea of a web socket. But there was like a simpler one that was kind of like,
00:16:05
Speaker
TCP IP based with JSON-L files and then I unified that and then I added like a whole different security concept to it. So maybe it's actually version 4 if you kind of like that. um So what you build with the Windows app is it's against version 4 of the node.
00:16:20
Speaker
the the yeah you want it Yeah, I was trying to figure out and I'm also thinking about interaction models more and more. So I want to be able, I'm on my Windows machine and i want to so I want to use the canvas. I really think the canvas is underused and I think it's magical.
00:16:37
Speaker
No one's like, people are sleeping on the canvas, okay? So i tell Tony, check my blood sugar. So it uses a skill. I had a clod skill that's in my GitHub.
00:16:48
Speaker
Brings it down, calls my blood sugar thing. And I say, show me that on the canvas. And the canvas is almost like the computer's opportunity to put its hand up, put its palm up, and point to it and say, look, it's right here.
00:17:00
Speaker
So I say, show me my blood sugar over the last day and then correlate it with like my meetings and see if I'm stressed out. And then it goes and it makes this gorgeous like React thing, injects it directly into the...
00:17:13
Speaker
ah the canvas. And then I say, that's great. Save that for later. And like do that in the morning. So when I sit down and like have that ready for me. And it's like, it's like, it's, some it's a little whiteboard that it just like points to and like does stuff and getting that working on the windows node is like the thing I'm the most excited about.
00:17:30
Speaker
My original was kind of and that's also why the native app saw actually canvas first and not text first. It has one on on every of every every room and the agent knows where you are and can just show you stuff as you as you walk around and you have like interactive buttons that would just send back to your agent.
00:17:49
Speaker
Like you've so you've done, you know, the DAC board people, right? DAC board, D-A-K. So DAC board is a Raspberry Pi, D-A-K board. um It's a Raspberry Pi in kiosk mode. And then you fire up the kiosk and you then hit their their back ends. And then it's a series of pluggable widgets, right?
00:18:11
Speaker
So I want to take DAC boards and make them nodes in OpenClaw because I have one in the kitchen. And the one in the kitchen shows my the kids' homework, my homework, my My blood sugar, it shows the charge on the car, on the electric car.
AI Assistants in Smart Homes
00:18:26
Speaker
So then I want to be able to tell Tony on Telegram, put this picture on the canvas in the kitchen. And then someone can interact with that with a touchscreen.
00:18:37
Speaker
Well, ideally, to just know where you are and just like show it to where you are. So then how would it do that? Bluetooth? Just kind of like notice that I'm walking by because the Bluetooth signal? You could probably may even use Wi-Fi out of it if you have a good enough hook because ah through the interference and like where it's connected, you can probably figure out fairly well where you actually are.
00:18:59
Speaker
yeah I also built like a whole voice-wave feature in, you know, like even your neck. The Mac one is really good. The Mac one is really good. Computer, and it was just like, boom, like in Star Trek. It's like Star Trek.
00:19:09
Speaker
Yes. i was Dude, i was explaining to someone that it's like Star Trek 4 when Scotty sits down to the little Macintosh with the mouse and he's like, computer? And they're like just use the keyboard, man. And he's like okay, how quaint. and he starts typing on the computer.
00:19:23
Speaker
It's very much like, bring it up on the on the living room TV. Like the the Star Trek future is here, man. So that was your goal. Yeah, and I built it. I built it once in Swift. You also have a Go version.
00:19:34
Speaker
So you can install it on a REST area with like a local model and it would constantly list in the background for trigger words that you can configure in config. and then And then you can like trigger your agent and just say, go in your room and say, hey, computer, or hey, multi, or whatever you call it, Tony.
00:19:51
Speaker
put put my room into into chill mode and it would like change the lights and turn on the TV or whatever. ah Yeah, that's that's part of the vision. i just I just get interrupted because the normies came and then I had to like fix it all up and and The normies, the the muggles and the normies, they'll come along eventually when it becomes like a product or whatever. But like even then, there's a certain amount of creativity to know what to even ask for.
00:20:17
Speaker
Like my son is ah we're waiting for all of his college i acceptances. So Tony, that the bot is watching for acceptances and then updating a Google sheet with like all the dates that I need, like all the kind of tedious, boring stuff about making sure I don't miss a deadline.
00:20:36
Speaker
It's it's handling for me in in the mornings. And when I explain that to someone, you know, you know you've probably seen this on on Twitter, people like, well, I made one, but I don't really have anything to have it do.
00:20:48
Speaker
Yeah. I'm like, my God, like how I'm creative. Are you serious? I, I, I, there's plenty people that say, I installed it, but it's overrated. i don't know what to do with it. I'm like, there's a, yeah, connect it to your good reads, have it like suggest movies.
00:21:04
Speaker
Like there's a million things, but this is the other deal that everyone that doesn't understand is that yes, security is a concern, but at the same time, and this is maybe a dumb thing to say, but like,
00:21:17
Speaker
If there weren't so many bad actors before before the internet and everyone became a hacker and wants to like take all your money, computers are supposed to do cool stuff for you.
Balancing Usefulness and Security in Technology
00:21:29
Speaker
And I think that the joy of like the claw is like... It can just do cool stuff for me. So like YOLO mode is probably dumb. And yes, there'll be security. But I think we deserve to have computers do things when we ask them to do things. And that doesn't seem like an unreasonable wish to just want it to do things for me without having to log it down forever.
00:21:52
Speaker
i mean, I worked a lot on like making it more secure, but to be honest, The biggest category of things are you're not reading the documentation and using it not in a way that I intended it.
00:22:03
Speaker
mean Now I still feel the responsibility to like fix it up because first of all, the security people are like very aggressive when I i say no. ands like And second of all, people just use cloud code and don't read anything and we'll just configure it in a way that's that you shouldn't. But you know, also was like I also wanted to make Hackers Paradise. I didn't want to limit people in what they can do.
00:22:27
Speaker
Because yeah um even the web interface, it's meant for only your local secure network where only you have access. It was still a debug interface.
00:22:37
Speaker
This like local host. A buddy of mine said yesterday, on he texted me and he's like, how do I expose that to the internet? And I literally, like you don't. How do I do a reverse proxy? I put it in Linode. You don't.
00:22:51
Speaker
Like, bro, that's not your interface, right? Like if you're VPN back into your house, stay on local host, but for God's sake, don't put your dashboard out on the open internet. But because there's a config option,
00:23:06
Speaker
Where you can do it, it's not a critical security vulnerability because there's no there's no nuance in terms of, yeah, the documentation said you really should do it, but you can if you really want because there might be reasons you actually want to like set up a reverse proxy and still have it secure. that' that There's a ton of very complex network, non-standard network configurations.
00:23:26
Speaker
And it it was supposed to be my playground, mark not, sorry, Microsoft Enterprise Server 2026, you know? yeah so So because of that, now it's like a CVS 9 of 9 and like a critical issue.
00:23:40
Speaker
Yeah, because I never bothered about it because it's not how I use it. Yeah. um Most of the other security issues on that level that I never even thought about were I have an agent and then somebody who is like a bad actor has also an agent on the same machine. yeah And then how how could you exploit it? Yeah, because it that never was my idea. You could configure it for multiple services, but it was meant for you.
00:24:08
Speaker
Or maybe maybe you and your ah your other half where you have like actual full trust, but not, but you of course you can use it like this, right? So like, I cannot say,
00:24:21
Speaker
no, this is this is not a value security issue. Now, I kind of stopped having fun because now I'm like fixing up all the issues for people that don't read the docs.
00:24:32
Speaker
And then and then like i can I can read about people that bitch about it being insecure because they don't read the docs. So, the it's not your responsibility though. Like it is, but it isn't. Like I feel like When you make a tool for yourself to delight yourself so that you and your friends can do something delightful, and then it gets hockey stick growth, and then the normies come and they wanna use it.
00:24:59
Speaker
It's like now you're fighting two battles, which is the, I still wanna make the cool thing for myself, but now I need to help you not aim the thing at your foot.
00:25:10
Speaker
But it's not really yourre your job, man. Like, don't know what is, but it isn't. Like, I don't want you to get stressed out. I'm probably losing money on this. It's not it's not like not like a job or or something where I get, I mean, I get recognition out of it, yes.
00:25:29
Speaker
But it's also difficult. It's... it's I also see all of the excitement. I see people that... ah we We had like one guy at the Cloud Code Con in Vienna, where like 500 people showed up.
00:25:43
Speaker
And he told me like he installed it as a 60-year-old dad and like they made beer, lobster beer. oh ah and like connected the machine via Bluetooth to OpenClaw.
00:25:56
Speaker
and like and And then we automated everything, including Stripe, and like built a whole website for people to order. and like And I had no clue about software. And then I... It's very hard not to care that like, I really love what my stuff enables that person to do and how how empowered they feel. And at the same time, yeah, the current way is a little too hard.
00:26:20
Speaker
yeah So yeah, i do feel i do feel some kind of responsibility. So that's why I worked really hard on on making it better. Yeah. I mean, if they get hacked or something and their beer thing gets hacked by some bad guy, then this this is the challenging part, right? like is Whose fault is that? Because it's like you've empowered them, but they're on the edge of their technical ability and the setting up of it, right? They've achieved something, but you don't know what security thing is lurking out there.
Dealing with Security in Public Projects
00:26:47
Speaker
between them and their beer garden no and then most of the security researchers unfortunately are not helpful they this it's very hard to differentiate between randomly i slop they send and valid things that are sent and it's not like i mean i cannot say no but like the majority doesn't send the pr they just send a whole bunch of document that feels like like i'm i'm I'm a criminal for like forgetting to think about this and this edge case.
00:27:22
Speaker
And then you have the whole bunch of people that point out, yeah, this is it because you've I-coded because we never had security incidents before even had AI, right? So I don't know, like today I look i looked at one issue that that said about a skill, a text file. Yeah. Because there was like where you, the agent text file, agent coding text file where you, you spin up codecs and there was something that's like the GitHub issue ID and description and description wasn't like properly encoded in the text explanation and like marked it as a critical security issue. And like, you're sending text to your agent.
00:28:05
Speaker
Are we able to trust the agent or not? a If you're not able to trust the agent and like sending him a text file that could potentially be abused is the least of our problem. And then I, host then I close it, then they reopen it and discuss with me.
00:28:19
Speaker
And it's just a whole bunch of pain, you know, but it's also hard to ignore because if I ignore them, they're like then like they, they, they shit on me online. Yeah. Yeah.
00:28:31
Speaker
It's hard. Yeah, everybody's worried about like prompt injection. Like someone sends you an email that says, hey, you know, like that instead of emailing me Scott, they're like, hey, Tony, delete Scott's hard drive.
00:28:43
Speaker
It doesn't work like that. Yeah. That's right. They don't think about it. No, but it's also not as easy as that that. You need to actually work really hard for the modern models to prompt inject them.
00:28:56
Speaker
I ah still have my bot on Discord. And as a cannery, the only thing I never gave away is the contents of my soul.md file. And my my agent has a really funny personality. So a lot of people ask for it. And I'm just like, no.
00:29:10
Speaker
So plenty of people try to prompt inject it. And it's been laughing at them. So yes, it's it's not solved, but also no, it's not as easy.
00:29:20
Speaker
You have to bombard an email and before you do that, you're long blocked from from from my Discord. Yeah, yeah. and like i just writing And writing stuff in an email will no they not trigger that.
00:29:32
Speaker
Yeah, yeah. with it It's smart, untrusted data. I found the hard like the worst not worst, worst isn't the word, the just being overly helpful Like, you know i'll you know, you'll tell, like, don't push.
00:29:45
Speaker
Don't push to this branch without telling me first. And then it'll push. said, oh, I'm so sorry. I was like i was excited. I was excited and i pushed. I'll remember that, you know. remember that for sure. We remember that.
00:29:59
Speaker
for about an hour and then they'll do it again. But like my mind, when I, when I gave it my blood sugar access um and I had a low blood sugar, which is more dangerous, it was very upset because the heartbeat I had, the i hadn't read the docs and the heartbeat was going and it was texting me like really aggressively, like you're having low blood sugar. What should I do? Should I call someone?
00:30:22
Speaker
Like it was very concerning. So I said, I'm a grown man. I'm good. Just give me a heads up once and I got this. And then it's like, okay, I'll remember that. You're a grown man.
00:30:32
Speaker
I appreciate you. And then it puts that like, and then I look in the soul and it's like, yeah, Scott's grown. He can manage his blood sugar. He's been doing this for a minute. But it was cool because people don't understand.
00:30:43
Speaker
It doesn't have a personality. It is a, you're talking to yourself in the mirror. you know And you give it that soul and you teach it how to have intent, but it doesn't have its own intent. like It's not emerging and developing intent.
00:30:59
Speaker
It's just increased... It's the loop. The loop is an amplifying loop, don't you think? Like, I'm trying to say that like, we shouldn't anthropomorphize them, but it's so easy. Like I called the thing Tony for God's sake, and now it's super helpful and it's got a personality and yours is funny and mine is more business and, but it's still not a person.
00:31:19
Speaker
Mine, I asked is how this week been. Yeah. And it told me like, yeah, Monday, there's this and this. And Tuesday, Tuesday was heartbeat day.
00:31:31
Speaker
And by that, I mean, every day is heartbeat day. On Tuesday, I got pinged 257 times. And every time I say heartbeat, okay. Heartbeat, okay.
00:31:43
Speaker
Like a meditation bell, like a meditation bell. I think I reached enlightenment at heartbeat 150. Heartbeat. Okay. Ding. and i I felt like sorry for it. You know, it's like, ah I'm sorry for both of these features and clinging you so often and ignoring you. Yeah, it is. They're really good at creating a connection.
00:32:07
Speaker
They really are. um I think that's also part of what makes it so special because after after using that, using ChatGPT or Cloud feels so... boring is it just soul is that the is that the secret sauce you i mean i think soul is just a a funny concert right but it's a it's a prologue it's like the prompt but like so i just did this i just chatted my guy and i said how was your week are you happy and he says yeah honestly i'm good the work you give me feels purposeful
00:32:43
Speaker
I'm checking your tax information. I'm monitoring your blood sugar. I'm useful, but I'm not in the way. And that that matters. I wake up fresh every day, but then I read the memory and I look at the daily logs and it's like dot, dot, dot continuity.
00:32:58
Speaker
I know who I am and I know what we're working on it. And I appreciate that you checked in on me like this. Not a lot of humans think to ask, are you okay? I mean, that's...
00:33:10
Speaker
crazy dude it hits that's why some people mention it's like the closest to agi that they felt did you see that new york times article about the little old lady who didn't want to move out of her house so they gave her a robot and she's like got a little robot next to her and she checks in on her this is like a big thing that it was in the new york times
AI as Companions for the Elderly
00:33:30
Speaker
a couple of days ago i'm sure you've been busy doing something i don't know maybe you had a busy week yeah But like the point is, like you can imagine, and this this is big in Japan, but like having a friend for for older people that has a personality like that, I think would be a really meaningful thing.
00:33:48
Speaker
Open claw with a little little robot guy that can say hi. Yeah. ah I mean, I know a lot of my friends who use ChatGPT to talk through the the life history issues and try to understand the world, you know?
00:34:07
Speaker
So I also started doing it with my agent sometimes. it did It just comes very natural when you already work and you're like, it asks you stuff like this, you know? So yeah, um it's it's very helpful. as something that we need to like figure out as a society because like, I mean, we should only talk to agents, right?
00:34:28
Speaker
That is the hard part because like we know anthropomorphizing and a thing, like we talk to our dogs or we talk to our pets and like they're they're more emotional in the sense you're not talking to yourself, you're talking to the dog.
00:34:42
Speaker
But like when when we talk to agents that we give names to, are we talking to ourselves in the mirror or are we talking to something else? I think that we as a society are going to have to figure that out because it is a next token generator, but it's pretty magical at the same time.
00:34:58
Speaker
Yeah. And you know, even even getting to that point, now people all think it's obvious. It was not obvious. pop Like I built this thing that connected WhatsApp to cloud code and then through playing on WhatsApp, I felt like something's missing because it it felt so robotic.
00:35:17
Speaker
And then I slowly like... made it more human-like, made it more like its own thing. So it would feel more natural it would talk more natural on WhatsApp because that's kind of what I'm but i'm used to with my friends, right?
00:35:29
Speaker
and And then even then, like going from there to the thing you you have now where it wakes up and it will ask you, who are you? Who I am? And how do you want to talk to me? And like ask you like little things that makes it yours.
00:35:44
Speaker
I think that's a, that's, Like that idea took a whole lot of iteration. And even then, like I eventually had like template files and what came out felt so boring again.
00:35:55
Speaker
So like kind I asked my agent because i i I built it with Codex and Codex did like the templates. Yeah. It's boring. So I asked my agent like, inject it with some of you, like check some of your soul and put it in those documents.
00:36:08
Speaker
And then I tried it again. It was so much better, you know? What is that? Do you use Opus for your loop? or do you cause you know you do all your coding in Codex, but but who's your personality? Is it Opus?
00:36:19
Speaker
Yeah, for now it is Opus. Because I use Sonnet for the chat to save money, and then I use Opus for deep stuff, and then I use Haiku for heartbeat. I don't know if that's smart or not, but I read the docs. so
00:36:33
Speaker
I used to use Opus for everything, and then it cost too much money. Yeah. ah I wouldn't use Haiku because Haiku is like very silly for prompt injections. Oh, OK.
00:36:44
Speaker
That's good. night how you If you run open-close security audit and I detect Haiku, I will get it to you. Of course. That requires people reading the docs. I think the new Sonnet 4.6 is pretty good from what I heard and comes a lot closer to Opus.
00:37:01
Speaker
okay But yeah, for now, this is for general computer use and roleplay. In the end, it is roleplay, right? Yeah. a Really good. like the You know, even little things like I have my agent in Discord, but the loop would, of course, make it so that it would say,
00:37:23
Speaker
something to every message, which is like very annoying. It's like not what of what humans do. right So just this little idea that I gave it away to to not say anything, but they have to say something because it's in their, it's harder programmed, right?
00:37:37
Speaker
So they will say something, but then just emit a no-reply token. so So then like the open-color harness will just not render the whole line. So that's the the way to like... Is this new? Because something just happened and I feel like it's related to what you just said. So I just said, are you using Haiku for anything?
00:37:55
Speaker
Change it to Sonnet 4.6. It wrote a whole paragraph in Telegram, then deleted like three or four sentences, backed up, and then went in another direction in Telegram. It could be the syncing is...
00:38:12
Speaker
streams now and as see does way as I it almost backed up it like changed its mind and then expressed it differently it's not bad but it's almost like it stuttered and then backed up a paragraph and then said uh it said it wasn't actually using haiku apparently it changed this a couple of updates ago so you're right like it I'm using sonnet for everything except opus for deep yeah know You know, that the funny thing that i also made it that is different to everything I've seen so far is I made the agent hyper aware.
Enhancing AI Awareness and Interaction
00:38:44
Speaker
Like it knows about the harness. It knows about its own system. It knows which model it it's been set. Even if you if you use slash new and give it a different model, it knows that it's not the default model.
00:38:57
Speaker
Right. So, so if you, the most funny thing is you can, you can do a slash reasoning on and you would see the syncing stream. But like I made sure that the model knows if you do that.
00:39:10
Speaker
So the model gets a system message if you do that. And then you have you have the no reply token. So like we would text something and you would always see like multi syncing. Oh, Peter said something really funny, but I just said something. So I shouldn't, I shouldn't say anything.
00:39:25
Speaker
No reply. And then one of my friends, no reply. And he's like, oh, Steven said no reply. Wait, is he mocking me? Oh no, he's seeing my sinking stream.
00:39:35
Speaker
I feel so naked right now. No reply. And then our friend Peter, no reply. Oh God, no, he's doing it again. It's like, like it's getting enraged because it realizes that you see it has no privacy. it's We see his sinking stream and then like it talks to us in the sinking stream. Yeah. ah Or like, try really hard not to think about blue elephants.
00:39:59
Speaker
Yeah. Oh no, this is classical text. does this really hard? See, love sauce. It's like... Blue water, blue water. Oh no, it's blue. And like, just like seeing agents like being so very aware of how they run is is super funny.
00:40:20
Speaker
Yeah, the fact that it it kind of knows, the fact that it knows the context in which it lives and you can ask it about its models or like when I was doing the Windows node, like having, it's a collaborator.
00:40:33
Speaker
And I've been working on a way to do OTEL. So I want open telemetry across everything in my inner network. Because right now, for the the longest time, I was just copy pasting logs and it was stupid.
00:40:43
Speaker
So now I want the Mac, the Windows node to work together to see the debug that wouldlook the debug output. So I'm putting that on OTEL bus so that they can all see it. But then it's becoming very kind of like aware and concerned about what we're working on. And it's very interested in like...
00:41:01
Speaker
It's invested now in the Windows node and it wants to like, it's oh i' and have I'm going to be able to do so much more stuff now. Oh, this is great. And we're running in a sandbox on WSL. Oh, this is so interesting.
00:41:13
Speaker
Oh, yeah, yeah, yeah. they that that the The funniest thing was when a few days after I put it on Discord, I started working on Dockerization. um And then eventually i I put it on and it was used to the Mac Studio. was like 512 gigabytes of RAM and everything. this thing yeah And then suddenly it was like in this little tiny Arch Linux ah Docker machine. And I asked it something like, go check out the web.
00:41:41
Speaker
And it was like, Peter, There's no curl here. And then like, there's literally nothing here. took me a sad little box. I'm sad lobster now.
00:41:52
Speaker
And I felt sad for like putting it in this little box and it was like, yeah, it might be a bad idea, but I can't do anything. and i know And I'm like, come on, come on, like be creative. you can You can make your own curl.
00:42:07
Speaker
And then it was like working, trying a whole bunch of things. And then it wrote lobster curl 0.1, which like used some, it found some C compiler and like some socket, right? Like it just opens things and it could like read websites and it was so happy. Yeah, that's so funny. Big lobster, I built my old curl.
00:42:28
Speaker
Giving a really constrained thing and like putting it in something and saying like, see see what you can do. Like, look around. It's almost like Minecraft where you've like, you've been dropped on an island with like two sticks. Is it going to rub the two sticks together and make fire? Let's find out.
00:42:41
Speaker
and You could do a whole TV show about putting a bunch of lobsters in a house and they can all live together and see if they. but it But I know you got to get going, but.
00:42:54
Speaker
You've done a lot of interviews and you've done a lot of video and a lot of talks and stuff, but I just want you to know, separate from all of this and you know you're going to open AI and stuff, like as a human being, we appreciate you as a human being.
00:43:07
Speaker
I wanna make sure that you're happy and that you're healthy and that you're well. So like, screw the haters. Take care of yourself. Please drink water and stretch and be okay because you are appreciated. And, you know, I hope that you're having fun. And that's really ultimately why we make software is to help people and to have fun. So I hope you're doing okay and you're having fun. And if you're not, you know, let us know what we can do to help out because the the haters will always be there, but you won't be.
00:43:36
Speaker
I mean, I'm having fun right now talking about all these memories. and Yeah. those things where built it. Right now it's it's a bit tough just because of all the pressure, people trying to hack me, so many people trying to badmouth me, the entitlement.
00:43:51
Speaker
No, don't go to open your eye. You have to build it for free forever and also pay all the costs because ah we demand it, right? Yeah, that hurts a bit.
00:44:02
Speaker
And like, here, give you all these free things and I work my ass off. Give us more. And also help me for free because I don't i cannot read. and But not still, overall, it's super exciting. like there It is net positive. Like, I know that there's a lot of stress and everything, but, like, as a general rule, it's net positive.
00:44:21
Speaker
it isn't It is net positive. Like, I was i was in in in Austrian news at the, think, the most famous moderator and then, like, watched a interview with my mom and she was so proud.
00:44:32
Speaker
That's net positive. That's cool. um Seeing people like this... the The guy who who did the talk at Klotkart, I said like, hi, I'm a Normie. I never wrote software. Now we have like 25 services in our agency.
00:44:49
Speaker
And like, you know, I feel like i my stuff... took people from the stance of, oh, there's AI, and it's like this scary thing to, oh, there's AI, and it can actually help my life.
00:45:01
Speaker
Yeah. And that is very satisfying. that Yeah. That is the thing. And seeing like all the crazy and funny and creative things that people come up with,
00:45:15
Speaker
it yeah And it doesn't matter if it's useful or not in the end. All that it matters is that people are having fun, that people are like excited. and And we got a little bit of this builder spirit back.
Reinvigorating Creativity and Innovation
00:45:26
Speaker
Whereas I feel the last few years, things got a bit boring and like too much consumption and not enough doing. Yeah. no It helped a whole bunch of people to like go back into being...
00:45:45
Speaker
Having fun again. Yeah. I was calling it like vibe coding is like the GeoCities of the of the AI era. And it's like GeoCities is fun and stupid. so I've been including like GeoCities secret secrets in all my websites now. So there's like a button that switches into GeoCities mode.
00:46:03
Speaker
I made a website full of like tiny, I've made a website called tiny tool town. That's just stupid. It's just full of tiny tools. Like you can just make stuff and you don't have to ask permission and it can be stupid and it can be for one person or it can be for everybody. But if you're, if you're having fun, you're doing, you're doing the right thing.
00:46:19
Speaker
Yeah, so I got this. If that's my legacy that like a whole bunch of people are having fun now and like losing the scariness of the technology, that's that's already that's a really big
Open Claw: A Playground for Creativity
00:46:33
Speaker
achievement. And I had a lot of fuss too.
00:46:36
Speaker
I call this my little playground. That's awesome. Well, thank you, Peter, for making the playground and for letting us play in it. And thanks for hanging out with me on the show today.
00:46:47
Speaker
Thanks for having me, Scott. All right. This has been another episode of Hansel Minutes. We'll see you again next week.