Transcript
riptide: Welcome to Bounty Hunters. We're back with another episode. We are sponsored by ImmuneFi, the biggest bug bounty platform in Web3. If you're a white hat, this is where you earn real money and make crypto safer.
riptide: Don't know where you want to start? Hit the feeling lucky button and let the hunt choose you. We're also sponsored by Rare Skills, rareskills.io forward slash Riptide.
riptide: get 10% off a bootcamp. Go learn Uniswap V3, go learn ZK Math, learn it all, go find bugs and make millions.
riptide: And we are also sponsored by GitRecon.xyz forward slash Riptide. If you so launch a fuzzing engagement with ah the chads at GitRecon, you will get five grand off with code Riptide. So check it out.
riptide: Introduce our guest today is Jocelyn Feist, the creator of Slither and X-Trail of Bits. He was there eight years. Good day, sir. How are you?
Josselin: Hey, good morning. Thanks for having me. Looking forward for the discussion.
riptide: ah Very, very excited to have you on. You, I would say kind of laid low in the Twitter sphere until recently. And I'm i'm just saying this because i yeah I took more notice of you recently. I was like, oh, yeah, who's this guy? Because you were behind, you know, the the shield of trail of bits for so long. And now you you're kind of you're out there on your own.
Josselin: Yeah, I think that's correct. I was was never you know too active on Twitter in the past year. I'm trying to be a bit more active, but yeah, I'm still, I think, on on the low side.
riptide: And um so what happened? why Why did you decide to to leave?
Josselin: Yeah, I think that's a good question. And, you know, a few people have asked me that. I've been at Trail of Beats for eight years and, you know, you know in the streets, like two lifetimes.
riptide: yeah
Josselin: So I just thought at some point I needed to do something different. I'm still in a really good relationship with you know everyone at Trail of Beats. I still talk to a lot of engineers there, but I just think after um eight years in the same place, I just needed a change.
riptide: Yeah, absolutely. And now, and you kind of, so now you're doing this private security reviews I saw on your, on your page. And then you also joined Perimeter, as you said, the the lead fuzzer over there, which is kind of cool.
Josselin: Yeah.
riptide: ah By the way, get recon.xyz fuzzing forward slash riptide. I just want to throw that in there. i don't know if you, if you've heard of Alex and, and recon and okay.
Josselin: ah Oh, yeah, of course. Of course.
riptide: Okay, cool.
Josselin: I've been discussing you know with Alex for a few years now, probably.
riptide: ah Okay, cool, cool. No, I think more the better in in the field of fuzzing and hybrid and symbolic execution. I think and think anything we can do to kind of find the bugs is is a good thing.
Josselin: Exactly.
riptide: um So on your end, I mean, you have, you know, I look at you and I'm like, man, I started with one of my first tools was with Slither. And you're, you know, I'm like in the crib and you're nursing me with the body. Like, here you go. Here's your here's your findings, young researcher.
riptide: And, you know, you know I thought that was so cool. I said, wow, this is... And think back then, I mean, when you first developed it and when that was kind of one of the few static analysis tools to look at, how cool it was to just be able to run through the code and find potential ah flaws. Like from from creation of it to where it is now to to kind of you know what we're doing with fuzzing, ah how do you think...
riptide: What's your view on like how this has evolved? Do you think what we have now is adequate? ah You know, I'm curious to get your viewpoint given so many tools out here now from when you first started.
Josselin: Yeah, I think that's a really, really good point. So to put things in perspective, you know when I started to to develop Slyther, was definitely really cool to see the outcome, but at the same time, you know I'm coming from like traditional cybersecurity and in the traditional like web to work, we already have like a lot of really, really good tools. We have a really good search analyzer. We have really good fuzzle. We have a lot of kind of like things you can use.
Josselin: i And when you know i started within the blockchain space, it was basically nothing, right? So on the one side, it was cool to see kind of like the impact of the tool, but on the other side, was like,
Josselin: Yeah, there's still a lot to be done just to get to a place where we are even close to what we can have in in traditional security. um So it does definitely evolve you know over all the past you know five, six, seven years.
Josselin: um We are in a better place now in terms of tooling, but it's still it's still far from what we have in traditional software security. um like Like the further are good, like Ekinah, Medusa, Foundry, they are doing really well.
riptide: Mm-hmm.
Josselin: But if you compare to what you can have in In traditional cybersecurity, the maturity of the tooling is just not there. um Even Slitter, I mean, I'm really happy where Slitter is, but in comparison to what we could have with more, you know, a couple of years, more time and more effort, I think there's still like a lot of opportunities there.
riptide: Yeah, I geeked out on your on your web page and like before you were doing the Web 2 bugs and and I looked at some of your submissions and the feedback is just like, yeah, fixed.
riptide: Thanks. You know, from. And, and ah you know, the Web3 space is like bounty, you know, multimillion dollar bounties hang is just fixed. Thanks. And that was that was really the expectation with a lot of I mean, many projects you just contribute. Hey, I found a bug. Yeah, OK.
riptide: But you didn't have millions and billions on the line.
Josselin: I mean, if you think about that, because I think what you you're referring is some of the bug I've reported in non-blockchain kind of project.
riptide: hmm.
Josselin: If you think about that like 20 years ago, when you were finding something somewhere you basically could report it, but if you were reporting it, you would have issue with law and you could get into trouble just because you were like genuinely reporting a bug to a protocol.
Josselin: um There were many cases where people were like, oh, okay, you know I found a way to you know get access to your bank account and do A, B, C.
riptide: Right.
Josselin: And the reaction 20 years ago would have been, oh, okay, this guy is a bad actor. We are going to send law enforcement against that person. ah So over, I think, like the past 15 years, it did improve the bug bounty space within Web2 has evolved and now we're in a better place.
Josselin: ah But yeah, for for many years, like but reporting, you would get and t-shirt maybe you know like a thanks you will be know happy and you know 15 years later when you do that in web3 you get like ah but a really good bounty so it's definitely it does improve
riptide: Yeah. Yeah. Cause guys, I think naturally, you know, with a good moral compass, they want to report, they want to help.
Josselin: yeah
riptide: I think the, my viewpoint on, on human society is that we like to contribute and we like to help each other. And we, we have a positive outlook and then you have the small subset, which is nefarious and they're going to do bad.
riptide: But by default, i yeah I think what you're saying is this wasn't rewarded. This wasn't recognized. There was just hackers. It wasn't black hat, white. It was, they're all bad. You're doing anything.
Josselin: Yeah.
riptide: What are you probing? This is the crime of curiosity, man. No, no, no Put you jail. Yeah, it's it's flipped significantly, I think, for the better. And I think these bounty platforms for Web3 have been such a ah great driver of that as well.
Josselin: Oh, yeah, definitely.
riptide: and um you know you have you have a ah crazy body of work here i i particularly liked your your post i didn't know it was from you until i browsed your page because i'd read it before the breaking of upgrade ability where you have the delegate call self-destruct pattern that kind of no one saw except you and you have a quote in the beginning you're just like even though, you know, we weren't engaged on the job. We just, I just, just opened up the contracts and just browsed and just yeah quickly took a look and found this devastating bug and, uh, for audit firms, I think didn't catch it is what you said. And, um,
riptide: Isn't that the case? Like we, and as this, you know, this moves on to kind of what I'm trying to get at here, but we have this human review problem where it still exists today from, this was in 2020, wrote this, it's 2025 now.
riptide: And it's the same thing. Now we have LLMs. We have, I think, better fuzzing tools. We have, um, you better, well, now we're getting concolic execution and symbolic execution, better tools there.
riptide: And I think LLMs are improving dramatically and we're working on something with that too. But you still have this human problem where you miss things. You know, what is your...
riptide: opinion on this. and And I also want to, not to give you too many things, but you posted something by Gustavo on the new Echidna capabilities around Cacolic execution. Could you talk a bit on that kind of where we're at now compared to five years ago?
Josselin: So I might take the first question and then you know we can move to like concordance execution.
riptide: Sure, sure.
Josselin: ah So I think now related to like the human problem you're mentioning, I think a lot of people and a lot of teams see security as something external to them.
Josselin: Like they are going to hire like a security provider, they are going to do like a contest, bug bounty, and then kind of externalize security. And I think that's kind of like the root of the problem. the will In my opinion, like the real long-term solution is for security to be embedded in the development lifecycle, to have maybe like external review along the path and maybe like know through through multiple cycles of the development.
riptide: Thank you.
Josselin: But it's really about enabling the developer to build safer code. because that's really what all of this is about there will always be you know an expert there will always be kind of like you know a new risk and the best the best weapon against that is really to enable developer to avoid making the mistake in the first place so so it's really about you know having a good design having a good testing strategy having kind of like a good process on how you develop the software rather than always kind of like thinking and and seeing security as a last step before deployment because that
Josselin: doesn't work as as we know.
riptide: Oh, absolutely. And I wonder if we're going backwards with the vibe coding LLMs. Like how can you internalize security? You're internalizing it to Claude, you know?
riptide: And I don't think the devs, I think what you're saying is right.
Josselin: Yeah.
riptide: Like the devs that prioritize that, you see that. And these are the top tier guys that build ground up with security in mind. And the other guys just say, well, let's kick it over to the trail bits and they'll they'll figure out everything.
Josselin: Yeah.
riptide: Yeah, well,
Josselin: Yeah. For, for, for LLM it's definitely going to be tricky because everyone is going to start using you know llm to write code but the quality of the code which is kind of delivered is not there as of today and i i actually don't recommend anyone to write production solidity code with llm it's really good if you want to write you know test if you want to do like you know this type of things but to write like the business logic through llm
riptide: Mm-hmm.
Josselin: most likely you're going to have a bug and it's going to be subtle and you're not going to notice it until it's too late. ah So anyone who is deploying a smart contract which is going to handle funds should not have a single line of code written by the LAMM in my opinion.
riptide: I would agree. And I would say principle, like I use it to write POCs all the time and you still have to double check those because they'll just make a crazy assertion and, and, or a crazy prank call.
riptide: But how do you, how can you write code, the production code with LLMs and put it out there? Even if you, you get the test running, whatever, and your contracts are out there, you don't understand it You don't know what it wrote.
Josselin: Yeah.
riptide: like Like just what you said, you don't know the design choices and trade-offs that it made.
Josselin: Exactly. And there's a lot of sub substantial, you know, like a lot of small and minor things within, within your cut base might have an impact, you know, down the path. So you need to control and understand every line of code that you're pushing out there. And if you didn't get that to LLM,
Josselin: it's it's not going to happen. And you know people are lazy. If you receive some code which is written by LLM, you're going to check it quickly, but you are not going to check it like you know in depth and really try to understand what is written and what's our so assumption.
Josselin: So I think as a you if i were if I was like a team lead in a DeFi protocol or this type of things, I would have as a strict guideline, no one can deploy code which was written by LLM.
Josselin: use LLM to write documentation, use LLM to write test, but do not use LLM to write production code for smart contract.
riptide: Absolutely agree. do you think How many projects you think are doing this, though?
Josselin: It's hard to tell.
riptide: That's the problem. It is hard to tell.
Josselin: I think, yeah, I think the ah team that are mature, they know about that, I think, and they probably don't use LLM. The team that are a bit more, you know, younger or they have like a a short time to market, you know, type of strategy, they're probably not going to do that and they're probably going to rely on LLM.
Josselin: So, yeah, bad things are going to happen for this team.
riptide: Yeah, like I was curious, what do you see is like, you know, when I was reading that, what you posted about congolic execution, which is they call it hybrid fuzzing or when you do symbolic execution and fuzzing.
riptide: do you Do you see this like maybe plus LLM reviews as maybe ah ah kind of a holy grail type thing to stopping most bugs?
Josselin: it is going to help it is not going to be bulletproof proof so one of the things i've also tried to to you know to push to to my client and to the industry is this notion of invariant driven development where really when you write smart contract what you want to know you know from even before writing a single line of code is the invariant. So invariant are things about the protocol that should remain true.
Josselin: A good example is in Uniswap where you have the K invariant and you know when you're doing the swap at the end, K should not change unless you have like some some fear counting there.
Josselin: ah But from the moment you know kind of like the invariant of the protocol, you can design the code base around them, you can design your testing strategy around them, and then you can use additional tools like fuzzing, symbolic execution, formal method to verify invariants.
Josselin: So there, fuzzing, symbolic execution, concordial execution, they are going to be a piece in the puzzle to improve the security But I will even go further and say it's really about the invariant, you know, from the first place. so And it's really about knowing what your code base is supposed to do.
riptide: and And so what if certain invariants are missed by the protocol team? Like as an auditor, do you kind of look for situations like that? when Like in your role as a fuzzer and in your prior role, are you going in there and maybe looking at their test suite and seeing, hey, what did you miss?
riptide: And then compare that with what you know about the code to kind of come up with with different pathways?
Josselin: Yeah.
Josselin: Yeah, exactly. like When I'm doing like an invariant engagement, I'm going to do exactly what you say. um So part of invariant engagement is going to be to write like the fuzzing harness and like all the setup for for the code base, which can be complex for for some code base.
Josselin: And part of it is also to to discuss with the engineer to understand the invariant, to understand how we are to cover, to understand what they have done so far and how to to to help them in the long term.
Josselin: um So there's definitely like a lot of kind of like ah exploration of invariant. um So yeah.
riptide: And when you were a trail biz, what could you explain their audit model? Because the only one um that I've seen clear transparency about is Hexans because they put it on their audit reports and and where they have basically have two teams.
riptide: It looks like they have one guy, you know, one set of teams review and then they send it to the other team. So they do a double review of each report. And I think that that sounds in theory like a great model.
Josselin: Mm-hmm.
riptide: How did you guys do it over there?
Josselin: Yeah, it's much more collaborative. um So we, like, know, when I was at Trail of Beach, we were not setting street guidelines into, okay, this is how you do a review. We, you know trusted the engineer to have, like, the right process by themselves.
Josselin: But basically, we foster collaboration. um We foster, like, internal discussion, discussion with the client also, because at the end of the day, the client have way more understanding than you would ever have.
Josselin: um you know, like if you review a code base for four weeks and they have been working on it for two years, they definitely have a lot of insight that you should try to extract. So it was less about competition and internal competition to know who is going to find the most bug, but really more about fostering a collaboration among the engineer so that they can build up on each other's understanding.
riptide: but said Did you always have a review no matter what is a final review by another team?
Josselin: No, no, it was not like multiple team in kind of, uh in different like segregation it was really like everyone together
riptide: Okay, and and and i'm not I'm not at all trying to flex on the great Joslin at all, but this is a follow question I have because i think your technical expertise is far beyond mine. And I just, you know, I threw some luck into here, but can we talk about the Arbitrum Nitro review from 2022, which you led?
Josselin: um Yeah, it might. It was a long time ago.
riptide: nothing Nothing crazy about it, right?
Josselin: We can try.
riptide: I mean, it's 150 page review. you know You guys went all in. And you did that in March, and I found that bug in September in Arbitrum. And the the root cause of this bug was due to this post-upgrade init function where they wiped three slots clean, and then they reinitialized two, and they left the third blank.
riptide: And they told me, and they had multiple auditors review.
Josselin: okay
riptide: I don't know if it was just you. They told me, oh, this we put that in there because of the feedback from one of our security partners as an optimization. And I didn't see anything in the review about it I've never got an answer for it.
riptide: who Who suggested it? And I was just curious if that was if you knew who was behind it.
Josselin: I mean, to be honest, from a review three years ago, don't remember very much. Usually we don't tend to recommend optimization because optimization are risky.
riptide: Yeah. Mm-hmm.
Josselin: um Because exactly what what you you you you you will say.
riptide: yeah
Josselin: I would like for optimization recommendation, I basically have two mode. Either the optimization is clearly having no security impact. Like for example, if you read twice a state variable and there is no external code, there's nothing and you can, you know, be sure like you can just cache it in a local variable.
Josselin: Sure, you can do it. But for optimization that are more like low-level, like you know you can use assembly or you can you know skip some other things, I would be surprised if it was you know something we recommended.
Josselin: Let's say from a report from three years ago, it's difficult to remember.
riptide: put you on the spot not at all flex just just curious yeah
Josselin: Yeah, especially especially if it's not in the report, I would be surprised like if it was coming from us.
riptide: yeah and uh you know also so like you put it you put on your page that you're doing security reviews and you specifically didn't use the word audit why is that
Josselin: So I've been using security review foe many years. And I know there is some kind of like, you know, Twitter fight about audit versus security review.
Josselin: um Security review does make more sense, but I might choose also audit, you know, from time to time because that's what people are expecting.
riptide: Mm-hmm.
Josselin: i To be honest, I don't think audit versus security review debate is that important to me reflects more like a kind of like ah internal bubble within the Web3 security space where you know people might debate about things that don't matter that much.
Josselin: um Like if you go to a client and if you go like to the developer and and you know, like that community, they probably don't care if we call it an audit or security review, even if security review is kind of like more Truth in in like in a straight sense.
riptide: Okay. Yeah, I was just curious how you were categorizing it because I use that terminology as as something I do, is which is like ah people come to me and I say, hey, wait until all your auditors review. And then you could also call it like a targeted bounty hunt.
riptide: But I say security review because I'm not going to give you gas optimizations.
Josselin: Hmm.
riptide: I'm not giving you low mediums. Like I'm only giving you things that will break the protocol, just critical size, just stuff like So that's how I defined it.
Josselin: OK.
riptide: That's why was curious how you were defining it.
Josselin: Okay. so So you are doing yeah really more like time bound, back bounty type of things.
riptide: Type of thing because beef, like say you, you get all your reviews and then your bug bounty is a million bucks. Well, Hey, just let me work on it first. And then if I find a critical, look, you're saving a million bucks and you're, I'm charging way less than that to do a a targeted review.
Josselin: That makes sense.
riptide: So, yeah.
Josselin: So the way i see it is that a security review can take many forms and it depends on the scope. So we can end up having like, you know, 10 different names for the services and for the products, but I think just, you know, having one name makes it a bit easier.
Josselin: And what I mean by that is that when you do a security review, depending on the context and the scope, you might adapt. For example, if you review something which is still in development and you know some components are not fully fleshed, some components are still like moving and the developer will know that, okay, this is going to be you know changing,
Josselin: During the review, you're not going to focus on that and you're going to maybe focus on on something different. um if you If the developer is more interested in like design because it's halfway through the development, you might focus more on long-term recommendation.
Josselin: While if it's like two weeks to deploy like the protocol, you might shift the focus into exactly what you say and more like know high and and and critical. ah um But you are still doing, and my you know in my mind, security of you, it's really more like you adapt based on needs of the client in that sense.
riptide: Right, right. No, good point. And what was I going to ask you? Oh, yeah. ah Something I have to know, like, what is in what is in your tool suite, your toolkit?
riptide: Like how are and you post something recently, like how are you using LLMs as a military term as a force multiplier in your kind of day to day work?
Josselin: that's That's a okay really good question. i use it for everything now. um
riptide: ah You admitted it.
Josselin: but Yeah, I mean, for for for example, you know, like I use cursor when I explore a code base to kind of chat with cursor to get some insight and, you know, draw some graph and um things like that.
Josselin: ah It's a really good assistant for for some of the things. ah When I need to write a proof of concept, I'm also going to delegate part of it, you know, through an LLM. ah It's not perfect, but it's going to, you know, scaffold like the test and this type of thing. So just saving time.
Josselin: um When I'm building tooling, now I'm also using LLM and I'm, you know, delegating part of the implementation to the LLM. um And this has saved me also like a lot of time. Like, yeah you know, I used to i know spend a day to write like a custom static analyzer for a specific target.
Josselin: Now I can do it in like two hours with an LLM. So it's pretty good.
riptide: That's what I know. It's, it's, there's so many ways to save time. It's just, it's, it's so hard not to rely on it, but then you have to you have to don't get lazy, right?
Josselin: Yeah, I think that's a real important piece, like don't get lazy, in the sense that you should not delegate your reasoning to the LLM.
riptide: That's it.
Josselin: You should still drive like you know what's important. You should still drive like the reasoning. ah You cannot just rely on LLM, for example, to find bugs, but you can rely on the LLM to help you to get an understanding quicker and so on.
riptide: Yeah, if you can if you could put it over the target, it could be pretty good. But if, hey, just go find all the bugs, it doesn't work that great. But if you can put it around, you know, your 25 meter target there, it could be, and and especially with, what was I using, Grok 4, where it has full web access, full research.
Josselin: Yeah.
riptide: My God, search whole repos. My God, what a time saver. If you're not using that for your bug hunting, you're crazy. Even though it messes up sometimes, it's like, my God, what a time saver.
riptide: Highly recommend
Josselin: Yeah. and have you Have you tried DeepWiki?
riptide: What is it, Deep Wiki?
Josselin: Yeah, it's it's from, I think, Cognito. It's basically like a code exploration for public GitHub, where you can just build any public GitHub and they have indexes, everything they created, like a summary and graph, and you can just query any code base remotely.
Josselin: um So if you do bug bounty and you explore like, oh, okay, like this new project on GitHub, DeepRicky is really, really good to give you a first insight.
riptide: I'm gonna check that out, DeepWiki. You know, GitHub Search, I've used that a lot and I cannot believe you still cannot do a regex on GitHub Search. Like, I feel like it's artificially neutered.
riptide: you're you're You're probably laughing because you you built some secret backend API tool where you're regexing left and right.
Josselin: might
riptide: And you're laughing at my novice, my fledgling behavior.
Josselin: No, I mean, i mean i'm I'm nothing because I'm thinking about like the backend of GitHub. And if you allow rejects, that's a lot of computing power which is needed. Like running regular expression can take some some some processing power if you if you allow any type of regular expression.
Josselin: um So it might not be that easy to do that.
riptide: Yeah, it's there's gotta be some reason.
Josselin: Yeah.
riptide: And then, ah I don't want to bring it up because it's it's such an old school tool like the Ethereum ah verified contract search. Like when you really want to get lazy, by God, it's the worst search ever, but it's it's it'll do it a pitch.
riptide: um ah what ah oh yeah Oh yeah, are you familiar with Lightchaser?
Josselin: Um, yeah, I never use it, but I'm familiar, like of the name basically.
riptide: So i I did an episode with Chase the Light and he has built his tool where he has, what does he call them? Not agents, but detectors.
riptide: And he said he's amassed this list of, I think, over a thousand detectors.
Josselin: Mm hmm.
riptide: And um he he says all hand-coded, you know, coded by hand. And he he does it all himself. Maybe works with one other person. But he he expressed very high confidence in his tool and the improvements in his tool in version four on where he's been able to consistently find high severity ah vulnerabilities just using ah static analysis.
riptide: I was curious on your thoughts on that.
Josselin: Oh yeah, I mean, I totally believe it. I mean, I found a lot of bugs in Slitter, so i'm not surprised if, you know, people building LightShare, know, have similar outcome. There are a lot of like simple, not necessarily simple, but some form of pattern which are really good for static analysis.
Josselin: um And you can find like bug at scale. I think the real challenge is not to find bug. And it's the same with LLM, like it's not to find bug, but it's to find,
Josselin: bug with a good ratio over the false alarm and the false positive. Because finding bug is the easy part. Removing false positive is difficulty here.
riptide: Mmm, yes, yes.
Josselin: Like even with Slitter, like the summer of the detector, Marc-Andre Vintour- there are some design shows where we could say okay we are going to cover more ground and you know we bought more but. Marc-Andre Vintour- we are going to be okay to miss some of the bug because we are going to reduce the ratio between false positive and and and true positive um so there is like a a lot of design direction and and and decision that you can make.
riptide: I feel like someone is is out there building the ultimate tool where they're combining like a light chaser with you know the right AI agents with fuzzing where it's a kind of one click, hey, look at this code base.
riptide: And it's just cranking on AI credits and it combines everything at once, fuzzing, everything.
Josselin: Yeah.
riptide: and And I mean, it's probably already in development, but I feel like that's that's kind of the ultimate auto tool that we need in this space.
Josselin: Yeah, from from what I can tell you not talking to a bunch of people, everyone is going in that direction. I think today, most of the last team are trying to build that capacity. And a lot of individual or like, you know, small group or like two or three people are also trying to build something like that.
Josselin: I don't know if you if you follow. um But there was like the AI XCC result a couple of months ago, which was a kind of a competition to find bug automatically using traditional program analysis and LLM in in like C and Java, funded by the DARPA.
Josselin: And it's exactly what you described before, like traditional analysis and traditional software.
riptide: Mm-hmm.
Josselin: So definitely everyone is going in that direction.
riptide: and And my thoughts on that are are, that's great, but I also know the blockchain and it's it's a place, when you when you were in DeFi summer, where protocols developed by, protocols, right?
riptide: Developed by 16 year old in India can amass, you know, 50 million.
Josselin: Thank you.
riptide: And this guy just picked up his Solidity course online and anyone could deploy things and attract capital. And so we can have all the tools necessary, but in in my eyes, we have the human element that is always vulnerable, not just in this case, but as far as just pushing out code and and permissionlessly.
riptide: So, you know what do we what do we do, Joseph? What do we do here?
Josselin: Yeah, that's what I was saying at the beginning, like the solution is not on the security researcher side. The solution is really on helping the developer to get better because that's how, like, you know like if everyone that start to learn Solidity or start to, you know, do all of that, get some flavor of, okay, you have to think about security, you have to, you know, have like all this mindset.
riptide: Hmm.
Josselin: um And if we educate kind of like, you know, more developers toward that direction, That's how we mature as an industry. That's how we improve in the long term.
riptide: Mm-hmm. Yeah. In some ways, I think our space is getting better and then a lot of lot of ways is getting worse as far as... like
Josselin: It's much better than like five years ago, let's say.
riptide: Yeah, I mean, there's there's definitely more awareness. Like a lot of people at least know how to read a smart contract and they know what's going on.
Josselin: yeah
riptide: But I mean, look look now, the amount of drainers and scammers and front end, like any any vector, when you have this much money involved, any vector is being exploited.
riptide: Now compared, way more now, I would say than compared to back then.
Josselin: Yeah, like I agree, like I think ah a lot of attacker moved away from smart contract and, you know, started to find any entry points to attack and steal money which are not smart contract related.
Josselin: So that's definitely like something we have seen. And for like end user, you know, like obviously like all the scammer or like the phishing and everything, they are making a killing for themselves.
Josselin: um
riptide: And salinity extensions or VS Code extensions.
Josselin: So.
riptide: like you know it's like It's like we're walking through a minefield every day.
Josselin: Yeah.
Josselin: Yeah.
riptide: And I remember you know past audit reports that you always look through and you guys would do it, everyone would do it, but it was always like, hey, um admin centralization, risk.
riptide: And everyone would write it off. Oh, it's not really a finding. And then we're seeing like, hey, really the wrench attack, the digital wrench attack is working. Offer them a job and then compromise their system.
riptide: Yeah, it's...
Josselin: Yeah. That's why I know like it's also important in the long term that like large team do like thread modeling and this type of activity for them to understand what are the risk, what are like the boundary in the protocol, um because there is so many ways to, you know, steal money from a protocol that are not related to issue with a smart contract.
riptide: Did you ever get into that at all? Like, you know, the physical side of it or exploring those kinds of avenues? Because in in our space, it's laughable. You'll see whole teams, the the whole multi-sig shows up at a conference. You know, it's so laughable. it's just, it's crazy.
Josselin: Yeah, so I've provided advice, but I'd never performed like a physical pen test or this type of things.
riptide: Yeah.
Josselin: It's less it's less of ah of a thing in Web3 just because everyone is remote. So like, you know, if you go if you go like in more traditional company, they have an office, they have like, you know, server, they have like all all of that, like like database somewhere.
riptide: Mm-hmm.
Josselin: um But it's a different set of that. Most of the blockchain team are remote and don't have even like or real office.
riptide: Yeah, yeah, I think the the only, I don't know, there's just an exploit all the time, but we have the hardware wallets, which was such an improvement off just writing down your keys.
riptide: i I think that that's really our are like iron gate, holding a lot of these attacks at bay, like these trainers, like thankfully you can do something that I wouldn't say 100% guarantees your funds not to be lost, but it's ah it's a very high wall to climb.
Josselin: Yeah. Yeah. You need like, you know, air gap machine or so, and you need a lot of different things because even if you have like a hardware wallet, like even when you need to sign a transaction, it's really difficult to understand what's going on with the transaction.
riptide: Mm-hmm.
Josselin: um So that's why you probably want to to have like an air gap machine machine that you just use to sign transaction if you manage any significant amount of money, right? um So there are a couple of like strategies that you can you can apply.
Josselin: um It depends on how much funds we are talking about, how many you know people are within the process, what's kind of like the context of your of your company.
Josselin: um But there are definitely like things to improve.
riptide: Yeah, and i don't I don't know your age, your maybe you're my age, something around there, but it I don't know if it's a ah thing with the younger groups because most of my listeners are 18 to 24.
riptide: And I've worked with some of them and I've seen how they work. And they're very, the the opposites of of what I would do, right? Like I'm very, um I don't trust the cloud.
riptide: I don't trust my file names have underscores in them, you know, like things like that.
Josselin: for
riptide: But they're they're very happy to put things in Google Docs. google Like everything's in the cloud. They're cool with it. They're running 10 different browser plugins that they're somehow cool with that.
riptide: And it's just a lot of things where I grew up kind of paranoid on the computer, where if the file's not on my computer, then i don't trust And like so we have all these things and all these kind of guideposts that are already out there. But do you feel like the younger generation, they just, you know, I guess a different mindset, they don't care as much or or don't know about it as much?
Josselin: I'm not sure it's a generational thing. It might be, but I'm not sure. And the reason for that is that a lot of people that are not security oriented, you know, above 30, 40, 50, they do the same.
Josselin: So I think it's it's more about like having kind of like a security mindset and having like a kind of an education around security um that change perspective on things like being a bit more paranoia, being a bit more, okay,
Josselin: how to trust something, what you can trust. And I think it cost comes more with kind of like your exposure to security.
riptide: Yeah. Yeah, I guess I would agree. True. yeah Yeah. Because i could I could talk to people that that are my age that would do some really dumb shit. That's true.
Josselin: Yeah, I think for a lot, one thing I've noticed for a lot of like, you know, younger people in the new web3 space, several or large majority came to security through Solidity.
riptide: I think you...
Josselin: They came through security, you know, through smart contract security. They never did any need like web security or traditional application security or this type of things. but They might not have like the traditional security background that you might have from, from people from other community.
Josselin: And because of that, they might not have this education about security because all they know, all they have experiment is around smart contract security, which is kind of like a a subset of what you can do.
Josselin: So I think like, and I've seen that, you know, with a lot of people where you have to kind of like help them to understand, okay, there's like all these things that can happen, there's like all these possible things, but it comes from exposure to like security risk.
riptide: that's That's a good point. Yeah, they they view security just through the Web3, yeah, the focus.
Josselin: Yeah.
riptide: They're not thinking Tempest attack on my ah network cables there. yeah And the Web2 thing is just like, you know, the the crowd now has grown up with the browser as the brains of the computer too, which is,
Josselin: Yeah.
riptide: which is totally different than what I'm used to. And I think that's affected a lot too, because the browser is now to me, I mean, it's critical infrastructure on my computer. So those, those extensions I've scrutinized highly and they're, they're only used absolutely when necessary.
riptide: And it just opens up so many things that I don't think people consider.
Josselin: Yeah. Or even, you know, having like a, like if you work, if you're an individual having ah laptop for work and a laptop for other things um is a practice that probably most people don't even follow if they don't work at a company.
riptide: Yeah.
Josselin: um So they end up, you know, mixing personal action with, you know, whatever using for work, which definitely increase the risk.
riptide: Yeah, definitely not good.
Josselin: I think it comes also from, in a way, I've noticed a lot of people, you know, started a journey through bug contest and you know going through like like individual and competition.
Josselin: When you do that, you might be missing a bit of the mentoring that you get at a company in terms of like traditional security, where you need to have something on your laptop that, you know, add like additional guarantee. You need like to have like some specific access control.
Josselin: You have a lot of kind of like hygiene that you get when you work in a security company that if you don't have again exposure to it, it's difficult to get by yourself.
riptide: ah Very good point. Yeah, you do kind of absorb things that are just around you at working for a company.
Josselin: Exactly.
riptide: That's true. No, good point. um I want to get to some questions, if that's all right with you from the discord. And you can, you you could choose the answer or decline, whatever.
Josselin: Go ahead.
riptide: I don't think they're too, they're too tough. This one's kind of interesting. So K42 says, if you were making slither from scratch today with all the wisdom and knowledge of today, but you only had two weeks to do it, what would you focus on for optimal results?
Josselin: Oh, two weeks.
riptide: Vibe code.
Josselin: Yeah, if it's if it's a two-week thing, I would focus on a few types of vulnerabilities that I want to discover and focus all the design, all the analysis, toward finding one or two specific vulnerabilities.
Josselin: Yeah, for two weeks.
riptide: What would those be?
Josselin: I mean, don't know, let's say you want to have the best Marc-Andr Goulettenau- only we enter on see detector in the world right and you have two weeks to do that, then you can focus on okay how to detect the pattern, and how to.
Josselin: Marc-Andr Goulettenau- Make sure you remove false positive what type of analysis, do you need to go through that um so it will be really about like what's the end result and how to work backward or that.
riptide: I love that class. Read only reentrancy. Do you remember who actually found that? I can't remember who did the write up.
Josselin: So.
Josselin: So who coined the term? I think it was chain security or it was open Zeppelin.
riptide: I think you're right. I i think it was chain security.
Josselin: Yeah, but people were finding this type of bug before, right?
riptide: um I'm going to say that.
Josselin: But they did coined the term and created like more awareness toward it.
riptide: You're saying curve. That wasn't the first one that it was founded.
Josselin: No, no, I'm saying people were finding like, you know, read only reentrancy before without calling that read only reentrancy.
riptide: Yeah.
riptide: Okay. Okay. No, that, yeah, that's a great one. I love that. That's, I think that's the the dream of a bug hunter is to find his own class of bug.
Josselin: yeah Okay.
riptide: Ideally he should get it named after himself, like the riptide recently. That's like the, the wall of nerd fame put me up on there. Okay. The next one is from a,
riptide: your hardcore competitor and and colleague, Alex. So he says he says, what is your setup for fuzzing? when When do you fuzz and when do you just do manual review?
Josselin: That's a good question. It depends on the context because if I am on a fuzzing engagement, then I'm going to do end-to-end fuzzing. I'm going to create like a complex harness. I'm going to discuss with the client to find the invariant and, you know, all of that. So that's like kind of like an AV process.
Josselin: Now when I do traditional in a security review and my goal is not to focus on fuzzing, the way I use fuzzing is as a targeted approach. ah For example, if I know there is some weird arithmetic going on and I notice, okay, like, you know I have a feeling that this rounding is not right. I have the feeling like, know, like this multiplication, this division, if something is wrong, i am going to use fuzzing in that direction and I'm going to you know, mark the contract, I'm going to create kind of a model of arithmetic in a separate out contract, and I will do targeted fuzzing ah based on the model of the contract.
Josselin: And I do that so that I can abstract all the harness, I can abstract all the kind of like configuration, and really focus on the part of the code that I want to fuzz. um So doing a manual review, that's really how I do it usually.
riptide: What would you say would be like some of those red flags where you're like, yes, im this is what I'm going to, going to fuzz.
Josselin: Like, arithmetics is really where I, for the most, like, you know, when I see no explicit winding direction, you know, over the code base, or there is like always like, you know, some, some um it's always winding up or something like that.
riptide: Hmm.
Josselin: Like, you know, when I see complex arithmetic, it's usually a place where I'm going to use a further. and And the reason for that is that even if I find the bug manually, I need like a confirmation because like arithmetic can be tricky.
riptide: Mm-hmm. And what about, do you do the do you do hybrid fuzzing as well?
Josselin: um By a hybrid, fuing you mean things like symbolic execution and combination?
riptide: Like the... Correct, yes.
Josselin: um Not directly. I do use from time to time almost when I need like some some level of confidence in in in what I'm doing. So I'm going to use Echina, Medusa.
Josselin: as as a first step to first. And then for some things where the further doesn't give me like information I want, I might switch to symbolic execution. However, that doesn't work all the time. And ah it depends on the complexity of the operation because if it's too complex, um like the solver is not going to be able to handle it.
riptide: Okay. And that that was his other question too. Do you use formal verification tools? Which ones and when? Mm-hmm.
Josselin: Yeah, so I use almost from time to time, not on a kind of like a you know continuous basis, but when there is a specific need, I'm going to use it.
riptide: Okay. And like when you do some of your fuzzing, is there, do you have like a specific, how do you come up with the amount of runs that you're going to do?
Josselin: Thank you.
riptide: and And do you have dedicated hardware for that? Like how, because I've i've seen, I've seen, I think it was Enigma Dark or what of these people, someone posted something where they had, they had things running for a couple of days on some fuzz tests.
Josselin: Yeah, so if it's for an environment development kind of engagement and I need to first for like multiple weeks, I might you know deploy something in the cloud, for example. But if I do it during manual review, that's more something I'm going to run for like you know over the night on my on my my ah personal machine.
Josselin: um So it depends on what I'm trying to do. Usually for targeted fuzzing, it's going to run for like a couple of hours at max because um um'm I'm not targeting like a large piece of code.
riptide: Okay.
Josselin: So it's going to find, or it's not going to find. For more complex engagement, it's going to depend.
riptide: Okay. and And what is your personal machine? Just out of curiosity, what are you running?
Josselin: um It's a desktop machine. So it's like a, I don't know, Ubuntu.
riptide: what What OS? What OS are you running? Ubuntu.
Josselin: Yeah, I have also Mac, so it depends what I'm doing.
riptide: You know, I got rugged on the Mac. I'll tell you what, like I've got just, I was running Ubuntu for many years and I was, you you know, you have your normal frustrations when it upgrades your, your network driver and you, you got to roll it back, you know, this and that, right.
Josselin: Oh, what happened?
riptide: And I, I eventually saw this stupid marketing, which they got me on, which was for this, the Mac M3 and I got an M3 max. And it was like, oh, it's so fast, this and that. And I saw, you know I've used Macs in the past. It is sleek.
riptide: I buy one. And it's just so disappointing. It's a Mac is great. Like the interface is great. And then it just does things so badly. Like the security updates all the time. There's always these holes in there.
riptide: And then it's just it just doesn't perform that the way you'd expect such high-end hardware to perform. So I think I was rugged big time. I think if I had this hardware and I was running Ubuntu on it, it'd be way better.
riptide: and then
Josselin: I mean, I do have a Mac too, so it depends on the need I might switch, but yeah.
riptide: the
riptide: You know what I mean. Yeah.
Josselin: and
riptide: Yeah. It's just disappointing. I think Steve Jobs is rolling in his grave. Anyway. Okay. So Alex has some other ones. He says, which improvements are still missing in security tooling from your perspective?
Josselin: um
Josselin: Okay, that's a recurring joke, but we are still missing a debugger. I mean, I remember on back in like 2017, 18, like people who were like, oh, we are finally going to have a good debugger.
Josselin: And you know, it's many years later and we don't have a good debugger.
riptide: I have one for you. Have you heard a Rumblefish?
Josselin: Oh, no.
riptide: I'm going to send this over. I met this guy at an ETH event and he wrote this on-chain debugger and he wants feedback.
Josselin: Okay.
riptide: I'll send it to you offline, man. But the thing is crazy. It's really cool.
riptide: So maybe we're part way there.
Josselin: Oh yeah, that would be perfect. Like if you have a good one.
riptide: yeah Yeah, sure. ah Okay, last question. He says also, in which security changes would you want to see to smart contracts and processes that projects go through?
Josselin: And processes, I think it's really,
riptide: In the context of of improvements, missing and security tooling.
Josselin: Oh, for security tooling. um I think it goes back to like the invariant-driven developments, because once you know you're invariant, you can use different tools and different techniques to verify them.
Josselin: um But it's it's less about the tool, it's really more about the processes.
riptide: Mm-hmm.
Josselin: ah So the things I wish you know to see more across developer, is thinking about invariants. And in fairness, have seen an increase of that over the past two years. I see more developers coming with, so you know when you start review of like, okay, this is like 10 invariants we know about and this type of thing.
Josselin: So it is improving and it is going in that direction, but there is still a lot of room for improvement.
riptide: All right, cool. um Joseph, the final ah final boss question. And i didn't prepare yeah i don't prepare anyone for it. yeah i So I do an alpha drop where I drop some sort of alpha for the bug hunters that listen out there.
riptide: And it could be code specific thing. could be a bug pattern. It could be general life advice. It could be anything you like. um So I'd ask you for an alpha drop. I'll give you a minute to think about it while I drop some alpha. So this is on, um and it'll be kind of vague. i'm going to keep it vague because I like to.
riptide: So this is on 4246 vaults, ERC 4246, and it's about vault wrappers. So just check check before the vault wrappers kind of introduce a vulnerability depending on how they're coded up.
riptide: um that could just check the shares and check the share checks on those wrappers ah before things make their way and the logic flows to the vault because you may run into some interesting findings there because some certain things aren't considered because it's kind of out of the spec when someone writes a wrapper that talks to one of these vaults.
riptide: um Mr. Joslin, do you have any alpha for the listeners?
Josselin: um Okay, one thing I would recommend like people doing bug booted to look for are the hook architecture from like Uniswap, Balancer and all the protocol that to follow it because I'm kind of expecting that people building hooks on top of like major protocol might not necessarily have like you know a mature security process and probably like the integration some of the access control some of the data a validation they do through the hooks is probably not that great so if you want to you know hands for that ah there's probably some chance to find something
riptide: Very good. I like it. Well, cool. ah Hey, we're we're at about an hour. Thank you for coming on. It's been a ah real pleasure and an honor ah to get your expertise on the podcast.
riptide: And everyone, I think, will enjoy this episode. So thank you, Joseph, for coming on. And everyone, we will see you next time on the blockchain.
