Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Episode 22 - mackenzie
0 Plays2 seconds ago
riptide & mackenzie discuss the inner workings of immunefi, what happens behind the scenes as soon as you click submit on that juicy bug report, mackenzie's unique omniscient view of bug reports and bug hunters, how to up your negotiation game to get paid, and much, much, more ...
Recommended![Episode 19 - 0xe4669da [SPECIAL n00b EDITION] image](https://media.zencastr.com/cdn-cgi/image/width=112,quality=85/image-files/677fcded3f04b4e1c3f96cfc/1b4f61b7-5c8c-48b2-9d61-66ec9549c632.png)
Transcript
Host's Return and Episode Gap
00:00:06
riptide
Oh God, the intro. I'm so fucking pumped now. All right. We are back, bounty hunters. A little little longer than ah then my normal week in between episodes, but hey, it is what it is for your free show.
Sponsors: Rare Skills and GetRecon
00:00:21
riptide
So anyway, we're going to pump out our favorite sponsors. Get some rare skills for you bounty hunters out there. If you're not finding bugs, maybe you don't have the skills.
00:00:32
riptide
Go to rareskills.io forward slash Riptide. and get a little discount when you book in one of the packages, the boot camps, so you can get learned up on all your skills.
00:00:45
riptide
ah Honestly, it's a great, great structure. They have a lot of free resources on there as well, but if you like the structured product, they got that for you. So rareskills.io forward slash Riptide.
00:00:56
riptide
And then of course, we got a shout out getrecon.xyz forward slash Riptide. These are for you projects out there that need to get your, invariant tests done, go to the experts, get a discount, mention Riptide, get recon.xyz forward slash Riptide, get
Guest Introduction: Mackenzie from Bug Bounty Platform
00:01:15
riptide
some of that. So our guest today is not a bounty hunter, but he is on the other side of one of the biggest bug bounty platforms out there.
00:01:24
riptide
Our guest today is Mackenzie. Welcome, sir.
00:01:29
Mackenzie
Good to be here. i I love that you have Rare Skills and Recon as your sponsors, as those are my favorite security projects out there right now.
00:01:35
riptide
Hell yeah.
00:01:39
Mackenzie
It's just Jeffrey's amazing at Rare Skills. Frankly, I don't know how he made it work after all these years. Nothing like it.
00:01:47
riptide
Yeah, he's, he's a great guy, man. and And they just, it was cool. I ran into him at the eth event and um just, you know, like you come on to these free resources and And you don't even know who's behind him. And obviously he's monetizing it with the boot camps everything like that, which I think is a good product, but it's so cool, man. The free stuff that's out there that he put out there. So much love to him and and I wish him a lot of success in his business.
00:02:16
Mackenzie
Yeah, the the level of quality is insane.
Challenges in Writing Bug Reports
00:02:18
Mackenzie
I've often wanted to add Immunify to incentivize a similar sort of really high quality posting because we'll do our you know bug fix reviews when someone finds ah epic bug and work with the SR to break down how do they find it, how did it work, um all those things.
00:02:39
Mackenzie
And the hard part is writing it, like writing it in a clear way that's understandable in like a logical sequence is a ton of work.
00:02:49
Mackenzie
And we would do more if it were not so much work on our side, just for a lot of the writing and editing. I'm like, I'd love to have some way of crowdsourcing people to do their best work, to support each other, to kind of give tips and rewrites, you know, take some scrappy or written bug fix review and turn it into something really polished.
00:03:12
Mackenzie
Even if you weren't the one who found the bug, that is super valuable. um
Bounties for Quality Bug Reports?
00:03:18
riptide
You know, that that could be something for, um you know, these guys that are in the Immunify Discord, up-and-coming guys, Maybe you could put a bounty out on that work and say, hey, you know here's here's what happened.
00:03:18
Mackenzie
Haven't got around, unfortunately.
00:03:31
riptide
Give us your best write-up. And if we use it, we'll pay you hundred bucks or whatever it's worth. you know And it gives these guys the time and the the kind of purpose to dive in and really, you understand a lot when you walk through a bug. So if they do a write-up, it means you really understand what the hell happened. So that could be a good learning opportunity to kind of pay it forward for the the new guys.
00:03:54
Mackenzie
That is a good point. And a lot of writing is you just got it or you don't. Not everyone's a but great writer. I'm like an average writer. And that's after lots of practice.
00:04:07
Mackenzie
And so I'm sure there's some guys who are new to security, but just they got they got away with words. I like that idea. And it reminds me of...
00:04:15
riptide
One of my favorites is ah Wrecked. You ever read them?
00:04:19
Mackenzie
Oh, yeah. Oh, yeah.
00:04:21
riptide
I love the noir style that they that they do the write-ups in.
00:04:22
Mackenzie
Oh, yeah.
00:04:25
riptide
It's great.
00:04:27
Mackenzie
Nothing like injecting some drama into it. Yeah. Which is, if anything, stereotypical of the personality of security researchers in general.
00:04:41
Mackenzie
they're They're very non-dramatic. In fact, when they be more dramatic and it would and makes sense, they're still too non-dramatic.
00:04:44
riptide
Mm-hmm.
Negotiating Bug Bounties
00:04:50
Mackenzie
Like my my best example is maybe the friendliest, Most serious, but also like incredibly considerate and gentle whiteout out there, and there's many to pick from, so this is a high standard, ah is Lonely Sloth.
00:05:06
Mackenzie
And there's so many times, you know, i've I've been in his bug reports and he's talking with the project and, you know, he he stands his ground, he's firm, but he's also so considerate.
00:05:18
Mackenzie
And, you know, he's leaving money on the table in order to facilitate a good interaction, a good relationship.
00:05:18
riptide
Mm-hmm.
00:05:24
Mackenzie
And, you know, it's it's paid off, no doubt, with private deals. But, know, I'm like, man, you couldt you could have put your foot down. And so many SRs will say things like, the project you know said they're not going to pay or you know they they did disputed Immunify's mediation.
00:05:43
Mackenzie
um Am I allowed to say something about it? And it's so funny to me because if Immunify says, hey, this is a critical, you got to pay 100K and the project disagrees, they're going to be like, hey, no, that's not right.
00:05:55
Mackenzie
We're not going to do that. you know We'll talk to them, get the reasons why, work through it. But whiteheads, on the other hand, will be like, hey, you know, yup this is invalid. These are the reasons why. And the whitehead will be like, you actually forgot this thing. You made a mistake.
00:06:10
Mackenzie
and And we're going make mistakes.
00:06:10
riptide
Mm-hmm.
00:06:11
Mackenzie
You know, we're humans, too. And they'll be like, am I allowed to say something, though? And for me, I'm like, this total opposite personalities where there's no hesitation on the project front to be like, nah, Munify, you made a mistake, you gotta fix it.
00:06:28
Mackenzie
and And that's good, that's constructive. But on the Whitehead side, so many of them are Not doing that. Just for one, i think it's mostly a personality thing.
00:06:39
Mackenzie
And I get it. One guy and your basement finding bugs on the blockchain is very different than and entire legal department.
00:06:44
riptide
Thank you.
00:06:46
Mackenzie
And, you know, the the marketing team of a big project and their technical developers all working together to not, you know, look bad. They got their reasonable concerns as well.
00:06:57
riptide
You also don't want to shoot yourself in the face to it. I mean, because if you're that guy, that lone guy in the basement, and you're looking at a six figure sum, there's there's this voice in the back of your head where you're thinking, well, if I say the wrong thing, even if you're trying to negotiate upwards, you're thinking, hey, maybe I could lose the whole amount if I say the wrong thing.
00:07:18
riptide
So I know that's a valid concern as well. In reality, I'm not sure, but I know that's got to be a concern because it's life changing money for a lot of people.
00:07:27
Mackenzie
On that note, the most, I would say, serious concern, and that that is very valid, is White Hats will say, the project offered, let's say, $20,000.
00:07:39
Mackenzie
I think it's actually worth, let's say, $50,000. Is it okay if I argue them or or ask for mediation in this? Because I don't want to lose the $20,000. I don't want the project to be suspended.
00:07:49
riptide
Mm-hmm.
00:07:51
Mackenzie
I don't want them to take it the wrong way. And... in I've never seen that happen. In all my years, if a project has made an offer and you think it's actually worth more and you argue for it or you ask mediation, which commonly happens, and I recommend doing if if that's what you believe, I've never seen a project like pull the rug out and be like, I actually pulled off the rug on the table.
00:08:13
riptide
This is big, big alpha. This is serious alpha we're dropping right now because I don't think a lot of people are aware of this because we don't know what's going on behind the scenes and people can think that they're going to go to zero.
00:08:24
riptide
And I used to think that as well. And now I look at it like I get an offer and I made a tweet about this, like my my skills in Saigon, you know, street street market negotiation.
00:08:33
Mackenzie
Oh, I love that. Right.
00:08:35
riptide
Because that's what it is, man. And I notice it more and more where I submit something. Okay. And then I just expect like a low ball in less, because if there's any nuance, any nuance, and I get it, I do the same thing from a project perspective. If there's any nuance where I could say, hey, well, you know, blah, blah, blah.
00:08:56
riptide
They're going to try to just, you it's like buying a used car. ah, well, these rims are scratched. Like, you know, guys got to get used to just like, you got to negotiate, man.
00:09:02
Mackenzie
bre
00:09:06
riptide
If you have strong points, you should use them and just try to get the max payout. When there's a clear limit where the guys aren't going to budge, you already know the problem with bounty hunting is like, you've lost your leverage.
00:09:18
riptide
And at a certain point, it's like, well, Hey, this is it. This is all they're going to pay. Unfortunately.
00:09:24
Mackenzie
There was the only caveat there, which is my genuine concern if someone's arguing for a higher bounty, is if you request mediation and Immunify comes in and says, hey guys, this bug's invalid.
00:09:38
Mackenzie
This bug's not worth anything.
00:09:39
riptide
you
00:09:41
Mackenzie
Which could happen. I've never seen it happen, but I'm always, just by personality, I'm very cautious.
00:09:49
Mackenzie
which works great because if I had to do these negotiations in conversation, I would be a whole lot worse at it. But getting to you know spend 10, 15 minutes thinking through the situation, another 10, 15 minutes writing out my response, being really precise,
00:10:06
Mackenzie
makes it a whole lot easier. And it is it is possible. you know you You find a critical, project lowballs you, Immunify comes in and says, hey, you you all overlooked this one thing.
00:10:17
Mackenzie
are these parts of world don't apply.
00:10:18
riptide
Mm-hmm.
00:10:20
Mackenzie
if Frankly, if the project offered 20K or whatever it is, I still wouldn't expect them to back out. I don't know why that is after all this time, because...
00:10:32
Mackenzie
there, it's consistent, though, I've always seen it that way. um
00:10:36
riptide
and No, that's that's good to know. I have had one bug and I can't remember the project, but you guys, it was a mediation thing and you guys recommended they pay out. and never I think this project ghosted, but they didn't pay out at all. They're like, well, screw you.
00:10:51
riptide
We're bouncing. And you win some, you lose some, but it is what it is.
00:10:58
Mackenzie
Yeah, there's there's a lot of
00:10:58
riptide
But yeah, if you get a low number and you have valid points, then then just argue it up, man. Good idea.
00:11:05
Mackenzie
i I did spend some time after, ah so so when I first joined Immunify, don't think this has come up anywhere before, I wasn't doing hacker success, I was doing product management.
00:11:20
Mackenzie
And then there was just a need, it was just like, hey, we got some problems to solve, you wanna jump in and do it? Started doing it, started talking to all and everyone. my my first, I don't know, 1,000, 2,000 followers,
00:11:33
Mackenzie
were just from people would tweet out, hey, I found a bug. Thanks for the help, McKenzie. And then I'd like boost up 50 followers. And that just kept on happening. I wasn't doing any tweeting myself,
Immunify's Rules and Mediation Process
00:11:43
Mackenzie
really. I was just reply guy-ing.
00:11:45
Mackenzie
And after doing that for a while, getting a feel for a bunch of similar problems, we ended up, I think most people don't know this, but we ended up developing some more I guess refined hot metal rules, which yeah yeah they're on Zendesk and they're called feasibility limitations, but it was just covering all of these really niche edge case scenarios.
00:12:11
Mackenzie
And one of them I think that people relate to most is like if I have to borrow $10 million dollars to execute a bug, is it still valid? and That very rarely comes up, but i yeah I think it's, when it does come up, we don't want have to figure out the rule on the spot.
00:12:30
Mackenzie
um And so when these tough situations do come up, we've got most of it in advance. And I went, I ah frankly, i could spend years just refining more and more bug bounty rules because there's so many edge cases, but it comes back to, you know, just having good negotiation skills.
00:12:49
Mackenzie
knowing when to stand your ground, how to word it. um you know to you know Negotiating on the streets of Saigon is is really the perfect example. i think of I saw this great YouTube clip the other day where some guy has taken a Snickers bar, which yeah for non-North Americans, it's it's just a candy bar, like unhealthy, high fructose and corn syrup sort of candy bar.
00:13:15
Mackenzie
and he's rebranding it as a protein bar, and then health snacks bar, and then a dieting bar, and you know he's going on the list, an energy bar. And he's doing a really good job. And the ingredients are the same, he's not lying, it's just a matter of how you're presenting it.
00:13:31
Mackenzie
And frankly, I think we could do the same thing in bug bounty hunting where I could take a valid report, something like a really serious bug and write it as a total noob, write it as someone who is really poor at communicating, write it as someone who looks like a spammer, write it as someone makes the bug look like it's not there just by how they communicate in the bug report.
00:13:47
riptide
Mm-hmm.
00:13:55
Mackenzie
And
00:13:57
Mackenzie
that's That's probably one of the biggest problems where you'll you'll do really good work and then you'll kind of start off on totally the wrong foot, which frankly, I wish the stakes were a little lower. I wish there were a way of making bug hunting a little more collaborative where there were more duos so you could share your bug report and be like, hey, can you give me some feedback on this?
00:14:19
Mackenzie
And almost every time I can guarantee 50% of it is going to get redlined.
00:14:20
riptide
Yeah.
00:14:24
Mackenzie
It's just gonna be like, remove that line, remove that line. Don't, you know, be so self um condescending. Don't speak down about yourself or the bug. Why are you negotiating? A lot of the time I'll see people negotiating against themselves in their bug report, like from the start.
00:14:40
Mackenzie
And I'm like, don't, you don't got to do that. The project is going to do that.
00:14:43
riptide
yeah
00:14:44
Mackenzie
They're going to be better at it than you are as well. You just present your strongest case, honestly, with integrity and, know, I know it's uncomfortable, but just wait for the criticism.
00:14:55
Mackenzie
Wait, you know, wait for the the attack to come. And then that's that's part of the healthy process. Yeah.
00:15:01
riptide
and And without being overly bombastic as well in your report, that is a clear GPT red flag. You know, like this is an ultimate critical.
00:15:10
Mackenzie
Yeah.
00:15:12
riptide
It will take down all blockchains of the world. Like shit like that. It's like, dude, okay, this the project is going to look at this and you might just get a triage that doesn't even read the rest.
00:15:22
riptide
You might get unlucky. You might just close it out and it could be valid as anything.
00:15:26
Mackenzie
there was I think it was Kan Kodu who said it well. It might have been someone else, but he was saying that the arguments you make in your bug report, they don't add up.
00:15:38
Mackenzie
You don't say, hey, I could do this critical thing. Also, I could do this minor stuff. Also, in weird edge cases, um you know I could do a little more damage. What happens instead is all of those arguments just average out.
00:15:52
Mackenzie
So you put a super impactful one, and you put rather meaningless one that probably would never even come up in reality. And it's not even better. In fact, it's worse now.
00:16:03
riptide
Mm-hmm.
00:16:03
Mackenzie
It's just the tension from what matters. and And that's the hard part. I mean, that's kind of writing 101, cut out all of the unnecessary stuff. And of course you get, i you find a bug, you get really attached to the thousand and one ways it could be exploited and all the theoretical scenarios in addition to the more practical ones.
00:16:24
Mackenzie
And it's just your baby. You just, you know, you just want to show all of it, tell all the world, talk it up. And that's like the opposite of what you got to do.
00:16:33
riptide
ah ah To be honest, and I agree with you. they The more wordy the bug report is, unless it's highly complex, ah it can be signs of a weak report. Because I'll tell you, the ultimate Chad report is two words, full drain and a working POC.
00:16:50
riptide
And when there's no nuance, it's just like, that's it. There's nothing you could say. I did it on a live fork. It's done.
00:16:58
Mackenzie
Yeah, 100%. um And frankly, if you got you know a bug and you're like, maybe it could be a drain, but all I can kind of prove is griefing right now or block stuffing, then europe you're better off, just frankly, I think it's a total waste of time to try to argue for the full drain if you know that's not true.
00:17:20
Mackenzie
Or not you or not a few know it's not true, but what I mean is if you don't know how it's true yet, if it's still a hunt,
00:17:27
riptide
Right.
00:17:28
Mackenzie
then either submit it and say, hey, this is griefing, it's medium severity, please give me $5,000. Or you go the other side and just hold on to it and be like, all right, going to chew on this one.
00:17:42
Mackenzie
I'm going to try and get a really big six-figure payout, but... ah you know Maybe someone else submits it before me. And frankly, i've seen that I have seen that happen as well, where someone will, I mean, this is probably one of the most painful stories. This is from way back years ago, when I first started helping people in their bug reports, and someone submits a critical, and it's combining multiple bugs together. It's really clever.
00:18:07
Mackenzie
And then someone else though, we had found out, had already submitted one part of that chain that totally solves all of it, but they'd only submitted it as medium, because they just didn't see the bigger picture.
00:18:19
Mackenzie
and And that did invalidate it.
00:18:19
riptide
Oh.
00:18:21
Mackenzie
that That was it. It was like, hey, this isn't a bug anymore. They've already paid for the bug that fixes this. And that was like a long dispute. There was miscommunications going around. It was a tricky scenario.
00:18:36
Mackenzie
um and then And then it ended like that. And I was like, dang, that's, that's painful. I've never seen it since, but I'm sure it comes up. and And that's kind of just the risk you got to take if you're going for if you're new, just, you know, get your reps in, submit as a medium.
00:18:54
Mackenzie
write a good bug report, keep on tracking. If you're more comfortable, more established, then I would say, you know, if like Riptide, you found and you're like, I got a hunch this isn't critical. It'd be like, yeah, it's probably worth it for you to hold onto it.
00:19:08
Mackenzie
And in a month, revisit it, even three months later.
00:19:08
riptide
Yeah.
00:19:11
Mackenzie
And and like, is there something more here? Or maybe there's something in a fork of that protocol that it sparks inspiration for. Keep that
00:19:19
riptide
If you're a sloth, you you lay low for two years. You just sit on it like your baby.
00:19:25
Mackenzie
insane patience and i i think that it is pretty correlated with age of that bounty hunter as well where you know if you're if you're 19 or you're like 23 i don't think from from what i've seen at least they seem to have a more hasty personality totally natural well meanwhile if you're
00:19:44
riptide
Of course. Hormones shooting everywhere. They're vaping, hitting the monsters. They're jacked up, ready to submit, ready to spend that money. Hell yeah.
00:19:53
Mackenzie
Yeah, and and it's a different game, and it it makes sense. If you can afford to go 16 hours a day, you know, every day of the week, why not?
00:20:04
Mackenzie
If you got ah family, okay, you probably need to play ah different strategy. Yeah.
00:20:11
riptide
but Let me ask you this to branch off. You're talking about new guys, right? So I saw someone on my discord. They were talking, maybe it was on Twitter. they They had gotten banned recently from ImmuneFi, too many spam submissions.
00:20:25
riptide
And he sounded like a new guy, sounded like um ah he was not native English speaker, ah which is another thing we'll talk about too. And how I think that those guys face a disadvantage just because communicating their bug report, you know, exactly how it should be, but how GPT can kind of help that.
00:20:44
riptide
But ah when someone submits, like give me the flow. So if ah a guy submits a finding and then the project, hits you know closed out or accepts? like what is the What's the flow on both of those?
00:20:59
Mackenzie
Yeah, yeah. All right, so we'll we'll take kind of the closed flow. And someone submits a bug report. goes through our early filter to kind of get rid of junk, save everyone some time, gets past that, goes to the project. They see it.
00:21:14
Mackenzie
ah This isn't right. They close it. um And they give a reason. And they have to give a reason. Many times it'll pop up on Twitter maybe once a month where a project forgets to input the reason.
00:21:24
Mackenzie
And the the message is just, hey, we're closing your bug report for insert reason here. And that's what that's what they send. And that is our template.
00:21:33
riptide
Not cool.
00:21:35
Mackenzie
ah Just to remind them, you do need to input a reason and give us such comical posts as that. um And then it goes back to the white hat. Do you want to dispute this, essentially?
00:21:46
Mackenzie
um And this is where bringing in Immunify for help would come in, request mediation, give more context on what the project missed or misunderstood.
00:22:00
Mackenzie
And from there, it kicks off the mediation flow. Sometimes it's just a matter of White Hat and project need to talk to each other. Sometimes it's a matter of, no, there's already a hard technical disagreement or rule break or rule misunderstanding and immunify needs to come in.
00:22:20
Mackenzie
um
00:22:20
riptide
And how many mediations result in, do you have stats on that result in, uh, towards the, uh, the researcher versus a project?
00:22:30
Mackenzie
So I the percent of paid reports that involved a mediation.
00:22:30
riptide
Is there any percentages that you know?
00:22:40
Mackenzie
And so this is just, you know, if we take 100 reports that got paid, so they're valid, low severity, critical severity, whatever. um How many of those required a mediation?
00:22:51
Mackenzie
Or just requested one, even. I'd be interested what your guess is, Riptide.
00:22:57
riptide
On how many require mediation? Jesus. I don't know. 30%. Okay. Excellent.
00:23:05
Mackenzie
Yeah, it's pretty close, yeah. it was It was about 20%. And...
00:23:08
riptide
Okay. excellent
00:23:09
Mackenzie
and That,
00:23:14
Mackenzie
um i was I saw that, i was like, yeah, frankly, i don't see any of the 80% that don't require mediation. um Because I only look at the problem cases. My view gets skewed towards problems and the way things can go wrong. But majority of paid reports, they come in, they go smoothly, project pays out.
00:23:36
Mackenzie
White Hat doesn't even need to ask for help getting the project to pay. um That's something that comes up sometimes. Someone pays too slowly. And that would count as a mediation.
00:23:45
riptide
Mm-hmm.
00:23:45
Mackenzie
Not even that happens.
00:23:45
riptide
Mm-hmm.
00:23:46
Mackenzie
About 20% of the time it goes smoothly for valid bugs. The vast majority of mediations are for invalid bugs. And that's, frankly, I think that's just a matter of it's really hard to know all of the ways a project might be mitigating against a given bug.
00:24:06
Mackenzie
Like the someone put it well, project knows their code base best, white hat second best, immunify third best. And the, yeah, so that's the way that one flows.
00:24:20
Mackenzie
So the vast majority of mediations, because we get, i wish I had the exact number here, um hundreds,
00:24:27
riptide
Are you limited to a certain amount? Like each white hat can only have so many mediation requests each month or something.
00:24:35
Mackenzie
we We used to not be limited. We used to be, um remember when I came in, there was no limits on a whole lot of things before ChatGPT wrecked the landscape.
00:24:48
riptide
Hmm.
00:24:49
Mackenzie
ah But even frankly, just when Immunify was smaller, just smaller communities have this advantage of higher quality. And nowadays we do not have any limit for, I think people who've got one paid report think that's, we have some threshold, something pretty simple and concrete like that.
00:25:10
Mackenzie
um And then you know you've proven yourself and you can request as many mediations as you like. And like in all, it's it's always kind of the low quality people or the spammers or the people who think they're great but are actually quite terrible that end up submitting, you know, but we have a rate limit of five bug reports per two days.
00:25:18
riptide
Okay.
00:25:33
Mackenzie
um
00:25:33
riptide
hmm.
00:25:34
Mackenzie
And frankly, i think there's like two or three times in my entire time at Immunify that someone has needed to post more valid bug reports in that. and And one of them was just trust found a bug that applied to like 30 protocols.
00:25:49
riptide
Okay, yeah.
00:25:50
Mackenzie
And I got to submit this to everybody. All right, thanks for letting know. We'll lift the rate limit for you. Here you go. and And yeah, a whole bunch of them were valid.
00:25:59
riptide
And so what about, but when they, when a project sees something, like, Hey, okay, this is, this is BS closing out. Do they get to ah rate the submitter? Is there some sort of feedback thing where, where you have more clarity on like, Oh, this guy, we're flagging for span. Look, he's, these projects are all saying low quality report. Are they giving you four stars? Like, is there any feedback from the project?
00:26:25
Mackenzie
Yes. So we we do have all of those. We do have a way they can flag for spam. And then that surfaces for us to investigate because often, frankly, the most of our spam comes from one person will come in and they'll spam like 20 projects all at once.
00:26:41
riptide
Mm-hmm.
00:26:42
Mackenzie
and And they'll just keep doing that. you know Every two days, they send another five bug reports. Every two days. And they're all critical. And you're like, wow, you're finding all these criticals across all these projects. It's sure amazing. No, no humility.
00:26:56
Mackenzie
um Before I mentioned the feedback rating thing, I just remembered the point that i've I've said many times over the years, where the vast majority of white hats, the more skilled you are, the more respectful you are.
00:27:13
Mackenzie
In fact, often the more too respectful, too polite, like, ah like I mentioned, people leaving money on the table, just being a friendly guy.
00:27:14
riptide
Mm-hmm.
00:27:24
Mackenzie
compared to spammers are the total opposite. They will, ah you know, they'll send me requests all the time being like, hey, this project is messing with me.
00:27:27
riptide
yeah
00:27:31
Mackenzie
They'll send these long paragraphs. They'll message a ton of projects. They'll request mediation on everything. They'll give these huge dissertations explaining why the project was actually wrong to close their report.
00:27:43
riptide
Is there a certain region these these people originate from?
00:27:43
Mackenzie
All these things.
00:27:48
riptide
Have you looked at geolocation?
00:27:49
Mackenzie
ah
00:27:51
riptide
I'm curious.
00:27:51
Mackenzie
I haven't looked at geolocation myself, but the vast majority of them are clearly web to bug bounty hunters that were spammers there.
00:28:01
riptide
Mm-hmm.
00:28:02
Mackenzie
And it works in that culture of web to bug bounty hunting places like hacker one, and then they're coming over here and they're kind of doing the same thing.
00:28:07
riptide
Mm-hmm.
00:28:11
Mackenzie
And, you know, they're like, Hey, I'm just trying to get a $20 payout. You know, I'm gonna, I'm gonna, he not how unify works but i'm uh you know it's insane how different the web 2 bug bounty hunting world is compared to in web 3.
Spam Origins and Cultural Differences in Bug Bounty Programs
00:28:28
riptide
who
00:28:28
Mackenzie
and uh i don't know where geolocation for most of them is i would expect
00:28:34
riptide
we we can We can guess if you're going to spend a lot of time trying to get $20 bounties, it's probably third world or developing countries.
00:28:38
Mackenzie
But yeah, you see more in the world where $20 is meaningful, where it's like, I can feed my family for a month on this.
00:28:43
riptide
Yeah.
00:28:46
Mackenzie
Wonderful.
00:28:47
riptide
Yeah.
00:28:48
Mackenzie
So, uh, You know, maybe there's a lot of assumptions you can make from there.
00:28:50
riptide
Yeah.
00:28:53
Mackenzie
And you there are some certain verbal quirks which come up where a lot of spammers will use this language like, you have you have wronged me.
00:29:04
Mackenzie
This is such an injustice. How dare you do such an affront against the good of people. and
00:29:09
riptide
That guy's probably from California. i don't know what you're talking about.
00:29:12
Mackenzie
yeah
00:29:14
riptide
All right. So let me, let me, I want to give you some feedback here. These are, since I have you on, I have to give you straight up feedback, a feature I would love to have. Okay. ah How about a whitelist for, if if you've submitted a valid bug on a project, how about you're on the project whitelist for,
00:29:36
riptide
comms with them. or some There has to be some way to do this because I can't tell you how frustrating this is if I submit something and then they close it out with whatever reason it is and then they miss something and then it's like, okay, now I have to do this mediation thing or i have talk to ImmuneFi because I can't respond.
00:29:39
Mackenzie
Thank you.
00:29:57
riptide
if there was a way, if I was, you know, a whitelisted guy could, could just have a chat with them, ah chat box, something where I could say, hey listen, you know, this is that because sometimes I have to go through back channels, through telegram, whatever, and talk to the guys on the backend and then resubmit.
00:30:16
riptide
and it's just like these, these little conversations can happen on the side, but there's no way to do that through the platform currently.
00:30:22
Mackenzie
Right. Yeah, that does suck. There's a few ways we've looked into solving that before. And, you know, The low bar that we do meet is a project can see your hit rate and your ranking on Immunify.
00:30:39
Mackenzie
So they have some kind of meta context about who's coming in. But the the core problem is there's no instant messaging function on Immunify. And we got this feedback and implemented some also relatively smaller changes to improve it where mediation feels intense.
00:30:58
Mackenzie
requesting Immunify mediation when it's just like, hey, it's just just like you know a two-sentence message, little thing to clear up, little question I have for you guys. And you know i'm I'm a top white hat. I'm a serious guy I'm not here to waste your time.
00:31:12
Mackenzie
And there's no way of doing that without you know kind of laying on the hammer and then Immunify comes in with our big mediation assessment. And that's not...
00:31:21
riptide
ah Right.
00:31:22
Mackenzie
you want You know, it's bringing a hammer to a problem that does not need that sort of force. um
00:31:30
Mackenzie
Unfortunately, the the ideal solution, because and implementing instant messaging, I'm pretty sure from a technical perspective is is not worth the lift in terms of other improvements to immunify in general and other things to make it work.
00:31:49
Mackenzie
The ideal, frankly, would be some direct communications channel like discord telegram or whatever people use where a project could just make essentially a ah chat with them and somehow keep it centralized, somehow keep it organized, somehow keep it on an ins instant messaging platform that people actually check every day, which
00:32:10
riptide
Just even your Discord. You could have, if they have a bounty, then have a private channel where, you know, i don't know, you could be whitelisted to be in there or something like that. It might be the easiest.
00:32:22
Mackenzie
that I would like to do that. One of the ideas before was pretty much doing that. do like ah and Let's say we got Polygon. And we're like, hey, Polygon really likes these three or four white hats who've done great work or or just submitted good bug reports. And if they got questions, they want to hunt on them.
00:32:40
Mackenzie
But the... work to manage that for 100 different projects and ensure you know the conversations are relatively moderated just grows out of control.
00:32:54
riptide
Yeah, yeah.
00:32:54
Mackenzie
And in Discord, you know ah adding 100 Discord channels um
00:33:01
riptide
Yeah, no, no, I hear you.
00:33:01
Mackenzie
for the organization of view is brutal.
00:33:03
riptide
Yeah.
00:33:05
Mackenzie
I do think it would be a huge improvement, and I do expect we'll have it one day. I did think maybe there's a way we can make projects pay for this, where it's like, hey, we'll connect you with five or six top white hats, and you'll give them one-on-one conversations and so on, and that'll incentivize them to hunt on you because you've already got this good rapport, and you're going to answer their questions so they don't go down pointless rabbit holes.
00:33:29
Mackenzie
um But launching new products like that and and services is its own huge amount of work, which is a long way of saying I agree that sucks.
00:33:38
riptide
No, no, good point. Yeah.
00:33:42
Mackenzie
I'm sorry.
00:33:43
riptide
Bullshit, man.
00:33:43
Mackenzie
i
00:33:44
riptide
Get Mitchell on the freeway call right now. Let's get him on this pod.
00:33:47
Mackenzie
yeah
00:33:47
riptide
One answer is right now. All right, I'm going to move on to, i don't know if you had a chance to go through some of the questions that I shot over to you. I shot them over kind of late.
00:33:59
riptide
ah From the Discord, some people had some some burning questions for you. Did you get a chance to look at those or are are we just going hot and unprepared?
00:34:06
Mackenzie
I did. I remember I saw them on Discord first. Let me pull that up because I remember there were a whole bunch.
00:34:10
riptide
Okay.
00:34:14
riptide
Yeah.
00:34:14
Mackenzie
And they were data-oriented questions. um
00:34:20
riptide
Whatever you can answer, answer, but we'll, we'll just, I'll kind of go through a few and then if you can't answer, it's fine.
00:34:20
Mackenzie
First of all...
00:34:27
riptide
But if you can give any input, I think it'd be good. So we'll go through
00:34:31
Mackenzie
So...
00:34:32
riptide
so Go ahead.
00:34:33
Mackenzie
Yeah, the first batch of questions is percent of reports where the white hat is happy. Were those paid with mediation, paid without mediation? unpaid with mediation, unpaid without mediation.
00:34:46
Mackenzie
And this ties back to the feedback question you were asking earlier, where we do have not just for projects to flag spam, but for projects to rank give feedback on the white hat and kind of give a simple numerical score.
00:35:03
Mackenzie
And same for the white hat to give a simple numerical score. Did the project treat me well? 1 to 5. Did Immunify treat me well? 1 to 5. um And the problem is almost no one uses that.
00:35:16
Mackenzie
I don't even know if you know about that.
00:35:18
riptide
I've never used it.
00:35:18
Mackenzie
And that's that's on us, making it making it more clear that they this feedback mechanism exists.
00:35:20
riptide
I didn't know there was that.
00:35:26
Mackenzie
um And as you would expect, ah well, I'll just tell you, because not as many people use it, sample size is really low. Don't really trust it too much. But the inferences there is if your report is unpaid, you're unhappy.
00:35:42
Mackenzie
And if your report is paid, you're very happy.
00:35:45
riptide
Yep.
00:35:46
Mackenzie
You know, kind of kind of stereotypical, um higher ranked white hats generally have higher satisfaction rates. Beginners, you know, people who've never been paid before, spammers or not, are usually unhappy with, you know, how things worked out.
00:36:02
Mackenzie
um i I frankly think that's just a matter of understanding how much goes on behind the scenes to make this work. from a technical perspective, but also just from managing the relationship with the project and making sure the rules are clear cut.
00:36:20
Mackenzie
um So not too much useful information there. the The more interesting stat was that of paid reports, only 20%, maybe a little, up like less than a quarter percent ah need mediation.
00:36:36
Mackenzie
That they can work it out smoothly with the project on their own, um which was... I didn't even know that myself until I looked it up in preparation for for this chat.
00:36:48
riptide
Less than a quarter.
00:36:49
Mackenzie
Less than a quarter, yeah.
00:36:51
riptide
Wow. Wow. Okay.
00:36:53
Mackenzie
which When we get hundreds of mediation requests a month, we do not get hundreds of paid reports.
00:36:58
riptide
Okay.
00:37:00
Mackenzie
i think the And you can look at our public stats on this, where it says how many bugs of each severity were found of valid bugs in the last month.
00:37:12
Mackenzie
And it's actually pretty consistent, right? It's about 80 to 100 valid bugs are found per month. And this is separate from audit competitions. This is just bug bounty stuff.
00:37:24
Mackenzie
and yeah The surprising thing here is that there's about an even distribution of severities. So 25% of them will be low, 25% will be medium, 25% will be high, and 25% will be critical, which I think about lines up with the last post we did for the July earnings.
00:37:44
Mackenzie
There's about 100 bug reports, 29 were critical. um Pretty standard stuff, just the way it works out.
00:37:52
riptide
Okay. ah What about, have you guys fixed this issue? I don't know if you've come up with some sort of solution here, because I know you have your own business model to to make some cash, but where the ah programs ghost the white hats, ah get their they get their vulnerabilities and they don't pay, they bounce and disappear from your platform.
Handling Projects Ghosting White Hats
00:38:16
Mackenzie
Yeah, those haven't happened as much lately. um But there have been there have been a few projects, I don't have an exact number, I guess somewhere around a dozen. But let's let's be liberal and say, maybe around 15 to 20 projects have been removed, in my experience on Immunify.
00:38:42
Mackenzie
um
00:38:42
riptide
Mm-hmm.
00:38:44
Mackenzie
and often we work really often it's just a misunderstanding frankly um a small amount of those they're just like nah we're just not gonna pay and this is like the fear situation maybe like three to five bug bounty programs over the last two and a half years have just come in in bad faith and refuse to pay one or multiple white hats And and you know we talked to them, we tried to work through it.
00:39:15
Mackenzie
It became clear they were bad faith, we we removed them. It sucks, brutal situation. More commonly, a project didn't understand what they were getting into.
00:39:29
Mackenzie
And so we say, hey, you got to pay out fifty k here 100K there. ah hundred k there and they are they're like and no that doesn't make any sense to us you know we'll pay 10k max out of the kindness of our hearts but we don't even think we should do that that's us being generous and those are the tricky ones because you know they're operating in good faith we just have different views on the situation and you know that's probably more on us for um not onboarding them
00:39:47
riptide
Mm-hmm.
00:40:02
Mackenzie
as comprehensively as we needed to.
00:40:05
riptide
but I think your latest issue there was with the guys that ran AP Wine.
00:40:05
Mackenzie
and
00:40:09
riptide
They rebranded to some other protocol, and it was a public dispute you guys had. And, yeah, they I think they're it sounded like there was miscommunication between or misunderstanding between what you guys were saying what they had thought they had to pay.
00:40:25
riptide
So, yeah, I don't know what project it was, but yeah, no, I see how that could happen.
00:40:25
Mackenzie
Oh, yeah.
00:40:30
riptide
i think I think it'd be good, like from our point of view, and I know it's difficult to do this and and it impacts getting new customers, but to say, hey there's ah and I know you had the vaults, which I don't know if they're really used, but put down a bond. All right. You need to put down, if your bounty's a hundred grand, you have to put down, i don't know, 10, 20% bond.
00:40:49
riptide
And we're just going to hold that in our custody. And then if you're proven to be a bad actor, well, you lose your bond.
00:40:57
Mackenzie
Yeah. When,
00:41:00
Mackenzie
i mean, frankly, the bond is a good idea we, we have more market dominance, I expect that'll be implemented just as due course.
00:41:11
Mackenzie
um I'm a little concerned that I don't see platforms like HackerOne do this in Web2 and these other big established 10-year-old plus Web2 bug bounty programs.
00:41:25
Mackenzie
um I don't know why they don't do it. Frankly, they treat white hats relatively terribly compared to not just Immunify, but everyone in Web3. And I expect that'll be more possible as you know the business solidifies more.
00:41:43
Mackenzie
um We do have some mitigations, like not just vaults and not just the arbitration we built on top of vaults, which is projects opt into, show of good faith.
00:41:55
Mackenzie
If they disagree with our mediation assessment and it disagrees with the rules, it's a real world international legal framework to take it to court and settle it in court.
00:42:07
Mackenzie
And the white hat isn't the one fronting $100,000 on lawyer fees. That's immunifies already got that set up. So the white head I think has to put up ah some dollars.
00:42:18
riptide
How many of those has happened? How many arbitration actual arbitration events have occurred.
00:42:27
Mackenzie
Oh, zero. it I think only, ah yes i think a dozen, two dozen projects are using arbitration right now.
00:42:29
riptide
Okay, that's good.
00:42:35
Mackenzie
Naturally, they're the serious projects, the ones that are already more security conscious, more good faith, more wise about why this is important and how it works.
00:42:38
riptide
Mm-hmm.
00:42:47
Mackenzie
And we haven't had any arbitrations escalate with them yet.
00:42:52
riptide
Yeah.
00:42:52
Mackenzie
um It's frankly going to be very exciting when it does, as there's been an insane amount of work from people far more skilled than I am to make this work.
00:43:03
Mackenzie
I remember they teammates were telling me they won some prestigious legal innovation award. I didn't know that was a thing, but It is, and so it's it's going to be a big deal when there's the first um case goes to court because that's never even happened in Web 2 yet.
00:43:20
riptide
yeah
00:43:20
Mackenzie
There's been some threats of it, and it's come close. I've inferred a couple times, but 20 years hasn't actually happened.
00:43:29
riptide
Well, i'm I'm sure we'll hear about it if it does happen on the Web3 side. ah So I want to get back to one of the questions from AV that we missed. He asked about predefined scopes. How are they determined, ah including severities?
00:43:43
riptide
are they Is it just by the projects? Do they just pick what they care about? Or Immunify also have some role in it? Because you can go to bounties and it can vary widely. some And from my point of view,
00:43:54
riptide
It makes sense for the project to be in control this because they should be paying out for what they care about. ah But sometimes you'll just see maybe two impacts in scope for certain projects. So how does that kind of work?
00:44:06
Mackenzie
The evolution there is fun, where when Immunify started, it was we had ah we had a loose framework. It was very collaborative.
00:44:17
Mackenzie
And then i can mostly look at bu um bug bounty program rules from the various, you know, Immunify is four years old from each year. And without knowing the year, just know which year it's from based on the terms used.
00:44:31
Mackenzie
because it's gradually become much more precise, much more standardized. And today, the project does decide the scope. And they within limits, they do decide the severity as well.
00:44:47
Mackenzie
And when I say it within limits, I mean that each impact has a given severity. Griefing is medium. um Stealing yield is high.
00:44:59
Mackenzie
Draining all the funds or permanently freezing funds is critical. And sometimes projects want to change that. They say, oh, you know, permanently freezing is actually high.
00:45:12
Mackenzie
And In previous years, we would accommodate to that newer company wanting to get traction. These days, it's much more, no that that's ah that's a critical severity.
00:45:25
Mackenzie
Unless you can present a really serious case why that shouldn't be critical, which almost never would come up, then we do not allow them to change the severity of impacts.
00:45:38
riptide
There's still a huge's there's a huge gap here, though.
00:45:38
Mackenzie
They do get do
00:45:41
riptide
There's a huge gaping hole that I have a big problem with.
Critical Payout Guidelines
00:45:44
riptide
is like I like what you said, but if you have a project that says this is a critical high whatever, and they want that big $1 million dollar payout for criticals on their page, and then they'll say, hey, critical, ah yeah, minimum $20,000, maximum $1 million.
00:46:01
riptide
And then the level of just... The question mark there for everybody involved is ridiculous. Like you submit a critical and well what warrants the max payout? What warrants a mid payout? Because obviously they're going to go lower on the scale and it gives them so much leeway to say, oh, we have this big program for advertising for their board. Everyone would see that.
00:46:25
riptide
And in reality, it's like, well, we could pay. i know they have and internal conversations. They look yeah critical. Okay, listen, we could we could downplay, we could pay very little unless you know it's the two word chat report, full train.
00:46:40
Mackenzie
Yeah, the and we we had this issue more on our older bug bounty programs where there wasn't a good minimum critical payout.
00:46:52
Mackenzie
Nowadays, the principle we use, and I think this is accurate, it's off the top of my head. I haven't looked at this for some months, but I believe it's intended to be either 25% of your max critical or 50K, whichever is lower.
00:47:13
Mackenzie
So it's always intended to be ah meaningful amount of money, despite that question mark. which especially applies when funds aren't directly at risk.
00:47:27
Mackenzie
If they are, great.
00:47:27
riptide
Those the hardest ones. Those are the hardest ones to justify. Because if you don't if you can't quantify it, it's a fucking mystery. It's a black box for everybody.
00:47:38
Mackenzie
and And it applies most for blockchain companies. level protocols rather than apps built on top of them because so much of that is network gets shut down.
00:47:50
Mackenzie
um How do you quantify that? And with especially those are usually the more big bounties and usually the more serious projects. And more of the new ones that, or when we update their bug bounty program with them, will say, let's put a number on each of these unquantifiable impacts.
00:48:12
Mackenzie
All right, fifty k is for maybe permanent freezing of funds, but what about the other critical blockchain impacts?
00:48:12
riptide
Mm-hmm.
00:48:24
Mackenzie
um What about, you know, unin chainined unintended chain split? So you got to do a hard fork. Is that worth more? that work worth less? How are we going to work this out?
00:48:34
Mackenzie
um And let them, and usually they'll, you know, they'll they'll put some variation on that. um But frankly, yeah, that's that's difficult. And if if funds aren't directly at risk, the usual method is just whatever is the minimum payout, the project has agreed for that.
00:48:56
Mackenzie
So you find, and and that'll often be, you know, for these bigger projects, 50K.
00:48:56
riptide
Mm-hmm.
00:49:01
Mackenzie
um Would it be better if it was more?
00:49:05
riptide
Of course, note to bug hunters, note to bug hunters, you have to take the money.
00:49:05
Mackenzie
doubt. No doubt.
00:49:10
riptide
That is how you prove and get the big funds, the big bounties. Dude, a lot of these questions, and think come up before with ImmuneFly, and maybe you have your own rationale for competitiveness and and stuff like that for...
00:49:26
riptide
for not releasing some of this stuff, but a lot of people want the data. They'd like to know, you know, all kinds of bounty size, severity, language, percentages, how big is the internal immune five findings database?
00:49:38
riptide
Um, I mean, there's there's a ton here of just like putting out a lot of data.
00:49:42
Mackenzie
right
00:49:43
riptide
Uh, do you think there's going to be a page where, you know, we could see, just see some of the stuff that that the guys are asking for?
00:49:53
Mackenzie
This has actually been project i've I've looked into before because the data is really fun and really cool. the And some some of it definitely needs to be private.
00:50:04
Mackenzie
you know Who submitted what, where the bugs were on, things like that.
00:50:07
riptide
Mm-hmm.
00:50:09
Mackenzie
um But we've got enough projects where we can say, hey, there were 10 blockchain level criticals found in July, and you won't know exactly who it is.
00:50:20
Mackenzie
there is There's enough of them that it gets it gets hidden. um The reason we haven't done it so far is it' it's just a lot of work.
00:50:31
Mackenzie
um Digging up the information, parsing out the meaningful part of the information. um and Naturally, Immunify doesn't want to look bad ourselves, so we we want to be thorough in and accurate in how we present it.
00:50:48
Mackenzie
um which, you know, that that was, know, to be totally upfront, that was one of the reasons we didn't post the monthly bug findings and earnings previously, because there were some months where only like 500, 600,000 in bug bounties were paid.
00:51:07
Mackenzie
And we were like, guys, that's not very much. and you know we're We're looking for like 2 million, 3 million, you know, 6 million. we yeah What are people gonna think about this? and And we went with it anyways.
00:51:19
Mackenzie
And, you know, no one ever one ever blinks. No one ever thought that was bad. Tons of bugs are getting found. Those are large amounts of money. Yes, ideally, it's it's at least a million plus. But yeah, there'll be slow months, they'll be busy months.
00:51:32
Mackenzie
And the hard thing.
00:51:33
riptide
Yeah, seeing seeing those numbers, guys, guysza we get like the FOMO, man. We see like, wait, how many millions?
00:51:38
Mackenzie
um
00:51:39
riptide
And then you you see maybe maybe nothing's been disclosed on X. No one's no one's raised their hand and you're like, oh shit, man, who was it? Who's earning all this cash? I haven't found shit.
00:51:52
Mackenzie
There was an idea, think it was Sorry Not Sorry suggested to me, where we post, we just make a Twitter account for like Immunify heartbeat.
00:52:03
Mackenzie
And every time a bug is confirmed or is paid, it just does a little post. It's like, hey, a critical was found on a smart contract protocol today. um
00:52:14
riptide
More FOMO.
00:52:14
Mackenzie
Shout out the white if they accept it. And, and, you know, if we got a hundred bug reports getting paid a month, that's about three a day. That's, uh, that's, that's something fun.
00:52:26
Mackenzie
That's something cool.
00:52:26
riptide
Yeah.
00:52:27
Mackenzie
Um, And there's a lot of stuff we could set up with that. And the hard case, and you know I'm i'm um open to anyone to DM me and please convince me to pursue these ideas, is that it's always a whole lot more work than it seems like at the beginning, especially Immunify wants to do things to a high quality.
00:52:47
Mackenzie
And especially because we gotta to make sure that you know we're we're respecting our project's information, um we're doing things accurately, all All of this. ah So there's not so much scrappy throwing stuff at the wall um just because of the security nature of things.
00:53:05
Mackenzie
And like every other company, the in the the tendency is to go too slow. So if someone could say, hey, here's the business case on why you should put out all of the stats. break down number of bug reports submitted per month, per year, based on the protocol type and the language, what were how many were paid, how many requested mediation.
00:53:28
Mackenzie
Already, you know you can tell this is becoming a very big data heavy page, which it takes someone data intelligent to make meaning out of. um then then I'd be all for that because, frankly, we've got dozens of internal data pages that we'll set up, and I struggle to parse information out of them.
00:53:51
Mackenzie
um So in terms of something more public, it'd be great to have someone else doing that, but it's a ton of work.
00:53:58
riptide
Claude Code, get on that two hours, you're done according to X.
00:54:02
Mackenzie
Yeah.
00:54:04
riptide
ah yoll YOLO your your data page.
00:54:06
Mackenzie
so and mess with the immunified code base. We trust you. Be secure. Pretend you're a top security developer here. Make one mistake.
00:54:16
riptide
That's great. Hey, I'm going to drop because you already dropped some some good alpha, man. I do not want to forget my alpha drop. I'm going to missed it the past podcast. I'm going to keep this one simple.
00:54:27
riptide
This is going to be a very simple one, but it is it could lead to bugs. When you are looking through the code and you see something that reverts, that is not always the end of the story.
00:54:42
riptide
You have to see if that revert bubbles up or not. And what I mean by bubbles up, if it reverts a whole transaction or if it doesn't, because if it doesn't, then you may have found some undesired or unexpected behavior.
00:54:59
riptide
Top alpha tip, alpha drop today. Do you look for bugs but like in your in your part-time when you're up in Alaska camping or whatever the hell you're doing up there?
00:55:08
Mackenzie
i um I don't. I think there is a rule where if you're at Immunify, you're not supposed to be publicly bug hunting as well for conflict.
00:55:20
riptide
I wonder why.
00:55:21
Mackenzie
I had an issue where a white hat got a big payout and was like, hey, I want to, you know, you helped me so much. I want to give you a gift. And I was like, I'm not, I'm not sure I'm allowed to accept gifts.
00:55:34
Mackenzie
I really want the gift, even if it's just, you know, some super expensive NFT.
00:55:35
riptide
Thank you.
00:55:41
Mackenzie
I, you know, please send money my way. um, And then, you talked to it over was like, Hey, is this going an issue? And yes, it's almost like, yeah, it's kind of a conflict of interest.
00:55:53
Mackenzie
we We can't have someone at Immunify explicitly trying to get white hats paid um with a financial incentive. And unless we started doing that for projects as well. And it's like, yeah, we do not want someone who's helping projects not pay white hats.
00:56:09
Mackenzie
We want to keep it impartial. um So no bug hunting.
00:56:12
riptide
So then you fired up your alt and then gave him a ah random ea Yeah, that's good, man.
00:56:14
Mackenzie
ah
00:56:18
riptide
That's good. No, that's man.
00:56:20
Mackenzie
Let's segue into one of the other questions where someone asked, when a new attack vector is found, how off how long does that take become before it becomes a common attack vector in the public auditing space, like in audit competitions?
00:56:37
Mackenzie
And frankly, think that... maybe maybe maybe a long, long time where, I'll talk to top white hats and I mean, Riptide, your point of view would be interesting on this.
00:56:52
Mackenzie
If you find some novel attack vector no one's really talking about, has doesn't occur very much, or at least doesn't isn't publicly known to occur very much, even if it is a common one, would you do a write-up on it or would you kind of keep that alpha to yourself and and milk it for as long as you can?
00:57:11
riptide
Usually everything that gets written up, and this is probably anybody, they've done an exhaustive search across GitHub, across deployed contracts on every chain to make sure that they've already they've exhausted. No one's going to give you literal alpha, like if you can convert that to cash right now.
00:57:32
riptide
So everyone's going to look and and say, hey, did I do my best to find if I can already report this? If not, hey, I'll put it out there because maybe A, I want some publicity or or just want to put it out there to to give some some guys a leg up.
00:57:47
riptide
But for the most part, I would imagine everyone thinks like that. They're self-interested to at least you know at least look as hard as they could to find if that's anywhere else first.
00:58:00
Mackenzie
there is There's an extra layer on top of that where people who are consistently bug hunting and finding criticals like Lonely Sloth, like White Hat Mage, like Kankodu, Blue Chads, they're hitting every pitch that comes their way and it's a home run.
00:58:14
riptide
True chats. Yeah, true chats.
00:58:22
Mackenzie
They don't have, it's not worth their time to write up these complex bug reports where, you know, writing for on one hand is hard and and takes a lot longer than you think it does.
00:58:30
riptide
Mm-hmm.
00:58:35
Mackenzie
And if you're going to send it out there, you you probably want to make sure it makes you look good. You're not leaving tons of typos and just writing a really confused ramble. And you could just be finding more bugs, man.
00:58:49
Mackenzie
Maybe you don't even like writing. i
00:58:52
riptide
yeah But I mean, a lot a lot of these bugs, you think they're going to be repeatable, but in reality, they're not.
00:58:52
Mackenzie
Many such people.
00:58:59
riptide
I'll find something and I think, oh, ah this has to exist somewhere else. And then I never see it again. So a lot of these are unique kind of vectors. And I think that's what gives me the incentive to do a write-up. I haven't done one a long time, but...
00:59:14
riptide
When I do see something interesting, I'm like, oh, this this is just cool on on how the mechanics worked. And yeah I'd like to share it. But you know if it's if it's if you just find something kind of repeatable and someone left a function exposed, it's like you know you're not going to do a write-up unless it it led to like you a huge bounty like the wormhole one or or something like that. But other than that, i mean, it has to be interesting to read about.
00:59:41
Mackenzie
there was I would wonder how much of the time those bugs are actually unique and how much of the time it's just... that particular white hat couldn't find other instantiations of it because I yeah remember the 100 proof bug write up on his quiet Kyber swap find, huge, incredibly technical, incredibly deep.
01:00:01
riptide
Mm-hmm.
01:00:04
Mackenzie
In it, i think he said even that, you know, when I had a hunch or when I found this bug, I wasn't in a rush to write it up and POC it and submit it immediately because of how deep it was.
01:00:16
Mackenzie
I know there was no one on my tail right about to front run me with it.
01:00:19
riptide
Ha, ha, ha.
01:00:20
Mackenzie
It took me weeks. And then he eventually does write it up. um Very unique, very math heavy bug. And you know later on, we have the Kyber hack, which was a was a variant of it in that still existed in the code.
01:00:40
riptide
Right.
01:00:41
Mackenzie
And it just, you know took a pair of fresh eyes. Unfortunately, it was it was a black hat instead of a white hat. um
01:00:47
riptide
You must address him as Kyber director or else the ego of that guy.
01:00:51
Mackenzie
So funny. I go i
01:00:54
riptide
That's Hall of Fame right there.
01:00:57
Mackenzie
ah i need that story completed. i need someone, you know, making a little YouTube documentary on that.
01:01:03
riptide
it's It's probably already out there.
01:01:04
Mackenzie
So great.
01:01:05
riptide
But that you know that's one of the hardest things to do is when you find something and I mean, hundred shout out to 100 Proof, is when you you find something and you're like, you have a big urge to report it
01:01:06
Mackenzie
Uh-huh.
01:01:18
riptide
even if it's not like fully sussed out sometimes. And it takes a lot of self-control to say, hey, wait, like he's able to do and just and just take a while to to fully investigate and write up the whole thing. And that way you fully understand it.
01:01:33
riptide
And hopefully you see any other kind of gaps in it as well.
01:01:39
Mackenzie
It's almost a platitude to tell people, you know, stay calm, go slow, you know, look where you're stepping, look what you're doing, don't rush it.
01:01:50
Mackenzie
But it's so true. I saw even one of the retired UFC champs, Mighty Mouse, um he was saying the same thing. and yeah He's 38 now, but he's like, I'm looking at my fights, especially my younger fights, and there's nothing more time sensitive than someone right about to punch you in the face.
01:02:11
Mackenzie
And he's still saying, you know what, I should have i should have taken it slower. I should have been more calm and attentive in my fights and not jumped on every perceived opportunity, but to have been more precise about it.
01:02:26
Mackenzie
And in bug hunting, I think it's the same thing. These big bugs are usually going to be deeper ones, or it's just going to be something that takes a fresh pair of eyes, looking it over from scratch.
01:02:40
Mackenzie
And there is no speed run in that.
01:02:43
riptide
Yeah. Yeah. Don't rush. That is, that is really, it's easy to say, but hard to do in practice.
01:02:49
Mackenzie
I will say ah for bug bounty negotiation about Literally 80% of the support I give people is, this is okay, this is normal, the project is trying to invalidate your bug report.
01:03:06
Mackenzie
Talk to the project. Okay, if that doesn't work, request mediation. No insults, don't lose your cool, take a day or two off before you respond, that's okay, and just make a really clear technical point.
01:03:23
Mackenzie
Leave it at that. don't Don't kind of get ahead of their argument and argue it before they've argued it and and you know go five layers deep. No, just say what is relevant to say right now.
01:03:35
Mackenzie
Let them respond. Take it slow.
01:03:37
riptide
Very good point.
01:03:37
Mackenzie
And don't blow them up on Twitter.
01:03:38
riptide
Don't blow them up on Twitter. Just chill the till the fuck out. I give this, as I've done this for a while, I've learned, and i I give this piece of advice, and and I look at things through the eyes of incentives. And I say, before you type that message, think about what is your intended outcome by doing this? And is it going to help you achieve that outcome?
01:04:01
riptide
Yes or no?
01:04:02
riptide
Talking shit on Twitter is not going to help you get anywhere and everyone's going to forget about that. And, you know, the the cred that you're getting from that, that you think you're getting is is nothing.
01:04:09
Mackenzie
yeah
01:04:14
Mackenzie
even and Even if you are going to go public, and sometimes it makes sense if you got treated wrongly, just let things resolve first.
Going Public with Disputes?
01:04:24
Mackenzie
No need to try and dunk on a project mid-conversation.
01:04:29
Mackenzie
Let them dig their own grave, you know deny you your just payout, probably get kicked from Immunify, or maybe they exploit some some gray area rule and and they get to stay around, but still, okay, now now I can write it up.
01:04:44
Mackenzie
Now I can go public with it. um And if you're not allowed to go public, you can that's its own piece of advice, being like, hey, Project, I'm going to write this up. you know Is that okay with you?
01:04:56
Mackenzie
Just get ahead of their fight about it. And if they're saying no, you're like, all right, does this make you look more suspicious or less suspicious? Because you know if someone slanders Riptide, I think Riptide as a business is going to be okay.
01:05:12
Mackenzie
If someone drops a huge criticism that's very valid ah that we see every now and then on crypto Twitter about some project, and it's like, hey, they were clearly trying to take advantage of this white hat.
01:05:25
Mackenzie
Then it's it's kind of a public good, really, um because you won't personally benefit from it. The story's done. You wouldn't have benefited from it from...
01:05:37
Mackenzie
criticizing them before it was done anyways. But other people in the ecosystem are going to be better. and And that's one thing I love about this space. I know we got ah wrap up in a bit, but the standards in Web3 security in terms of ethical behavior are insanely high.
01:05:59
Mackenzie
And this didn't happen by accident. A ton of people did a ton of work to say, this is what we tolerate, this is how we treat each other, and we won't we won't do otherwise.
01:06:14
Mackenzie
um I think the kind of less active but face of high quality behavior in Web3 was Socks over at C4.
01:06:26
Mackenzie
And he was always... very friendly, very supportive of projects, always trying to have a conducive relationship between all parties, not tolerating um anyone taking advantage of each other, really.
01:06:40
riptide
Mm-hmm.
01:06:40
Mackenzie
And that And that had huge ripple effects. you know People come into the industry, and they're so friendly to each other. You go into web2 bug hunting, people are are not so friendly.
01:06:52
Mackenzie
you know Your relationship with a project is antagonistic. It's me versus them. and And the platform is going to take their side. Everyone knows this.
01:07:03
Mackenzie
And so yeah, I'm going to spam, and I'm going to abuse, and I'm going to try and lie to get small payments here and there. um And we don't have any of that in Web3, which is...
01:07:18
Mackenzie
you know really remarkable. We kind of have an issue where some people are spammers are more empowered to take advantage of AI now, but how often do we see on Twitter some AI audit firms saying, hey, we we won this competition or we were better than private audits manually done.
01:07:35
Mackenzie
And everyone's like, no, no, we we know you're lying. People are like, I will do a competition against you. People are not tolerating any level of bullshitting.
01:07:46
Mackenzie
um And especially, you know, especially in terms of like the businessy, salesy sense of things, there's very, very little tolerance for people misrepresenting themselves.
01:07:53
riptide
Mm-hmm.
01:07:59
Mackenzie
And think that's great. I think everyone benefits in bug hunting, specifically, the projects, can have a much better relationship with White Hats. As much conflict as you think there is now and you know incentive to fight each other, things are so good.
01:08:15
Mackenzie
We are so lucky that projects and White Hats are able to have a good relationship with each other. Not always a pleasant one, but good faith, technical arguments, and big payouts. there' There's no fake arguments.
01:08:32
riptide
That's what we like to hear. That is what we like to hear. Onward.
01:08:36
Mackenzie
But my thing there is I appreciate when people call out Immunify. Like I did a tweet on this earlier. Someone was saying, hey, I had a bad experience in the bug report. And, you know, we had a big conversation internally. i talked to him, you know, sent some responses as well. Other people did also. And it's like, this is good.
01:08:54
Mackenzie
we we are going to slip up. We're going to lose sight on what's important. Unless other people hold us accountable, we're going to hold you accountable in return. And I mean, it's just life's so much better that way.
01:09:08
riptide
Yeah, that's what we like to hear. And I think you guys are doing a great job. And I think clearly dominating the space for that reason. And we are at our hour mark. Mackenzie, thank you for coming on.
01:09:19
riptide
ah We will see everyone next time on the blockchain.