Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Episode Title: Fast, Reliable Recovery for Veeam Backups with CloudIBR
16 Plays1 month ago
In this episode, host Tom May and co-host Jonah May sit down with Greg Tellone, CEO of CloudIBR, to explore how CloudIBR is revolutionizing backup and disaster recovery for Veeam environments.
We dive into how CloudIBR delivers ultra-fast, reliable recovery by leveraging its purpose-built platform designed specifically for Veeam users. From air-gapped, ransomware-resilient storage to instant-on virtual recovery environments, Greg breaks down what makes CloudIBR different—and why organizations are turning to it for true business continuity.
Whether you're a Veeam admin, an IT leader, or just curious about how DRaaS (Disaster Recovery as a Service) is evolving, this conversation is packed with insight. Learn how CloudIBR enables recovery in minutes, not days, with built-in testing, compliance reporting, and 24/7 expert support.
🔒 Ransomware-proof backups.
⚡ Instant VM recovery.
☁️ Fully managed, worry-free disaster recovery.
Find out why CloudIBR is becoming the go-to partner for businesses that can’t afford downtime.
Links & Resources:
🌐 cloudibr.com
🌐 differentdev.com
🌐 cyberfortress.com
🌐 rebeldevs.com
Recommended
Transcript
Introduction to Rebel Devs Podcast
00:00:17
Speaker
Rebel Devs podcast, a show that challenges conventional thinking in the tech industry. Each episode features conversations with solutionists who have taken unconventional approaches to problem solving, pushing the boundaries of what's possible in their respective fields.
00:00:38
Speaker
Join the devolution.
Meet the Hosts: Tom and Jonah May
00:00:41
Speaker
Hey, RebelDev listeners, welcome to our latest episode. My name is Tom May, one of our hosts. With me always is my partner in crime, Jonah May.
00:00:50
Speaker
Jonah May, good morning, good day. How's it going? Good morning. It's going. I mean, work is work, and I'm getting ready to bomb out of here later today and head to Florida for a week, so that'll be nice.
00:01:01
Speaker
Nobody feels jealous for you and anywhere way, shape or form.
Special Guest: Greg from Cloud IV Art
00:01:06
Speaker
So we have a special guest as always here. Um, we have Greg from cloud IV art, Greg. Good morning guys. Morning, John.
00:01:13
Speaker
How you guys doing? Pretty good. ah Tell us a little bit about yourself since some of our listeners may not know you.
Greg's Journey in Tech
00:01:20
Speaker
Cool. So have a pretty cool background in disaster recovery and backup.
00:01:26
Speaker
I started in the business when I was about 15 years old. I had a Novell network 2.12 server in my bedroom at my parents' house.
00:01:37
Speaker
And i had lost all my data, all my homework and my gaming scores. So I went out, bought a tape drive. I got a copy of ArcServe at like some trade, you know, it's swap meet or something ah back when things actually came on floppy disks.
00:01:51
Speaker
And I set up ArcServe on my network server. I backed up my data. And then later that year, we got me and my buddy got 10 megabit laser Ethernet beams.
00:02:03
Speaker
We mounted them on the roofs of our parents' houses. And we had a 10 megabit WAN between our two houses. So I was doing off-site backup. as a teenager, which was pretty cool.
00:02:15
Speaker
And then a few years later, when I was 19, I got hired at Cheyenne Software, the original makers of ArcServe. So I did that for a bunch of years, went to McKinsey & Company, and I built out their disaster recovery platform. Back then, it was with IBM up in Sterling Forest, New York.
00:02:33
Speaker
So basically, I've been doing DR my whole life, and we started Continuity Centers 25 years ago, 2000. Wow, that's pretty amazing. I almost blacked out a little bit when you said the word arcsurf because I had so much trauma in my early days of tape jockeying and T4 tapes that never worked.
00:02:53
Speaker
It always worked when you were there testing it and then you'd run it at night. It wouldn't work. And so I don't know. I had to like come back into consciousness there. But. um we've We've kind of moved further away, I think, in the world from ArcServe and whatnot. um Good, bad memories there, but continuity centers. And today we've brought you on to talk about a product that you have, Cloud IBR, which on our podcast, Jonah and I really is just kind of the DR vanguard sort of weird guys.
00:03:21
Speaker
We really just like talking about products that work. So whenever our listeners hear about a product, it's because we use it and we believe in it. And so I think our relationship started... maybe a year, year and a half ago, somewhere in there.
00:03:33
Speaker
And I'd just love for you to tell us a little bit about what cloud IBR is.
Innovation in Disaster Recovery with Cloud IBR
00:03:38
Speaker
i mean, just give us the whole bang. Sure. So it started back in 2020. That's when we started using Veeam at our company and replaced.
00:03:48
Speaker
Ironically, we were using ArcServe D2D at the time. um So my my life had come full circle. I came back to ArcServe, you know, ah two decades later, three decades later. ah So we switched over to Veeam. And the reason we switched was because they were first to market with immutability.
00:04:05
Speaker
So immutable backups before that, everyone was storing things on Windows servers and I had seen backups get either corrupted or ransomware, etc. So we we did a full cutover of every client.
00:04:19
Speaker
It took about a year. we cut everyone over to Beam. That was version 10 when that first came out. And I was actually doing a lot of work with Backblaze. That's also when object storage first came out. So we started working with Backblaze, we did some work with Wasabi, and we were basically doing server offloads of all our customers' data from our vaults to the the customers.
00:04:43
Speaker
And one of the questions that Backblaze asked me is they said, we get a lot of customers who call and say, hey, it's great that we can send you all this data, but how do we make sure it's good? How do we recover from there?
00:04:54
Speaker
And there's manual ways you can do it, but as you know, IT, you're always focusing usually on the production stuff. you know What's down? um What's the squeaky wheel that we have to oil today?
00:05:06
Speaker
What's the next big project we're working on? So a lot of people don't really focus on DR. You know, they they talk about it and it gets brought up once a year and they just don't have the the time or resources to focus on it.
00:05:17
Speaker
So one of the cool things about Veeam is they have a lot of great stuff that you can automate with PowerShell, with REST API calls. ah So we put a team together a few years ago to start writing an application that would stand up servers in the bare metal cloud for you.
00:05:36
Speaker
ah It powers them up. It installs Windows on one. It installs Veeam on that server. It adds your object storage repository ah or your Cloud Connect backup if you're dealing with a partner.
00:05:48
Speaker
ah and it decrypts the backups, and it begins restoring them to a bunch of yeah ESXi hosts. So we basically build out this full infrastructure with yeah the ESXi hosts, firewall, port mappings, OpenVPN, IBSEC tunnels.
00:06:03
Speaker
ah The entire infrastructure you need is completely built out via automation, and recovers all your servers and gives you access to them. that's That's pretty amazing. I know Jonah has a little background at a previous service provider we worked at building his own version of this.
00:06:19
Speaker
Jonah, it was absolutely easy, wasn't it? it just always worked. It was never bugs. I mean, it took us probably a good two years to build it there, and that was back you know before Veeam even had REST APIs. So part of that conversion was you know upgrading it from our shell to REST APIs where we could in version 11 when those were initially added. And Even then, there was a heavy reliance on databases and making sure our own infrastructure was up. And, you know, we also had to go in in and performance tune a lot to get better restore performance. And a lot of it was essentially taking what we had as VMware clusters for replication customers and adding some additional resources to piggyback off of those clusters. So, you know, for a customer who could afford...
00:07:04
Speaker
services like host us hosting their data in house, it made sense. But there was also and I find this issue at my current service provider to a challenge in a lot of the maybe small business and MSP market where they don't necessarily have that budget and they need a lower cost offering, but also don't care as much about recovery times, but still need some sort of off site target.
00:07:25
Speaker
Well, you know from what i went from what I've seen, Cloud IBR kind of perfectly fills that niche with object storage. That definitely does. I know in our world of doing professional services, we get a lot of customers that don't really fit the mold for a um theme cloud service provider. Some do, some don't.
00:07:43
Speaker
um But even if they were with a cloud provider, we're always go to S3, have a tertiary solution and put your backups there. And where we live in is just they go somewhere. And on your worst day, you'll go ahead and pull it back.
00:07:56
Speaker
But alternately, we know that in even the ArcServe days, right, that was even the MCSE test where tests your backups. You know, backup's not good if you don't test it. And when we met Greg, it was, oh, well, now we can hook to these back lays or wasabi backups that are sitting there for pennies on the dollar.
00:08:12
Speaker
But we can't really use them unless we go through this manual process and kind of layering on the idea of like Veeam Recovery Orchestrator. Oh, Veeam goes, it'll go ahead, it'll mount, it'll test, it'll do all these sorts of deals.
00:08:25
Speaker
Well, here comes Cloud IBR. And so now I'm getting excited in my world. I'm going, here's a product that does something that I had designed a long time ago and was hard to hold. Now it's doing it in the S3 buckets.
00:08:37
Speaker
Well, that's great. And then Greg starts showing us some reports. And it was like, wait a minute, we can automate this. It can test us. It can bring us reports. Now, all of a sudden, the auditor side of me gets excited. So when we go through a test, Greg, I mean, tell us kind of what we're doing in that. Because also, I think we should touch upon, you mentioned like OpenVPN in there. i mean, you have connectivity. That was the next thing.
00:09:00
Speaker
How many times even in service providers do these guys spin you into their cloud? And they're like, Look, we failed you over. You can't access it and you can't get back. So I know there's more to it. Tell us a
User-Friendly Disaster Recovery for Small Businesses
00:09:10
Speaker
little bit about that. Like what's our connectivity?
00:09:12
Speaker
How do we fail over those sorts of deals? Sure. ah So the inspiration for all of this comes from doing this for decades. ah When we first started Continuity Centers in 2000, people would bring their tapes to us and I would take their tapes, we'd put them in tape drives, we'd run rest restores, we'd stand up their environment, prove that the tape worked or didn't. They usually didn't.
00:09:35
Speaker
And then we would give them remote access. Back then it was ISDN and T1s. So it was you know much different than the internet we know as it today. and Over the decades of evolution of that, of of setting up the remote access, so all the complexities of it, what we determined you know as we were building CloudLive VR was we need to build something that's easy, that doesn't require any technical know-how because most small and mid-sized businesses usually only have that one internal IT person.
00:10:04
Speaker
And that person is running their IT for the entire company. They know all their applications. They know the infrastructure, but they're not DR experts. you know Disaster recovery isn't what they're accustomed to doing.
00:10:16
Speaker
so And most of those customers that we've been meeting that fall into that, they have an RTO system. before they meet us of several weeks and they're okay with that.
00:10:27
Speaker
And they're not okay with it, but they have to be okay with it because that's all they know. They're going to basically order new equipment. They're going to drop ship it somewhere. Uh, you know, if it's a building type ah issue at their facility, where are we going to ship the equipment, power it up, configure networking, you know, just doing a whole new greenfield environment and then trying to restore their backups.
00:10:47
Speaker
So what we've built is the ability to stand that all you know within an hour. It takes one hour to stand up the entire infrastructure. And then the time after that is how long it takes to restore your data. So ah we typically see about two terabytes an hour on the restore speed. So depending on the customer size, ah your RTO can be anywhere from two hours to usually 24 hours is about the max that we see.
00:11:13
Speaker
um The OpenVPN on the firewall that we built, ah it and you basically add in all of the users that you want to have access into our portal and they will get an encrypted email at time of recovery.
00:11:26
Speaker
ah It has their credentials for the ah firewall. They install the OpenVPN client if they don't already have it. And they just get access into their environment like that. ah We also do IPsec tunnels directly from that same firewall to branch locations, your corporate, all that.
00:11:43
Speaker
And it's just, it's real simple connectivity. No, and I love that that you kind of qualified the RTO in there also. I think a lot of times people have two different perceptions. There's the, I want to buy a service. It better be quick and it better not fail and it better be perfect and it better be tight. And then there's the reality in the middle of a a yeah disaster. They realize they don't care about that. They just want return to service.
00:12:09
Speaker
When we talk about S3, I always think of if that's your primary backup, this is the do-it-yourself or your home service. or you've read something on Reddit, you decide to go into the office, you're like, here's Veeam, let me hook to S3. Oh, great, right?
00:12:23
Speaker
What they don't realize is these time periods to take S3 down, this could be 24 hours manually. This could be 36 hours. When you're talking about bringing up, standing up in an hour, And then you're talking about bringing in connectivity and then two terabytes an hour.
00:12:37
Speaker
I mean, even if we're sitting at four, six hours here for a decent sized environment, the reality is that's light years ahead. But in our architecture, like I chose the word tertiary on purpose because we're already seeding data is center one to two and two to one.
00:12:54
Speaker
Our tertiary solution is everything has gone awry. Like it's just not there. Everything's down wherever. um I'm in an Azure. I'm in an AWS. They go sideways.
00:13:05
Speaker
I have this ability to be over there. I'm in the Veeam data cloud. I have somewhere else I can recover into. So it really brings a lot of sanity and ease in here.
00:13:17
Speaker
Um, but as we start moving over there, um, I think it's probably important to do some education. So you're a guy that's been doing this now. So we're going to call you the expert on your, your application. And because you deal with a lot of the S3 backups, I don't think a lot of people do.
00:13:32
Speaker
I think they set it and forget it, but because you're actually testing it,
Ensuring Data Integrity: The Importance of Health Checks
00:13:36
Speaker
what's some of the things that you've learned or can recommend to the listeners? I mean, what do I do when setting this up to really get it healthy and good?
00:13:44
Speaker
So two ways to do it, you can either do direct to, um actually three ways, direct to object storage if you're not working with a partner, or you can use the partner's object storage through their Cloud Connect, ah or if you're running a sober scale-out backup repository internally, you can do offloads to the object storage cap capacity tier.
00:14:03
Speaker
And... what What we've seen and what's nice about Cloud IBR is it runs monthly by default. Some people run it weekly, ah but every month when that automated recovery runs, it's actually reaching into your bucket and pulling that data out. This isn't a simulation.
00:14:20
Speaker
ah we're not you know It's not just standing up servers. It is actually pulling that data out live from the bucket to make sure it's all readable. and A couple of things that we see ah quite often is Cloud IBR recovery runs, and by default, we skip servers that are older than seven days, backups.
00:14:42
Speaker
So if the backup if the server backup has a run in seven days or more, by default, we'll skip it. Reason being is we're assuming it's um been decommissioned so that we're not restoring all your old servers.
00:14:53
Speaker
And we've had customers come on who've been backing up to whichever object storage it is. They've been doing it for the past year or two since V10 came out.
00:15:04
Speaker
And they sign up with Cloud IBR, they run a recovery, and we skip 100% of their servers. And it says the latest backup is from 2022. And here we are, 2024, 2025, and backups haven't successfully actually been sent to Backblaze or Wasabi or whoever.
00:15:21
Speaker
actually been sent to backblaze or basaia or whoever ah in the past couple of years. Or, and actually, um one of the things I recommend is that you enable health checks on all of your direct-to-object storage backups.
00:15:37
Speaker
We've seen customers who the backup is saying it's running successfully every night, it's sending it to object storage, and it shows there in the dropdown menu ah from the UI of Veeam. But when you go to restore it, it gives an unrecoverable or unreadable error.
00:15:51
Speaker
Because at some point in the chain, one of the backups got corrupted, and every backup after that, even though it says successful, it isn't recoverable.
00:16:04
Speaker
And the way you determine that is you enable health checks. And as soon as you enable health checks, next time it runs within that next day, ah you will see that it says, you know, this backup from this point is unrecoverable, which means everything after it is no good.
00:16:19
Speaker
um So that's super critical is have health checks enabled. i don't believe they are by default as of the latest meme. ah So definitely get that enabled. and make sure you're checking the logs, make sure you have some type of monitoring set up to let you know.
00:16:33
Speaker
And then essentially the only fix at that point is to run an active full backup so that it creates a brand new ah full backup VBK of all those servers. And now every month when CloudLibR runs, you can find out, you know have your backups been 100% successful.
00:16:50
Speaker
No, that's really important. i think everybody skips over the health checks. I always forget to put them on honestly as much as I do it and I have to go back and and do it. It's funny that you mentioned about finding things that weren't backed up. We were working with a demo customer, ah all three of us at one point, and talking about different things with IBR. And I looked at one of my customers and sure as anything, I realized, oh, it ran a test and it hadn't backed up because something was disabled at some point. And so while I don't monitor that customer, I was able to reach out and go, um you know, nothing's going up to the cloud, right?
00:17:23
Speaker
And it was, oh, my goodness. So you're right. It's just there's another fail safe in testing that. Why would I ever think that my job has not run? And all of a sudden this caught it rather than something else.
00:17:35
Speaker
um With our Veeam monitoring, we would catch it because that's what we do. But if some customers for us, they just kind of call us for pro services. don't have insight. All of a sudden now I did just because of the hey, let me show you what Cloud IBR does.
00:17:48
Speaker
um Jonah, you've been using S3 since Veeam first started hooking into it. What are your thoughts of what Greg's saying with health checks and whatnot? What are some of the gotchas or things that you recommend to come people?
00:18:00
Speaker
I mean, the health checks are spot on, even from a, you know, service provider perspective who may still be running Windows or Linux rather than in-house S3. That is something we've always recommended customers enable. You know, anytime you're traversing the public Internet, it increases the likelihood of that sort of corruption. And the newer Veeam releases, I know you can go into some of your advanced job settings, but like encryption health checks and you can save settings as default but yeah that initial setup encryption's not enabled on jobs health checks aren't enabled you know and to me it's long overdue that i really wish fiend would maybe figure out a way to do those by default like maybe on first launch tell you hey here's your default encryption password oh and by the way health checks are enabled monthly because i think those are both extremely long in the tooth
00:18:46
Speaker
Absolutely. Well, I think if trade question we we probably don't, because sometimes people land to data stores that are encrypted on the hardware levels. But you're right, a prompt be a health checks. I've never understood why that's not just turned on in there.
00:19:01
Speaker
That's silly. um yeah It is for Sobers. So for the Scalaback repositories, they have them enabled by default, ah but not for the direct object. um Another great thing that we come across that shocks a lot of the customers ah is their encryption keys. They have it documented.
00:19:18
Speaker
They think they have it documented correctly. And they will run four or five recoveries in a row with Cloud IVR. And our software will tell them in the UI, the encryption key you know couldn' not ah that encryption key could not be used to decrypt these backup jobs.
00:19:34
Speaker
you know You have to add another one. and a lot of them will swear up and down, the other this is the key, this is the key. And one of the cool features with Cloud IBR is you can do a manual recovery. So we have two types, auto shutdown and manual shutdown.
00:19:47
Speaker
The auto shutdown, you can run anytime you want, and by default, it runs once a month, and it... It builds out the physical bare metal cloud. It does all those restores. It does 100% of everything i mentioned earlier.
00:20:00
Speaker
And at the end, it just automatically shuts down all the physical hardware and wipes everything out. The manual shutdown, you it does all of the identical stuff, except it doesn't shut down at the end.
00:20:11
Speaker
It leaves it up running up and running indefinitely so that you, your staff, your colleagues, you can all interact with it. You can test the VPN. You can test the servers, et cetera. So in the case of the encryption key where it's not working, what they'll do is they'll do a manual recovery, log into the VBR that we've built for them, and they'll try to run that, you know, try to decrypt their backups. And they'll we've had some who never could get it to work. So the fix for that is you have to go create a new encryption key.
00:20:41
Speaker
and you create a new encryption key that will force an active full that night for all of those backups. And now they'll use that ah encryption key with con.ob.yark and it will run successful. As long as they don't, because that the original encryption key is stored in the VBR at their site, as long as they don't lose that configuration backup, they'll always be able to restore older backups.
00:21:04
Speaker
ah But should they ever lose that VBR and not have it backed up, um which is a great point, always make sure your VBR is backed up and included in the bucket that you're recovering with Cloud IBR.
00:21:15
Speaker
um And there's actually a few different reasons for that. ah One, you'll have those old encryption keys should you need to you know restore any older data. ah But for fail back, after you've, you know, a lot of customers ask, hey, we failed over to Cloud IBR, we're running there.
00:21:30
Speaker
It's been a week, a month, whatever it is that we're running our production out of the bare metal cloud. How do we get back? um So currently VMware has the the easiest method of getting back.
00:21:41
Speaker
And essentially what we do for you, we have an offering called Platinum Plus where we'll 100% manage this. We'll help our partners with it. um Or you could do it yourself with any version of our software, ah which basically you take your VBR that we recovered for you.
00:21:57
Speaker
and you need to have the IPsec tunnel back to your production network, and you will and your ah sorry you will add the ESXi host that we built for you at Cloud Air IBR into that VBR, and you'll set up replication jobs.
00:22:13
Speaker
and You could do that initial seeding, do hourly replication, snap it severed its snapshots, Veeam calls it replication, but it's basically an hourly snapshot that it sends over and commits back to your production server.
00:22:25
Speaker
So, you Even if the seeding back to your site takes a couple of days because you don't have as much bandwidth as, say, that you know the data centers have, you can still be using your production network at Cloud IBR the entire time, and then you create a failover plan.
00:22:40
Speaker
And Saturday morning, 10 a.m., you right-click on the failover plan, run it now, and it will gracefully shut down all of your guests at Cloud you create it will do that final snapshot over your ah production environment and power them up.
00:22:54
Speaker
So less than and typically less than an hour to fail back. um And that's VMware does that really cool. um For Hyper-V and for Proxmox, Nutanix, all of them, ah typically what you need to do is you need to, well, and you should do this for any environment.
00:23:11
Speaker
Once you fail over to us, this is your new production environment. You want to log into your VBR that we recovered for you and create backups for all of your servers to get them out of cloud IBR, out of the but bare metal cloud, because you need backups if you're going to stay with us for you know any amount of time.
00:23:28
Speaker
So you're backing up all your data to object storage or your Cloud Connect provider. And the the ah won't say it's easy, and this is why we offer it as a service, because it's definitely not easy.
00:23:39
Speaker
But the fastest method of failing back in that environment is you want to have a sober at your facility. So you have your local performance tier and your capacity tier.
00:23:52
Speaker
And let's say you lost all that and you you you built a brand new physical environment. you would create that same server and using the Cloud IVR bucket, the the one that you've been backing up to, and then you can select all of those backups from object storage and re-download them to your capacity tier.
Efficient Data Recovery Strategies
00:24:12
Speaker
So that might take 24, 48 hours to seed all that data into your building. And again, you could still keep using your production environment at Cloud IVR. So you seed your SOB and then the day of the cutover, you're gonna you're going to manually do what a failover plan does. you're going gracefully shut down all your servers.
00:24:31
Speaker
You're going to run that final backup into object storage. Then you go to your Sober, you download those final incrementals, which should be pretty quick because they should be pretty small at that point.
00:24:43
Speaker
And now you can run a local land speed restore from your Sober to your production gear. If you don't have a Sober, ah then you just basically have to shut down ah Cloud IBR and restore everything from object storage, and that could take a while. So Sober definitely speeds that process up.
00:25:03
Speaker
Absolutely. And on the backup configs, the one thing I'll add in that we mentioned it, but we didn't tie it together, it's important to make sure you encrypt those backup configs so that way all those encryption keys will be in there.
00:25:14
Speaker
If you're just doing a regular config backup by default, it's not going to grab that. It's going to grab some job names and whatnot. So always encrypt it. Keep that password off to the side because Greg's right. That's your Hail Mary.
00:25:26
Speaker
Blow away that beam, import in that config, put in that encryption key. Everything is back and you'll be able to work through it. Absolutely. Yeah. And I'd say one of the most important things you just said was keeping the encryption key to the side. Because one thing I see commonly is, you know, customers are self-hosting their own password vault, or maybe they're not using a vault, but they store their encryption key in an Excel spreadsheet on their file server and you know, if your infrastructure's down, how do you get into those systems? You know, you really should be running a USB drive with that encryption password in your break glass accounts or a cloud instance somewhere. Or, you know, even these days too, with newer Veeam, I don't think most customers know it was added in one of the 12 releases. You know, you can auto rotate your encryption passwords and run it all through a KMS, which you could put up in one of the, you know, run out of one of the big three public clouds.
00:26:16
Speaker
Yeah, and I definitely like the USB break glass scenario, like you're saying, like old school key pass or something sitting there on a USB. Two or three key people have those.
00:26:27
Speaker
They're rotated regularly and they're put in fire boxes or whatever. So really, that complete offline copy has always made me kind of go, Greg, you were going to say something and I kind of ran over you. So where were you going?
00:26:40
Speaker
It's funny. So on that topic of rotating encryption keys, we don't have this functionality yet. That's something that we're looking into for in the future is for that customer to be able to reach out and send us those encryption keys and put via REST API and put them into Cloud IBR.
00:26:56
Speaker
Because if you're rotating the encryption key and and something that actually happened to one of our customers last year was their Cloud IBR recovery ran and it failed to decrypt 100% of their backups.
00:27:09
Speaker
Uh, The backups were all there. They were successful. It saw them. We just couldn't decrypt them. So customers said, well, I didn't change it. And what ended up after they did some forensics, they had been hacked.
00:27:19
Speaker
The hacker hadn't launched the ransomware yet, but had breached their Beam server and changed all their encryption. They only have one key. Changed their encryption key. And the reason they do that is they're now waiting for your data to time out. Some of your retention, you know, get it 30 days out.
00:27:35
Speaker
And now they launched the ransomware attack. The backups were immutable, so they wouldn't be able to delete the backups. But by changing the encryption key and then and then launching ransomware, the customer has a choice. I either need to pay the ransom or i need to roll back 30 days, you know or however long that the hacker let that data age out.
00:27:56
Speaker
so ah And Cloud IBR caught it. the The timing, it was about four to five days after the key had been changed is when we couldn't recover any of the servers anymore.
00:28:07
Speaker
So that was a ah real cool thing for the customer to you know, to get that notification. um At first they thought it was, you know, something wrong with the product. And within about a day, they realized the product saved them.
00:28:19
Speaker
So that was cool. So the question, how's I made a hair fit from the new four eyes authorization then too? Yeah. Yeah. Well, don't you tell us a little bit about four eyes and then I was going to actually pepper over and say, do you think enterprise manager with key recovery could help out of that to both of you all? So lot of people don't run enterprise manager, but I like it for key recovery. So Jonah, I'll let you tell us a little bit more about four eyes and and kind of what that works with and being.
00:28:45
Speaker
Yeah, so I believe it was added back in 12.1 back over the summer. So it's getting close to a year old now. So the thought with it is any critical backup or restore procedures, such as, you know, I believe changing immutability period, changing retention, potentially changing encryption key, once it's enabled can no longer be performed just by a single user. So, you know, if a single account, you know, the local admin or a user gets social engineered, if that account was compromised, the hacker can't just go in and change.
00:29:15
Speaker
you know, the encryption key anymore, if they somehow get past two-factor authentication. You know, basically there would be an alert that would be sent out to the other Veeam administrators who would then, someone, one or several people would have to go in and then approve that change.
00:29:30
Speaker
So it's essentially a layer of change management on top of critical Veeam processes. yeah And it's good to do it via distribution group because if it's just two separate email addresses, there's always the chance that two people could get hacked.
00:29:42
Speaker
And now I can issue the the change with account A and approve it with a account B. ah You know, it does bring down the likelihood a lot, but using a distribution group, and it's it's such a great feature because everyone will get that notification that so-and-so tried to make this change.
00:29:58
Speaker
And before you approve it, you want to talk to them. And it's a, you know, pick up the phone, do a Teams meeting, walk over their office, if you're in the office, and actually have a conversation of, hey, why are you doing this? What's the reasoning?
00:30:09
Speaker
um And men better yet is for the person making the change, notify everyone ahead of time. You know, FYI, I'm going to be doing this. This is why we're doing it. ah Because so many, the more security you put in place, the more creative the hackers get.
00:30:23
Speaker
And we just need to be more diligent at that than them in communicating with each other and, you know, trust but verify.
00:30:32
Speaker
That's a good point with the four eyes. A lot of people do overlook that. And Greg, that's right. I would never even have thought using a distribution group, honestly, um because they are going to try. I mean, that's the whole thing. Those of us that have been kicking around forever, we've gone through iterations of DR, right? It was make sure you have a backup because that would get you back.
00:30:52
Speaker
And then it was ransomware and they would attack the backup. So then we put them off site. Well, now they can't touch it. Well, then they realize, well, if your backup application has touches for, ah retentions and whatnot you manipulate the backup system And they just keep getting more creative and people get so frustrated with it that we become so akin to security. And the reality is they don't get paid if you have good backups. So they have to get more creative to get paid.
00:31:19
Speaker
And so really, there's just this blending of the two worlds of security. So that's that's a really good touch point in there. And that's kind of why even in the most secure environments that we're in, we're always like, bring us ah at different dev in for an audit. You might be the world's leading Veeam engineer,
00:31:38
Speaker
Sometimes you just need that second person saying, have you thought about this? And that's where I think a lot of people miss that is. And ah now I'm going to play the cheap card. Ready? And I think it's a little silly, but we're going to go, Greg, I don't really need cloud IBR because I have sure backup.
00:31:55
Speaker
Beam gives it to me. It's in my license. I have some horsepower. Like, come on, man, I'm testing my backups. Why in the world do I need cloud IBR? Sure. so So sure, backup is awesome, but it's only doing those recoveries for the servers in your building that for those local backups.
00:32:16
Speaker
So it's doing it from your Linux or your Windows repository. It doesn't touch your object storage backups as well. And hopefully in your disaster, you get to use those local backups for recovery.
00:32:31
Speaker
There's two main reasons why people aren't able to do that these days. One is a physical disaster. Same thing that's been around for, you know, for forever is if your building is affected, your equipment's affected, it doesn't matter how good your local backups are, they're not available.
00:32:46
Speaker
the of the The newest thing over the past few years is with ransomware. If you're... production network gets encrypted and you're going to call possibly the FBI, the insurance company, a lot of, you know, the insurance is usually the first person they call right after IT.
00:33:02
Speaker
Um, the insurance company won't let you restore over the encrypted servers because they want to get copies of everything. They want to, they want to do forensics on it.
00:33:13
Speaker
And they basically put yellow tape around your data center and say, don't touch. Um, And most companies don't have twice as much storage available that they would need to run side-by-side replica of theirs.
00:33:26
Speaker
Plus, it it could be desktops in your building that are corrupted, there's so ah sorry, that are infected. You don't really want to stand up locally in a lot of cases. You want to stand it up somewhere else that's isolated from your building.
00:33:40
Speaker
on So most like for most ransomware attacks, when you recover with Cloud IBR, don't bring up the IPsec tunnel back to corporate by default. Recover it so that it's in its isolated environment.
00:33:52
Speaker
Only let users who've whose machines have been ah you know ah have been scanned, let them log into VPN to start accessing it. um But yeah, you don't... yeah The Shure backup is great for that.
00:34:06
Speaker
Another good comparison, it was... ah ah Marty Williams at Veeam, two years ago at VeeamOn, when I first showed him the product, he said, wow, this is like VRO Lite.
00:34:19
Speaker
And Veeam Recovery Orchestrator, is an awesome, awesome product. But similar to you brought up Enterprise Manager a couple of minutes ago for ah for resetting your encryption keys or you know retrieving them, that's an awesome product too.
00:34:34
Speaker
But just look at the the first word of that sort a product name, Enterprise Manager. So 80%, I'm going to guess, so because 72% of statistics are made up on the spot.
00:34:46
Speaker
ah So I'm going to guess that 80% of customers using the 80-20 rule of Veeam customers don't use Enterprise Manager. And it probably could be higher than that. ah So they di why don't they run it? Well, it's another server to support. It's another server to upgrade. It's another application.
00:35:03
Speaker
And they're not in enterprise. you know Look at how many people run Veeam Community Edition and don't even have to, yeah they use a free version of Veeam. So enterprise manager is kind of like VRO.
00:35:15
Speaker
Beam Recovery Orchestrator is awesome. You get site A, production, site B, you stand up a bunch of cabinets, your SAN, all your hosts, and you replicate you know either CDP, love CDP.
00:35:27
Speaker
We still sell that. That's a great product doing real-time replication or continuous data protection, ah hourly snapshots with replication. we do all that as part of a DRAS, Disaster Recovery as a Service.
00:35:40
Speaker
But you know you're talking thousands a month to tens of thousands a month between management, staff, colo, bandwidth, hardware. you know It's an expensive endeavor that...
00:35:53
Speaker
20% or less of the world, you know, is actually going to spend that cut type of money. Um, and that's where with Cloud IBR, we start at $149 a month.
Affordable Solutions for Small Businesses
00:36:03
Speaker
So you don't get the same thing as VRO. VRO, you know, you're getting, you know, several minute RTO.
00:36:10
Speaker
Uh, you're getting real-time replication with CDP or hourly snapshots with replication. um But you can do, which is cool, you can do hourly backups to your either sober or direct to object.
00:36:22
Speaker
So a lot of our customers, they used to just do hourly backup, ah sorry, nightly backups. What they're doing now is, because they have Cloud IBR, is they run hourly backups, maybe from 6 a.m. to 6 p.m.
00:36:34
Speaker
Some do it twenty four seven so that when Cloud IBR goes to run a recovery, Your RPO, recovery point objective, might only be an hour or two, depending on when you last backup round. um So yeah Enterprise Manager, VRO, all awesome ah products. we we We sell them, we support them, we love them, but not everyone can afford them. So that's kind of what we built Cloud IBR for was the rest of the world, the people who you know don't have that budget.
00:37:02
Speaker
And I'm about to win it down that with Recovery Orchestrator, because that's where I was going to go next. But even just beyond the cost of it, I mean, you've already got to go up to the highest like tier of EAM licensing, number one. So you're talking incurring a cost there just to run it. But, you know, I've always said Recovery Orchestrator is a great product. It came out around the time that I, or at least started to become mainstream around the time that I built out the internal product.
00:37:25
Speaker
automation set my last service provider for DR testing, but you know, it has a lot of limitations. I say it's a great theory and a bit of a lackluster implementation. Like I wish I would see it grow a little bit because a lot of it under the hood for the backup side of things is built on shore backup and shore replica. So there's no support for Veeam Cloud Connect.
00:37:44
Speaker
It's, I'm sure possible with scripting, but a bit of a pain to actually test anything that's offsite backup. It tends to prefer local backups first, and it is a little limited in the workloads. You can even just recover from it.
00:37:57
Speaker
Yeah, and most people, most companies don't have the bandwidth necessary to be able to do a full restore of object storage in a timely fashion. So that's what's cool about Cloud IBR is you're standing it up in a data center, um you know a tier three data center that has you know ah multiple ah you know dozens or hundreds of gigabytes of internet connectivity, ah depending on which location.
00:38:22
Speaker
And the restore speeds, like I said, we get about two terabytes an hour restore speed ah And Veeam Data Vault, the latest object storage to market, um that actually has been getting the fastest restore speed. So if you haven't, anyone who hasn't checked that out yet, that's a really cool product that Veeam released you know a few months ago.
00:38:43
Speaker
No, that's definitely, I love VRO. The complexities are there. It's great as a professional services partner because we get to go ahead and implement those. and And why wouldn't we?
00:38:55
Speaker
But when it comes to that offsite, I mean, here's the deal. the Even in the VRO documentation, it hooks into Azure and right away they're sitting there and they're kind of doing this whole thing.
00:39:07
Speaker
Well, make sure you have your Azure guru figure out your price points of how much it's going to cost to run because we don't know. We're going to put it there. We're going to run it, but you're going to be stuck with the bill. And that's something I love about your pricing, Greg, is you have multiple tiers in there.
00:39:22
Speaker
But also, I'm not paying for a full instance of Azure where I'm figuring out about cycles and CPU and all that. Can you kind of high level the pricing because it is really sweet?
00:39:33
Speaker
And ah listeners, go to the website because he's got it all there. So go ahead, Greg. Cool. Yeah, it's cloudibr.com. And there are four plan levels, silver, gold, platinum, and platinum plus.
00:39:47
Speaker
ah Silver starts at $149 month if you pay annually month if you pay monthly. um anyone who signs up gets the first 30 days for free.
00:39:57
Speaker
And not only do you get ah usage of Cloud IBR for free, but you actually get unlimited free usage of the bare metal cloud for the first 30 days as well. So there's literally zero spend to onboard, which generally takes about 27 minutes on average. We've seen as long as you know your Veeam environment and how it's set up,
00:40:19
Speaker
you can onboard in less than 30 minutes with Cloud IBR, run your first recovery. All right. Gold, you can get the pricing online, but basically it goes up to $7.49 month Platinum Plus or $9.99 a month if you pay monthly.
00:40:33
Speaker
ah month if you pay monthly And that Platinum Plus includes us being there. It's a human level support versus just software, which is what silver and gold are.
00:40:44
Speaker
You're getting a software subscription, software as a service. You onboard yourself. You run your own test. You fully manage it. ah And that's what a lot of our partners use is they're reselling that to their customers and putting their own services around it.
00:40:59
Speaker
And then a lot of our partners actually also buy the Platinum Plus because they know that yeah know we've got ah dozens of years of experience of just doing DR. That's all our company does. They're backup of DR.
00:41:12
Speaker
So most of our partners offer many different purposes. They're not as beam focused as like you guys are. They are, you know, selling the security, the management of their desktops, their laptops that, you know, they're a traditional MSP.
00:41:27
Speaker
So they bring us in in the background as the DR experts with the Platinum Plus model. um If you don't have Veeam yet, that's one of the requirements to use our product is you need to have Veeam.
00:41:40
Speaker
If you don't have Veeam and you want to give Veeam a shot, um there is a promo code when you're signing up on our site. It's Veeam30, V-E-E-A-M, 3-0, all lowercase. And what that gets you is a 30-day trial license of Veeam software so that you can ah start doing backups to object storage because you can't do that with the community version.
00:42:04
Speaker
So you need a licensed version of Veeam in order to do direct-to-object or to create a server, I believe, as well. No, that that's great that you have the Veeam license in there because you're right. A lot of people are community edition.
00:42:16
Speaker
And I think that the landscape is, I know I need something. I go free. It's writing it. I have an incident. Okay. I have now lost everything, paid ransom or just rebuilt. Hopefully I'm still alive.
00:42:28
Speaker
Now I want to buy Veeam and go. So it's nice that you you have that in there. um As we can start to kind of wind down a little bit, we've thrown a lot out there. I kind of want to get some inside information here, Greg. So what's coming up new in the product? Like um we're here, we're in the U.S., we're talking Veeam.
00:42:48
Speaker
i mean, what sneaky information can you divulge to us? well Where's the product going? Sure. so the the next big thing that we're working on is to bring the product international to dozens of different countries so that you're not just recovering in the US.
00:43:05
Speaker
ah you
Global Expansion Plans
00:43:06
Speaker
know Most other countries have GDPR, et cetera, where they can't you know send their data across international lines. And then actually, ah one other point on pricing, I talked about the four plan levels.
00:43:18
Speaker
There is one other feat, and that is the cost for renting the bare metal cloud. And on our site, if you click on the DR calculator, you basically punch in how much memory you need in your ESXi hosts.
00:43:32
Speaker
ah Do you want a half a terabyte, two terabytes, three, four? And you put in how many terabytes of storage you'll need in your production environment to recover all your servers too. and the On the low end for the smallest clients, it comes out to be about 74 cents an hour for recovery of their environment.
00:43:51
Speaker
On the highest end, the biggest customers are around $18 hour. Most are in the $3 to $5 range. So if your recovery takes six hours to run at $4, you're going to spend an extra $24 a month.
00:44:05
Speaker
So nothing that breaks the bank. um But for what we're talking about internationally, what we're building right now is integration with Google's VMware Cloud and or sorry it's google cloud vmware Engine.
00:44:20
Speaker
and what that is is a full-blown version of VCF, the VMware Cloud Foundation. So you're getting NSX, vSAN, vCenter. You're getting a full-blown VCF environment built out on Google hardware, and we will recover your environment two ah that and to the Google Cloud.
00:44:41
Speaker
So it's not like Azure, what you were mentioning. It's not, you know, it's not ah virtual servers. We're restoring your servers virtually, but restoring them to physical dedicated hardware for you.
00:44:52
Speaker
ah What's really cool about that is the pricing is very, ah easy to determine because they bill per node. So we know, and a node is a host. So we know this is how much the host is per hour. Do I need one host, three hosts, four hosts? um you You can't do two in a in a VCF cloud. it has to be either one or three or more. And that's for redundancy.
00:45:15
Speaker
One is not production ready because now your vSAN is running on all local hardware on a single host. So you you basically can't do anything to the host, reboot it ah So three or more gives you that vSAN redundancy.
00:45:29
Speaker
ah But for Cloud IVR, for your monthly test, one is perfect because you don't need redundancy if you're not going to be accessing the server production. So we keep that really inexpensive with Google by standing up a single node, restore your data there, and everything else that you've seen in Cloud IVR and that you guys have used, it'll all be identical.
00:45:51
Speaker
It'll just be a different physical location. And they have locations, believe it's in the several dozen at this point. I think I counted 30-something the other day ah in over 12 different countries, different regions of different countries. So um that would be really cool for a lot of... Veeam has a very international presence. Just you know you guys, Veeam Vanguard, Veeam 100, Veeam.
00:46:16
Speaker
ah you know, a lot of them are not U.S. based. And a lot of them use our product, which is really cool. ah And that's, you know kind of how we met you guys a couple years ago at VeeMont. Yeah, no, that's awesome because I know I do have international presence with some of my customers. And so we look to put their buckets in their regions.
00:46:33
Speaker
And that has been one of the challenges of how do we as a U.S. based operation, i mean, we all are here, of service these other companies that want what we're doing in our services, and then they're bound by those GDPRs or those data locality.
00:46:48
Speaker
So that's that's going to be really great. I look forward to that because I know we're ready to kind of chomp at that ourselves. um We're going to kind of move into some final thoughts here and I'll let you go last, Greg. So I'm going to jump over to Jonah.
00:47:00
Speaker
You've been using it. You've tested it. You've been cloud service provider, Veeam Vanguard. You're all over the map with Veeam and hackathon leader and whatnot. I mean, what are your takes on cloud IBR or anything we've talked about today?
00:47:14
Speaker
I love Cloud IBR. I actually signed my company up as a reseller just, I think, two weeks ago. We've undergone some new leadership in the last few months. That leadership is trying to kind of, as I said earlier, address figure out how we can address the lower cost parts of the market where we have a gap in our portfolio right now.
00:47:34
Speaker
And the main date is essentially for net new customers and existing customers, where can we start moving some of our workloads to the cloud, either as secondary copies or as lower cost solutions?
00:47:45
Speaker
You know, using the Wasabi's, the Backblazes, the data cloud vaults, and the cloud IBRs of the world. I love it, um Cloud IBR, from the concept of... So I talk a little bit about the tertiary. We are always telling everybody to go to these tertiary spots, these S3 buckets and whatnot.
00:48:03
Speaker
What I really like on the pro services side is um I pitch to my customers, okay, you have multi-sites, you're going up to S3 buckets, but really what we're trying to architect in our world is every one of their sites should have a Cloud IBR instance. So if If everything went sideways or two sites went down, those two sites could be up in the cloud. They could talk and then they can you know go back where they need to. But really the driver that I like is if we look at the Azure side of vrl
00:48:34
Speaker
that pricing is going to be monthly. If I'm going to fail up into there and spin in there, I'm going to get dinged for a lot of that month. It's not going to be those hourly numbers that you're producing over there. So when we're talking, oh my gosh, it's $7 an hour. Let's say it's extreme. It's $7 an hour.
00:48:51
Speaker
You think this is the end of the world. And you're like, well, seven by 24, however many days. And you're like, well, that could be expensive. Well, no, because you didn't buy a data center. You didn't have all your hands over there.
00:49:03
Speaker
You didn't have the total cost of ownership of patching and all these different firmware updates and security teams. And the cost grows exponentially and hardware recycles. So, of course, you just want Amazon or AWS. Well, now you're back at what do I pay?
00:49:18
Speaker
with With Cloud IBR, I can take my customer to their website very transparently and I can say, listen, which do we want? Do we want gold, silver, platinum, platinum plus? Let's run a little calculator.
00:49:29
Speaker
That's your cost. How many sites? Here we go. It's done. And like you said, when it comes to pro services, I probably bill about two hours in there per site to onboard, figuring that there's some intricacy that I'm not quite knowing yet, or if I want to run some tests with them, but it's very upfront to the customer where they understand it's a short ramp. It's on Because I know in our cycles, everybody tends to take too long to sign their professional services agreement.
00:49:56
Speaker
Then they ink the contract and they want it done yesterday. When we bring on an IBR, it's very easy for us to go, okay, well, you've delayed for two months on this thing. We inked it. Now, guess what?
00:50:07
Speaker
You're right. It can be done in a week. And it's very exciting. And there's your report. And now... Somebody literally can answer this because this is the crux when we meet with people. We say Veeam beyond the clicks at Different Dev.
00:50:18
Speaker
When we go beyond the clicks, it's not just I have some software that runs. There's always some executive profusely sweating in a room going, but can I recover? And everybody goes, well, I think so, because I didn't get any errors.
00:50:32
Speaker
You give them this report and it's like, yes, I can recover. Yeah, but is it off site? Yes. Can we access it? Yes. And if you want, you can do the manual test. Like you said, the manual shutdown, they can see it. They can benchmark it. It ticks all those boxes without having do those complexities.
00:50:50
Speaker
Hooks to the Veeam data cloud. It hooks to S3 buckets. Like anywhere you seem to be, you're good. And now with your feature set coming around the globe, now it really does expand out. So those are my kind of thoughts. Greg, you get to kind of close us out before we play some music and stuff. So bring us all around.
00:51:07
Speaker
Pay the bills. Plug yourself. Thoughts. You know, you brought up a great point with the DRAZ and the Colo and doing all that. By the time you ink the deal, by the time you order the hardware, get it all in, and then by the time you actually get it live, you usually new need new hardware at that point because it took so many years to get there.
00:51:27
Speaker
Uh, cloud IVR, you sign up and literally in less than 30 minutes, you can be running your first recovery. And that $7 an hour, even at the highest of, you know, $18 an hour, that would have cost them five, $10,000 month. Uh,
00:51:42
Speaker
ah And if you have a real disaster and you fail over to us and you know you're going to be there for more than a month, you basically can sign up for monthly pricing and get a better rate. ah We've seen a couple of customers who loved it so much, they signed ah three-year deal and they haven't left because it's their new production.
00:51:59
Speaker
um And we'll recover them to a separate facility if their if their dedicated hardware goes yeah has any issues there. um So ah working with partners like you guys, we just had a great quote from ah one of the ah ah legends or sorry, Vanguard who posted, ah he was explaining his experiences on boarding. And one of my favorite sentences was was he said, honestly, it's quicker to do it than to explain it.
00:52:29
Speaker
The onboarding process. And I was like, and that just sums up how, you know, we we make this interface as simple as possible to give you the fastest onboarding, yet the most capabilities on the backend. And we do that based on our decades of experience of here's the questions I need you to answer for me. And I'm going to build out what will work for 99% companies out there.
00:52:52
Speaker
And for the 1% here and there where, where it's not a perfect fit, we work with partners like you guys and we take your feedback. You know, we have a very agile development cycle. Uh, you know, we only have one goal and that's to just keep bringing on new customers and making sure they're recoverable. So when we get feedback from guys like you, from our direct end users, feature requests, you know, if we see something, you know, we get some great ideas from our customers, you know, that's They ask for something simple and it's, yeah, let's get that in. And we get it in within a week or so.
00:53:22
Speaker
all Case in point, you know, Anthony Spateri from Veeam, he reached out to me and said, you know, Veeam Data Vault, it would be awesome if you could get this to work with Cloud IBR. A week later, i messaged him back. I said, it's in production. Check it out.
00:53:36
Speaker
And he asked me on Teams, he said, who were you working with on the inside to get that up and running so quickly? I was like, nothing. We just, no one, we just figured it out. You know, it's just make it work.
00:53:48
Speaker
ah Like Veeam says, it just works. So that's our goal. No, that's great. Well, thank you both for, of course, being here today. My co-host, Jonah May, Greg, Cloud IBR.
00:53:58
Speaker
Listeners, go out there. go ahead and check out CyberFortress. If you want a cloud service provider, you have Cloud IBR for ah your recovery needs as well. They work together. Different dev pro services. Funny, all three of us work together.
00:54:12
Speaker
That's what we all learn about the Veeam environment. We always joke and say you can't say anything bad about anybody because you'll probably work with them or for them. at some point. So ah thank you both for being here. We're going to play some music on our way out and we'll catch you next episode.
00:54:26
Speaker
See you V-Mark.
00:54:31
Speaker
Whether you're seasoned solutionist or just starting out, the Rebel Depths podcast is a must-listen. So join us. Comment, share, and subscribe. Be part of the Rebel Depths.