How do you make a snapshot backup?

You found the backup wrap up your go-to podcast for all things

backup recovery and cyber recovery.

In this episode, we tackle a confusing phrase, snapshot backup.

I know what you're thinking, Curtis.

Isn't that an oxymoron?

Well, that depends on what you mean by snapshot.

We're gonna break down the difference between what I'm calling

traditional storage snapshots and what vendors like AWS call snapshots.

Which are actually more like backups.

We'll talk about copy on write?

Redirect on write.

VMware snapshots that are just weird.

How to turn your snapshots into actual backups that follow the 3, 2, 1 rule.

Uh, pretty much everything that you need to know about making.

A backup snapshot.

If you don't know who I am, I'm w Curtis Preston, AKA, Mr.

Backup, and I've been dealing with this topic for over 30 years, ever since.

I had to tell my boss that there were no backups of the production

database that we had just lost.

I don't want that to happen to me.

I don't want that to happen to you.

That's why I do this.

And so, enjoy this episode of the backup wrap up.

Welcome to the show.

Hi, I am w Curtis Preston, AKA, Mr. Backup, and I have with me my QOS advisor

Prasanna Malaiyandi how's it going?

Prasanna?

I am good Curtis, and I'm glad that we don't have a random five second lag

every time we talk, so That's good.

Yeah.

So, uh, we figured out, we, we were, we started to try to record this

and there was this massive lag.

Um, and, um, I realized that basically like I counted 'em real quick.

There are four people currently watching Netflix.

Uh, I was able to stop one of them, but I don't control the other ones.

And so you came up with the idea of, you know, what about QOS?

And so I, I have a firewall.

Um, and, uh, so I, I enabled SmartQ, I put this website on it and I

said, give me, gimme all the stuff.

Gimme everything I can.

And uh, so now it's, you know, beautiful.

So I guess that's a plug for firewall as well, not a sponsor.

Yep.

Or for any router software that actually supports proper QOS, not like most

Yeah.

consumer level ones, which don't do a great job.

Yeah, so this is, I I've been very happy with the, with

the firewall since I got it.

Uh, every once in a while I use something, you know, fancy on it when this is nice.

So, um, so, um, we haven't recorded in a while.

You, you went away.

I went away.

I went on vacation for the longest vacation I've been

on in seven years I think.

So I was gone out of the country for three weeks, which was nice.

I was off in India for a wedding, visiting family.

Lots of travel.

So much good food.

Curtis, um, and

Yeah.

I might be addicted to caffeine.

So

In India as you've had, right, uh, they serve in the south, they

serve filter coffee, which is sort of a small cup of coffee, which

is very, very concentrated though.

And, uh, I was drinking three or four a day.

Now this is normally someone who drinks maybe one cup of

coffee a day, other than when

Yeah.

office where it's maybe two.

So.

Yeah.

Yeah.

Well anyway, well welcome back to the same time zone.

Very excited.

Thank you.

but It is hard with the 12 and a half hour time zone difference.

You know, it was like

Yeah,

we get a brief window to chit chat, or in the evenings we get a brief window to chit

yeah, yeah.

I basically wake up and call you and then before I go to bed, I call you.

Right.

Um, and, um, uh, yeah, so a lot of people don't know that it's,

that it's 12 and a half hours time difference, which is just odd.

yeah, I don't know why they do the half hour.

Yeah, just funny.

Maybe they, they saved the half hour to drink the filter coffee.

So, uh, all right, well, today we're gonna talk about, uh, you know, and I

thought, you know, you thought maybe we'd covered this already, or at least

recently, and, and we really haven't.

So this is a, a word that comes up a lot in, uh, in the show.

And that is this word snapshot, which.

If, if we, if we go outside of it, it, it, it means that, you

know, it's, it's a picture, right?

That that's what the word snapshot and also.

There's another word that's gonna be used in this episode, which is image, which

is really just another word for picture.

But in our world, an image is very different than a snapshot.

And as you've often heard me say, words mean things.

And um, I want to make sure that this is sort of a, I'm gonna

call it snapshots basic episode.

So if you're, if you're just wondering what you know, what are snapshots, what

are the different types of snapshots?

Both what I'm gonna call the traditional snapshot, which is the ones that you and

I, you know, grew up on, if you will, and then something that is relatively new.

Again, comparatively speaking, new, uh, that is also being called

snapshots, which is actually something completely different.

So, um, uh, and then what purpose they serve, uh, et

cetera, et cetera, et cetera.

Any, any other sort of introductory things you can think of before we get started?

no.

I also did like your analogy of a snapshot is like a picture

that you would take right

Yeah.

of like tech, right?

And,

Yeah.

thing to add is that's important is it captures a moment in time.

Yes.

Right.

A

One moment in time.

Sorry,

and

go ahead.

but, so we're not referring to a video or whatever Apple calls

their eight second or six second

Yeah,

that they send over.

Right.

That's not what we're referring to.

It's a single

Yeah.

Yeah.

So first let's talk about what I'm gonna call the traditional snapshot

and, um, which I, another word, it's funny now that I think about it.

This other word that I'm gonna use is also sort of like, um, it's

also a word that is used outside.

Id to also mean.

You know, looking at a thing, right?

And, uh, the word that I'm gonna use is view.

Mm-hmm.

So, uh, if anybody has any experience in databases.

Right.

There is this concept called a view, which is a different view into the database,

which gives you, it looks different than, um, than the, the, the entire database.

Right.

It is a, it is a view, a a particular point of view, if

you will, into the database.

I don't want to go too much into that for two reasons.

One is not really relevant.

Two, I will probably mess up the description, but, um.

Because it's one of those areas where it's right at the edge of my, of my experience.

I am not a DBA, um, but I've often had to pretend to be one

in order to back up databases.

But this goes to your concept or your, the, the point that you were

making is that a snapshot, uh, is a view of the storage at a particular

point in time, so it is a view of your typically file system, but also

it could just be a volume, right?

A virtual volume at a particular point in time and what your, when

you look at that snapshot, the actual blocks that you are looking at, or

you're copying or you're accessing.

What you are looking at.

Is going to like where the blocks come from is going to be based on how the

snapshot is actually being delivered.

Sometimes you are going to be looking at the blocks from

the original storage device.

Sometimes you're going to be looking at blocks from some

sort of cash snapshot area.

Mm-hmm.

Um, and so this view that you're looking at is this sort of virtual

view of a particular point in time and it just keeps track of everything.

And it says, okay, at this point in time they go to ask for this block.

It knows what was where.

So it goes, at this point in time, block A was.

You know, at this status, well block A is still that status over on the original

storage device, so we're gonna go get it.

Or if you ask for block B, you go, block B was this status, and we can

see that that status has changed.

And so we're gonna get that block from the snapshot area.

Um,

And

go ahead.

does a different implementation.

Yeah.

think most of.

As of what I know right now is they're all kind of, all the blocks

are shared, and like you said, as things change, new copies are created

that contain the updated version.

But there's all this metadata and mapping, all this stuff is what

the storage vendors, that's really like what they focus on, right?

Is how do they make it fast?

How do they make sure that you have that plausible point in time view to look at.

Yeah.

And, uh, we're gonna talk about, uh, like a couple of different

ways that this actually happens.

Uh, but the, but when you think about the way I described how that works,

the important thing to understand is I said that if you grab blocks, one

part of the, you know, one set of blocks where you're going to get those

blocks from is from the original.

Thing, right?

The, the device, the volume, et cetera.

And the other way is you're gonna get blocks from the snapshot area, but

the, but the thing that you should infer from that is that in order for

the snapshot to function, you need the original device to continue to be

Mm-hmm.

functional and et cetera, et cetera, et cetera.

Yeah.

Yeah.

Um,

thing to

yeah.

that's important for snapshots Curtis

Mm-hmm.

is that snapshots are read only, right?

Which is a critical fundamental, uh, the, the property of a snapshot, right?

Just like a

Yes.

you take a picture, you're not modifying that picture,

Right.

Good point, good point.

Um, and because otherwise it wouldn't be a snapshot of that point in time.

Now you can take a snapshot and you can make it read, right?

But at that point, it's really no longer a snapshot, right?

Um, it is a completely different sort of function.

Yeah, and typically for those ones do have that snapshot, and then

normally when you make it read, write, you're creating a new entity.

So the snapshot still exists, but then the all the writes go

Correct,

entity.

correct.

Correct.

Um, so there are.

Two different ways that snapshots, again, traditional snapshots are typically done.

And there's what I'm gonna call, um, sort of the the traditional Traditional, okay.

And that is this concept.

And this is pre NetApp, basically right before NetApp started doing snapshots.

'cause NetApp really kind of reinvented snapshots and, and how they work, right?

Before NetApp, you had what was called the copy on write Snapshot and where

it gets, or CCOW copy on write Snapshot and where it gets that, that name is

that you, you have a block, right?

When, when you make the snapshot at the very beginning of making the snapshot,

all blocks are in the original device.

Because nothing has changed.

The moment you go to do something, you go to update, you know, a block.

The question is what happens now?

So in a copy on write situation, the um.

We're going to, before we change that block, we're going to copy that block out

to the snapshot area so that later when we go to, um, access that block, we say, oh,

we need the block from this point in time.

That block, at that point in time, is only available over

in the snapshot area, right.

What's important to understand is that when you go to update a, a block

with a copy on write, um, snapshot, it has to do a right of that block.

It has to move the data, then copy, you know, write that.

Old block in the snapshot area, and then it's gonna write the

new version of that block.

So there are three IO operations for every right operation

in a copy on write snapshot.

And then the more snapshots you have.

And the more, and the longer you keep them, the more, uh, of your blocks

that you're going to have to copy every time you, uh, write new data.

Is that, how did I do with that explanation?

yeah, you did.

Well, um, the other thing also is at some point you end up with a

Yeah.

As well, just because of how writes are being done.

And remember all the time, these aren't just small blocks, right?

So even if you go modify, say a hundred bytes in a block, you still

have to write that entire block back,

Yeah.

Yeah.

sort of the underlying building block for the file system.

And

Right.

this could lead to a lot of wasted reads and writes and IO that you're consuming.

Are you, did you just say that the block is the basic building block?

It is funny, the words that we use, right?

Um, yeah.

And so, um, what's it, I, I think one of the most important things to understand

about a copy on write Snapshot set up is that over time, if you have a lot of

snapshots and you keep those snapshots for a significant amount of time.

The performance of your primary array because it's having to do all of this

copying, you know, on, you know, on write.

The performance of that, both the read performance and the, and

especially the write performance can be significantly degraded over time.

Yeah, and I'm just thinking, don't know any systems to my

Mm-hmm.

that still use copy on write.

I am sure there are some, but I, but I would agree with you that

most newer storage vendors realize sort of the evil of copy on write.

You know, the, the, you know, it's, it's, I. It's, it's, it's sort of like

the famous Maya Angelou quote, right?

Like, we did what we did when we knew what we knew, but now we know

different and we do different.

Right?

Uh, that's a, that's a massive, uh, I'm sure misquote of, but I think I

got, I think I got the concept there.

And I remember being at a large, very large oil and gas company

and we were, um, you know.

Helping them do a, it was an, it was a RFP, right?

Um, for, for just a massive, uh, storage change.

Right.

And, um, they knew how many snapshots they wanted to create, and they knew

how long they wanted to create them.

They were already a, a NetApp customer.

The, the problem with being a NetApp customer was they knew every.

Bad thing about NetApp, right?

They, they, they could see right through the stuff that they said that was bs.

Right.

You know, every vendor has got some amount of bs, but they were a customer,

so they knew everything that NetApp, you know, they knew every blemish about

NetApp, but they, and they were bringing in other vendors to see if perhaps they

could get what they get from NetApp.

Uh, while changing the vendor.

And one of the things that they want is they wanted 90 days of user

browsable snapshots, and they had all this data to show how much money

that that was saving them because of the number of user generated

restorers that were happening, right?

Because one of the great things about having snapshots is that if you give

them access, your users can just go to the right area and they can see their

directory from a different point in time.

Right?

And so they, they had all this data that showed that they had all these user.

Generated restores and um, uh, and so they said we want 90 days

of user browsable snapshots.

I won't name the vendor, but let's just say it's a fundamental law of physics.

Um.

That vendor said, uh, if you do that, you know, you're, you're, you're, 'cause

they were, you know, they were doing copy on write snapshots at the time, and

they basically said, you're going to have a significant performance degradation.

And they, and they asked them like, how bad, and they just sort of, you know.

Spitball the number, and they said it was 50%, like a 50% performance

degradation, but they're like, we actually don't know because no one does

that, no one does that with our storage.

Right.

Um, and, um, uh, yeah, so it's, it's a really big deal.

Right.

So that is,

Yeah.

I. Copy on.

Right?

Which is the old school way of doing snapshots.

And if you're currently having any problems with your snapshots,

maybe you investigate and see if they're copy on write?

That's gonna be your core problem, right?

Um, and, and it's gonna fall under the category of

doctorate hurts when I do this.

Well, don't do that.

You need to do fewer snapshots or whatever.

I just realized there is one vendor that still does copy on.

write.

And do you know what that is?

Who?

Microsoft with VSS Snapshots?

VSS.

The volume shadow services.

Yeah.

Yeah.

Yeah.

I was just thinking about that.

I'm like, there has to be someone, and I was like, oh, yep.

Yeah.

And, and by the way, that brings up a, a really good, uh, thing that maybe

we should have covered earlier on.

And why do you, why do you make snapshots?

Right?

There's two reasons.

And VSS is the second reason.

The first reason is that it acts as sort of, kind of like a backup

Mm-hmm.

that it gives you this view, this place that you can go get and get the file

the way it looked three weeks ago, three months ago, six months ago, whatever.

However long you keep your snapshot.

Go ahead.

can we call it little B backup?

Because, because here, here's my

I what you're saying?

Yeah.

gets, because it doesn't follow the traditional

Yeah.

backup of 3, 2, 1 rule and everything

Yeah, that's why I'm saying it's kind of like a backup, right?

Yeah.

Um, it, it's not really a backup until you copy that snapshot

to some other location, right?

So, and it also needs to be managed and all that kind of stuff,

and you need to know what's in it and all that kind of thing.

Anyway, but to, to follow the 3, 2, 1 rule, you need to at least

copy it to another location.

Hopefully that location is offsite and now we got three

copies and two locations we got.

Okay.

So, um, actually, if all you're doing is.

Snapshot of replication, you're actually not following 3, 2, 1 either because

it's not, you don't follow the two, you don't have, you know, 'cause the two is

meant to be two different risk profiles.

And if you, if you go from one filer to another filer and you're, yeah.

Okay.

Um.

The second reason that you make a snapshot and VSS falls under this

typically, and that is what you're doing, is you're creating a stable

point in time that is now read only that you can then use to back up.

Mm-hmm.

You can point your backup.

And most backup vendors that are, that support windows, they integrate with VSS.

And what VSS do?

You, you, you talk to VSS and, , basically, you know, quick

summary there, there's, there's this concept of a, of a VSS writer.

You talk to the VSS and you say, Hey, I'm here to do a backup.

What kind of, um, you know, what kind of stuff do you have?

And they're like, we have SQL Server.

You're like, okay, do a snapshot for SQL Server, and then you, and then

that gives you a, a stable point in time to back up this, this device that

otherwise is moving around, right?

So it, it gives you a read.

Only copy that isn't changing while you're backing it up.

And then if you're just there to do a backup, it actually immediately

deletes the snapshot as soon as you're done with the backup.

Right.

So that's the second reason that we use a, a snapshot, which is

just to create a stable point in time from which we're gonna run.

I'll call it a big B backup to use your, to use your, your terminology.

All right.

So that's copy on write?

Uh, do you wanna describe, redirect on write.

Yeah.

So this is basically what NetApp does

Yeah.

Although they would say no 'cause they say there's a slightly

different, but that's okay.

You know?

Yeah.

NetApp is probably most famous for this with its write anywhere file layout

Yeah.

right?

Which basically says when you're going to actually do your writes, you don't

need to overwrite the existing block.

Right.

gonna pick any other random block, and then you're just

gonna keep track of where it is.

So all new writes go to all new blocks, so you don't have to ever

worry about going and updating all the metadata and copying data out, and so

you're efficient in your write code.

Pap and managing snapshots becomes a lot easier.

And so the where the term redirect on write, is instead of overriding the

block in its current location, you're just gonna redirect that, write?

Put it in another location, and then your snapshot manager just needs to keep

track of where the old location was.

So the big difference between a redirect on write, so there's a,

there's a plus and minus here.

The plus is much better performance,

Yep.

right?

Which is why.

Pretty much most vendors have gone with that as the newer.

Um, the downside is snapshot data and production data all

in the same volume, right?

There's no snapshot area.

It's, it's everything.

So if you keep snapshots too long, you can actually fill up your, your

volume, um, and have a problem.

Yeah.

And they, that's why they do try to do things like setting up a

snapshot reserve space so you can automatically start deleting it

once it gets to certain too high.

But yeah, there are cases where that could also still be problematic.

So,

Yeah.

And, and this is why, because you can do snapshots essentially.

With impunity, you can do as many snapshots as you want, as often

as you want, without, um, and, and keeping them as long as you

want without any zero performance.

Issues as long as you keep enough storage around for your snapshots, right?

Uh, 'cause then we start getting low on blocks and we start, you know,

then we, that, that, that, that, that's a totally different problem.

So as long as you have enough space for your, uh, for everything,

then you shouldn't have any performance from redirect on write?

Snapshots.

Yep.

And I think with it now, I think the latest NetApp snapshot limit

is I think a thousand snapshots.

It used to be 2 55 for a while, but

Yeah, the,

a

the famous 2 55 and yeah.

Yep.

It's funny how that number just sort of, well, 2 56 really is the number,

but 2 55 is, we can't do 2 56.

We can do two.

Well, it's 2 55 because active is always zero.

Oh, there you go.

There you go.

Yeah.

Yeah.

Um, so that's redirect on write.

And that, you know, and we, we really talked about it very quickly, but

basically it means that you can have as many snapshots as you want, as long as you

want without any performance degradation.

So that's generally what you find in most common, uh, snapshot providers right now.

other thing I wanna also mention, copy on write?

Versus redirect on write.

Mm-hmm.

discs, it made a huge benefit For Flash.

It makes an even bigger impact with Flash there are issues around

right amplifications when you're

Mm-hmm.

um, as well as how many wear cycles you can have on your flash.

And so redirect on write is actually much better, both from a performance

and longevity perspective for Flash

Right.

was on disc.

So

So the prob the problem there is that you're worried that you're having

too many writes to one cell, and each individual cell in, um, flash has.

A limited number of times that it can be overwritten, right?

And so the worry that you have is if you have too many writes to the same block,

you actually end, uh, to the same cell.

You actually end up, that cell can become, uh, no longer usable

but basically each cell has a finite number of times that you

can overwrite that cell and you hit that and that cell is done.

And now you know, it's like a bad block on a, or a bad sector on a disc, and

you gotta move on to the next sector.

and, and one other thing to mention too is I know so far when we talk about

writes, people are probably thinking, oh, you're just writing a block of data.

But remember with that block, you have a bunch of metadata

associated with it, right?

Indirect blocks that point to other blocks that contain metadata and everything else.

That all needs to be updated by a file system.

So even just doing like a hundred byte right,

Mm-hmm.

require, say.

20 other blocks to be updated as

Right, right.

Exactly.

Yeah.

So those, so those are the two sort of what I'd call traditional snapshots.

Right.

Um, and now, and the second one is now what I would call

sort of the standard snapshot.

yeah.

is a third.

Really weird way that his snapshots are done, which I'm

just gonna cover really quick.

And that is VMware.

So they do a, um, I, I believe it's a copy on write, but what's real

actually, actually they do a, I'm gonna call it a weird on write?

So what they do when you write a new block.

They don't write the block to the, and and by the way, this is my understanding

of the way VMware's, um, snapshots work.

If I am now wrong, please tell me, but this is, I've checked repeatedly and this.

I believe is still the case.

So what would happen is when you take a snapshot in VMware, usually for backup

purposes, um, when you go to write a new block in VMware, it doesn't write

the new block in the production area.

It writes the new block over in the snapshot area, leaves the

original version of the block on the disc, and then it stays that way.

Until you delete the snapshot, in which case it updates all the blocks to the new,

Yep.

thing.

And, and, which is why, and, and I, I administered VMware for, for

a while at least not really in a production environment, but, but

you know, in a lab for a while before I realized this was the case.

And so it means, what that really means in practice is you really can't use.

At all.

You can't use VMware snapshots the way you can use NetApp snapshots

or even copy on write snapshots because the performance is way worse.

Right.

Uh, at some point you go to delete snapshots and you think deleting

snapshots would be a nice thing, except when you delete the snapshot, there's

this flurry of IO activity to update the primary volume with any of those blocks

that are now the, the current block.

It's the messiest thing I've ever seen.

So two things to add.

One is for VMware, it's okay if you keep those snapshots around.

For a short duration.

For a short duration.

Yeah.

for a short

Yeah.

it's fine, but don't think about using it for like that 90 day user restore use

Mm-hmm.

that you were mentioning earlier,

it for a day or so.

Right.

And then delete it.

Right.

Make that a normal part of your thing.

Exactly.

And then the second point is with all of these snapshots, VSS VMware, most

of them support two modes of operation.

One is sort of a software based snapshot approach where they're

dealing with all of this.

So like Curtis mentioned with VMware, right?

They're keeping track of everything.

The second thing, second way that they also support it, is they

integrate with the storage vendors.

So when you go say VMware, go take a snapshot.

VMware will do some things and then it'll call into the underlying storage

provider and say, okay, now go do a hardware snapshot and lock in that data.

In which case then you don't have all of these issues that Curtis just

Exactly.

That's a really good point.

Thanks for, uh, bringing that up.

Right.

So VSS and VMware and maybe other sort of software level snapshots, they

do have this concept of talking to a storage array and then putting that

storage array, making, making, letting the storage array do the hard work.

Right.

Which is, which is.

I, I think certainly the preferred way to do snapshots is to do it in, in hardware.

Yeah.

. So the final way of snapshots, and this is the snapshots that most of you

know, because most of you are very, you are your head's in the clouds, right?

And that is, and, and I'm, I blame Jeff Bezos, um, because it

was Amazon, I believe that first started using the term snapshot.

To actually refer to something completely different.

If you go into a, you know, if you go to a number, it used to just be

like EBS, the Elastic Block storage.

Right.

Um, that, or you take, uh, a snapshot of a EC2 E image.

Yeah.

RDS Right.

You know.

If you take a snapshot, as I made quotes in the air, it doesn't do

anything like what we just said.

It creates what I would call an image copy, right?

Of that thing that you just took a snapshot of and it creates a

basically bite for bite copy of that.

And typically, I think it's, I'll just speak in terms of a, you know, AWS,

it creates that bite for bite copy in.

S3, right?

It stores it in a special reserve area in S3, and so when you create a snapshot

in pretty much anything in Amazon, what you're actually creating is a backup.

You're creating an image copy of that device.

You're putting it in S3.

Um, you can even, um, you can often specify where an S3, right, so you

can specify that you want that backup to go to another region, for example.

Um, and you can also copy these snapshots around, um, and, um, yeah,

But you can't access it directly via S3,

correct, correct.

Like, yeah, that's why I was saying it's like a special, like reserve area.

NS three.

Yeah.

it's

That's for, that's for a lot of safe safety reasons, right?

Yeah.

It's called a snapshot, but it honors a lot of our backup

with a big B requirements.

Yeah, because it's an actual copy, you can specify that it be sent to a, to

another region, to another account.

Right?

And that's what I wanted.

I want it in another account.

I want it in another region.

I don't want you to just do an EBS snapshot, for example, to the

same availability zone, uh, that.

Isn't, doesn't really follow the basic 3, 2, 1 rule, which is

something we talk about a lot.

Yeah.

But I do, I do think it's important to understand that a snapshot in,

in A AWS is so much, most, so much closer to what I would call a backup.

In fact, depending on how you do it, it definitely is a backup

than what we typically talk about when we say snapshots.

Right.

And, and it's interesting because there are vendors that have made

a, a deal of, you know, they're like snapshots are not backup.

And they've been saying that for so long that they continue to say that.

When what we're really talking about is AWS when it really is

a snap, it really is a backup.

it's interesting you bring this point up because think other vendors, other

traditional storage vendors are also now starting to go the same route

where they're saying our snapshots can now be offloaded into object storage.

Right.

Yes.

I, yeah, I have seen some of the, some vendors do that.

Yeah.

Right, which is very similar to what AWS has been doing, but I think

they still call them snapshots.

Right?

So

Yeah.

of this, you have to, I think it's important as administrator, an IT

person to really ask the question.

When someone says snapshot, what do they mean?

Yeah.

Right.

I'm gonna, and we had, we used the phrase snapshot backup in this,

um, in the title of this episode.

And this is a snapshot backup, right?

So it, it's some would see snapshot backup as like a, like a oxymoron,

like, like military intelligence, you know, um, that, you know,

two words that can't go together.

But if by snapshot you mean what they do in AWS and I. Similar, uh,

you know, other cloud vendors, then I'm fine with calling that a backup.

Just make sure, so the question is, how do you make a snapshot backup?

Well, there's two ways.

One is if you're talking the traditional type of, you know, storage snapshots,

you need to copy that snapshot to another, uh, storage array.

Hopefully one that's offsite.

Um, hopefully one, you know, my dream would be.

Is to another vendor or you, then you, you, you change the

form of the snapshot, right?

You know, you're doing, you're doing, you know, filer to filer and then you're,

you're backing that up to some other, uh.

Object store

system, right?

Yeah.

might do NDMP, who, you know somewhere, uh, Steven, uh,

his, his ears are perking up.

Um, the, um, uh, anyway, but if it's, but if it's, if it's a cloud vendor,

typically, and again, I can really only speak with, with authority and

AWS, but from what I've seen, the other cloud vendors are very similar.

When you take a snapshot in AWS, you're actually taking a backup.

And when you're doing that.

You can use AWS backup to sort of control all of this, which is AWS backup is

really just a control plane for all the other stuff that's going on, and

you can make sure that you know that you're following the 3, 2, 1 rules.

You have a different account, different availability zone, different region even.

Um, yes, it makes it cost more, but it actually makes it a real backup.

So, questions for you.

Yeah.

So you mentioned that sort of doing snapshots, replicating it from

one filer to another filer doesn't quite meet the rules of the two.

Right?

In 3, 2, 1, backup.

Uh, in the 3, 2, 1 rule.

What if so many of these vendors now allow you to run virtualized

instances in public clouds?

Mm-hmm.

So I could run a NetApp instance in AWS

Yeah.

My answer is no.

I.

because it's still the same software version.

Yeah.

Okay.

Yeah.

Now AWS

it's better.

Better, you know, good, better, best.

It's better than the other thing because at least it

isolates you from the hardware.

But it's still, the os still the same os.

And would you also consider the same like AWS offers file systems, right?

So they have like support for NetApp by FSX.

Would you consider replicate if it's, I'm not even sure

Yeah, I don't, I don't know.

I don't know if I can speak to that.

Right.

I don't know if I can speak to that.

Right.

And then, and then

do you have to be for the two is my question.

I, I, again, you're, you're just trying to, it, it's sort of a

good, better, best thing, right?

The, the, the more different you can be, the better you are.

It's difficult in this cloud world, right.

Um, the, um, and some would say, well, isn't a w aren't AWS snapshots the same?

And it's like.

They're actually a little bit different because it is a full image

copy, um, and it's stored in S3.

If something were to happen to your EBS volume because of a bug in EBS,

you could still use that snapshot, which is stored in a completely

different system, which wouldn't have the same bug and store that in.

You could restore that into EBS.

that your volume wasn't corrupted by some EBS bug

Well, but you're, but S3 isn't EBS.

no.

But when

Oh, you mean, you mean before?

Well, if it was created, if it was created, if it was corrupted

by the EBSB, then you know Yeah.

You're, that's gonna happen no matter what you do.

So, so the I, I like where you're going, Curtis.

I think there's also sort of the practicality aspect that we have to

also look out for, because ideally, like you said, right vendor, you

have multiple vendors, right?

You're using different technologies, but then you have to balance that with

pr, sort of the practicality, right?

Are you gonna be able to understand all these various technologies,

build out the skillsets, integrate it, pay for it, right?

We all know how small backup budgets are versus production budgets.

Yeah.

Yeah, yeah,

just.

And, and I've always tried, you know, throughout my career to try to be.

To say, look, if you're asking me my opinion, you know I'm

giving, I'm giving you the bar.

Right, right.

And you know, and I understand that people have to, they have to, they have

to live in the real world where, where backup is not the most important thing.

Ah, it hurts me to say that.

Um, right.

Um,

but at the same time, maybe there are certain use cases where

you can focus on it because of the high value of the data or

Yeah.

like that, where it is important to have

Yeah,

type solution.

also want to add, you know, we talk a lot about the 3, 2, 1 rule, and I

say that because the 3, 2, 1, like if it doesn't follow the 3, 2, 1 rule

and it's just not a backup, right?

I'll say the number one.

Number two, I think I'd be remiss to say that also I. You know,

we're starting to talk about things like the 3, 3, 2, 1, 1 0.

You know, you do want at least one of your backups, your copies

to be on immutable storage.

Right.

To be on truly immutable storage.

One that even you can't delete even if you want to.

Right.

Um, because only then is that backup, uh, protected against a ransomware attack.

Yeah.

Or a, or a, or a direct bad actor.

You know, a, um, what's the, what do, what do we call it?

Um.

threat.

Thank you.

You know?

No, there's a. Uh, uh, a rogue admin, right?

Uh, you know, a rogue admin, either a rogue admin, right?

'cause there have been rogue admin stories, you know, uh, but also someone

who is able to gain access to your administrative account and then go

and delete all your backups you need.

Only if even you can't delete it, even if you want to, is a backup, truly immutable.

Um, and so you can do that with snapshots.

You just need to copy them into something else.

Yeah.

All right.

This was fun.

Yay.

Welcome back, Curtis.

Yeah.

And our, our lag wasn't too bad on our thing.

Hopefully our QOS did its job and hopefully our recording, uh, works.

And with that, that is a wrap.