April 27, 2026

Stop Using VSS as a Backup Before Ransomware Deletes Your Shadow Copies

Stop Using VSS as a Backup Before Ransomware Deletes Your Shadow Copies
Apple Podcasts podcast player badge
Spotify podcast player badge
Castro podcast player badge
RSS Feed podcast player badge
Apple Podcasts podcast player iconSpotify podcast player iconCastro podcast player iconRSS Feed podcast player icon

Stop Using VSS as a Backup Before Ransomware Deletes Your Shadow Copies

Ransomware deletes shadow copies using your own built-in Windows tools against you — and if VSS was your backup plan, you just found out the hard way that it wasn't. In this episode, W. Curtis Preston (Mr. Backup), Prasanna Malaiyandi, and Dr. Mike Saylor break down exactly what shadow copies are, why they don't qualify as a real backup, and how attackers are weaponizing vssadmin to wipe your recovery options before you even know you're under attack.

If you've got Windows systems and you've been thinking "eh, we've got shadow copies," this episode is for you. We cover the history of VSS — what it was actually designed for, why it became a crutch, and why using it as your primary backup strategy is a bad idea on multiple levels. Performance, the 3-2-1 rule, and the fact that one attacker with admin rights can delete every single copy in seconds. We also get into the living off the land angle: how attackers do recon on your shadow copies, how they use them to scope out valuable data before going full ransomware, and what you can actually do to detect and respond to this behavior using EDR tools.

The bottom line: VSS is a great tool. It was just never meant to be your backup. Get a real one.

Chapters:

0:00 — Intro

1:39 — Welcome & Book Talk

3:26 — What Are Shadow Copies and Why Do People Use Them as Backups?

9:14 — Performance Problems with VSS as a Backup

10:19 — Living Off the Land: How Ransomware Uses VSS Against You

12:36 — Can You Monitor or Lock Down VSS Admin?

14:26 — Why Shadow Copies Fail the 3-2-1 Rule (They're Not a Backup)

18:01 — How to Protect Yourself: Configuring Your EDR

21:31 — The Local Admin Problem and Security Culture

27:00 — Virtualization, Snapshots, and Shadow Copies

29:00 — Final Thoughts: Just Don't Do That

Speaker:

You found the backup wrap up your go-to podcast for all things

Speaker:

backup recovery and cyber recovery.

Speaker:

In this episode, we're talking about something that makes me

Speaker:

want to flip tables upside down.

Speaker:

People using Windows shadow copies or VSS as their actual backup system, and

Speaker:

then ransomware comes along and then deletes those shadow copies using your

Speaker:

own built-in Windows tools against you.

Speaker:

And suddenly you have nothing.

Speaker:

I'm joined again by Dr. Mike Sailor, our intrepid cybersecurity

Speaker:

expert and persona ndi of course.

Speaker:

And we break down exactly why shadow copies are not a backup,

Speaker:

how attackers are pulling off this move, and what you can do to protect

Speaker:

yourself if you've been counting on VSS to save you when things go wrong.

Speaker:

This is your episode.

Speaker:

Uh, just a quick note about me.

Speaker:

I'm w Curtis Preston, AKA Mr. Backup, and I've been obsessing about backup recovery

Speaker:

and now cyber recovery for over 30 years.

Speaker:

If that's your bag, then I'm your guy.

Speaker:

Well, you're not gonna find anybody that cares about it more than me.

Speaker:

Ever since 1993 when I had to tell my boss there were no backups of

Speaker:

the database that we had just lost.

Speaker:

Now I've written five books, a blog, and a podcast.

Speaker:

And here we turn, unappreciated Admins and the Cyber Recovery Heroes.

Speaker:

This is the backup wrap up.

Speaker:

Welcome to the backup wrap up.

Speaker:

I'm your host, W. Curtis Preston, AKA, and Mr. Backup.

Speaker:

And I have with me a guy who couldn't bother to on time.

Speaker:

Prasanna Malaiyandi, how's it going?

Speaker:

Prasanna.

Speaker:

You've heard about Indian standard time, right?

Speaker:

I am not making any comments about anybody's stuff based on

Speaker:

where they may or may not from

Speaker:

so I do have an excuse because I had to change into my shirt,

Speaker:

which Curtis, I see that.

Speaker:

shirt.

Speaker:

I should get one of those.

Speaker:

Yeah.

Speaker:

I think you have to just find yours is more what the.

Speaker:

to God, I have turned this house upside down looking for that shirt that I

Speaker:

paid good money for, I cannot find it.

Speaker:

anyway, including yesterday's event where I literally took

Speaker:

everything out of my closet.

Speaker:

have no excuse, but maybe it's being used as a cleaning rug somewhere.

Speaker:

Anyway.

Speaker:

Of course have our intrepid cybersecurity expert, Dr. Mike Saylor.

Speaker:

How's it going?

Speaker:

Mike,

Speaker:

It is going well guys.

Speaker:

Thanks for having me.

Speaker:

have you still have you yet?

Speaker:

I have not.

Speaker:

Have you seen our book yet?

Speaker:

I have not.

Speaker:

It's killing me.

Speaker:

So we're of course referring to this book up here on my video, which if you're

Speaker:

not watching us on YouTube, feel free.

Speaker:

the same, this channel by the same name, the backup wrap up, and,

Speaker:

learning ransomware response and recovery, available now, at, all the

Speaker:

places that you buy books, although

Speaker:

is.

Speaker:

the.

Speaker:

All the places, if you prefer Rainforest in South America, or

Speaker:

if you prefer buying them directly from O'Reilly, or Barnes and Noble.

Speaker:

I was in a Barnes and Noble yesterday.

Speaker:

apparently it's a building in which you can buy books.

Speaker:

It's a fascinating.

Speaker:

Fascinating idea.

Speaker:

maybe that'll take off.

Speaker:

Anyway, so today, we're talking about, something a little bit up my alley.

Speaker:

And I'm gonna just start out, imagine, if you will, you have developed a backup.

Speaker:

I'm gonna make quotes in here, a backup system, because I don't

Speaker:

think this is a backup system.

Speaker:

You have developed a backup system around something called shadow copies,

Speaker:

which is a ver a feature in Windows that we're gonna talk about in a minute.

Speaker:

And then you just come in and you've heard that something bad has happened

Speaker:

from some sort of cyber attack perspective, and then you find out

Speaker:

that, you know that those previous shadow copies are not available to you.

Speaker:

I like the, this reminds me of, for those of you that have been around a minute,

Speaker:

this is the ultimate sweep, the leg move, that is of course a karate kid.

Speaker:

Reference.

Speaker:

and this also refers back to living off the Land, when we talked about living

Speaker:

off the land a few episodes ago where they use your own tools against you.

Speaker:

In this case, the tool would be VSS, admin, and they just delete this thing

Speaker:

upon which you are counting for recovery.

Speaker:

this sounds like a bad day.

Speaker:

What do you think, Mike?

Speaker:

I agree.

Speaker:

Yeah.

Speaker:

I think one of the problems, that people have is they become complacent.

Speaker:

And I think that's, it's not just with your backups.

Speaker:

I think in general with security.

Speaker:

but for sure, whoever set up your backup strategy, whether that was you

Speaker:

or your grandkid, or a volunteer or even someone been in it for a while,

Speaker:

shadow copies have been around.

Speaker:

and they come with, they come with the batteries, but they're, to, to

Speaker:

your point, they're not, they're not a legit backup solution.

Speaker:

Yeah,

Speaker:

A VSS volume, shadow services, AKA, shadow copies, and the, is a service

Speaker:

within Windows where you're able to create a stable, consistent, image

Speaker:

through which you can view the platform.

Speaker:

Let me go back in time.

Speaker:

There was a time when, VSS did not exist, and when you went to go back up.

Speaker:

Windows.

Speaker:

the problem is you're backing up.

Speaker:

When you have a large system, you're gonna, part of the backups

Speaker:

are gonna be from this point in time, and part of your backups are

Speaker:

gonna be from that point in time.

Speaker:

And if the thing that you're backing up needs what we call referential

Speaker:

integrity, meaning integrity between the different points of the system.

Speaker:

You're not gonna have ref referential integrity and

Speaker:

you're gonna have a bad backup.

Speaker:

And there used to be a product, I don't know.

Speaker:

person, do you remember St.

Speaker:

Bernard?

Speaker:

Is that before your time?

Speaker:

That's.

Speaker:

Or my time.

Speaker:

Okay.

Speaker:

Mike, do you remember St. Bernard.

Speaker:

I don't.

Speaker:

Man, this is okay.

Speaker:

By the way, St. Bernard was what you bought something before VSS existed.

Speaker:

And for the record, St. Bernard was a San Diego company.

Speaker:

and, it came from the idea of the, that St. Bernard's, the image of St.

Speaker:

Bernard's with a thing of whiskey around the, they came to save you.

Speaker:

So that's where the Saint Bernard name came from.

Speaker:

The, so VSS was this method through which you could create a stable

Speaker:

image, and do various things with it.

Speaker:

So Prasanna, why don't you talk about like the first, what,

Speaker:

VSS was typically used for.

Speaker:

Yeah, Because one of the things that you need to do with a Windows system,

Speaker:

or if you're running say Microsoft SQL or Exchange is do your backup.

Speaker:

And especially on physical machines, before we had

Speaker:

virtualization and everything else.

Speaker:

The only way to actually get a stable point from which to

Speaker:

do your backup was to use VSS.

Speaker:

And so you'd have the backup apps call into and VSS.

Speaker:

It's a framework that Microsoft provides.

Speaker:

So you'd call in, you'd say, Hey, I wanna do a backup of this particular app.

Speaker:

Microsoft would then.

Speaker:

quiesce

Speaker:

the application.

Speaker:

Take a volume snapshot, right?

Speaker:

Shadow snapshot.

Speaker:

And then it would unes, so then you can continue doing your stuff, right?

Speaker:

And then what the backup app would do is it would then read from

Speaker:

that shadow snapshot in order to be able to do the actual backups.

Speaker:

And so this is what it was useful for, and it was more of

Speaker:

like a software snap snapshot implementation, pre VMware snapshots.

Speaker:

but yes, it was their mechanism to get a consistent point for which

Speaker:

you could do your backups from.

Speaker:

And there, like you said, it was a framework, right?

Speaker:

So each application that wanted to be supported by VSS, they wrote

Speaker:

what was called a VSS writer, and then VSS would, and then they just

Speaker:

needed to plug into that world.

Speaker:

And like you said, a backup app would just say, Hey, I'm here to do a backup.

Speaker:

It would talk to V-S-S-V-S-S would talk to the writers.

Speaker:

Everybody would do the right thing that worked for their application.

Speaker:

Then we would take the snapshot.

Speaker:

it's great for that, right?

Speaker:

It is the easiest way to get a backup of SQL Server or Exchange or any, or

Speaker:

even Oracle running on top of Windows.

Speaker:

Just use the VSS infrastructure.

Speaker:

but some people, Mike, have you seen this in the while where people use

Speaker:

VSS, like as a backup mechanism?

Speaker:

Have you ever actually seen that you have.

Speaker:

I have.

Speaker:

Yep.

Speaker:

You're killing me.

Speaker:

What?

Speaker:

any you?

Speaker:

Why?

Speaker:

Why?

Speaker:

Not that he's killing me, but,

Speaker:

No?

Speaker:

oh, you're saying why?

Speaker:

Why would they

Speaker:

Yeah.

Speaker:

Yeah.

Speaker:

And what was the purpose?

Speaker:

Bike.

Speaker:

Could you elaborate a little?

Speaker:

I think it's just in the absence of an educated solution.

Speaker:

so they, they tried to understand the manual or did a Google search or, saw the

Speaker:

word, Or the idea that it was a backup of some sort, and they just went with that,

Speaker:

Because you could create multiple shadow copies over time, right?

Speaker:

and you could hold onto them.

Speaker:

These are,

Speaker:

a problem.

Speaker:

which is also a problem.

Speaker:

It's a problem for multiple reasons, right?

Speaker:

Prasanna, I'm pretty sure these are copy on write.

Speaker:

Snapshots, right?

Speaker:

yes.

Speaker:

Yeah, For those that don't know that there are multiple reasons why using

Speaker:

VSS as a backup app are a bad idea.

Speaker:

We're gonna primarily talk about one of them in the episode, but one of the

Speaker:

other ones is if you've got 30 days of VSS snapshots, your performance is in the

Speaker:

crapper, because of the way copy on write.

Speaker:

Snapshots work.

Speaker:

I'll see if I can find them.

Speaker:

We have a, we have an episode where we just.

Speaker:

Delve into, copy on write, snapshots and what the differences between

Speaker:

them and redirect on write snapshots.

Speaker:

but because of the way VSS works underneath you are going to significantly

Speaker:

hurt the performance of your system if you're using it as a backup app.

Speaker:

go ahead, Prasanna.

Speaker:

one thing I do wanna.

Speaker:

Clarify is my knowledge of VSS is slightly dated.

Speaker:

They may have improved things over the years, but

Speaker:

I doubt that they changed fundamental architecture from copy

Speaker:

on write to redirect on write.

Speaker:

You agree?

Speaker:

I would agree.

Speaker:

I, think it's highly unlikely, but today.

Speaker:

We're talking about the cyber aspects of this.

Speaker:

so Mike, why don't you talk, let me just look at the outline we have here.

Speaker:

Mike, you wanna, you want to, just give a, again, a brief overview of

Speaker:

the concept of living off the land and what, how that A applies here.

Speaker:

Sure, yeah.

Speaker:

Living off the land is a bad guy's strategy of utilizing data,

Speaker:

applications and trust, already associated with a compromised host.

Speaker:

So if I. I'm able to get access to any degree, to a laptop or a server,

Speaker:

what things are already there that I can utilize to further my attack,

Speaker:

whether it's escalating my privileges or collecting data, or moving from

Speaker:

this computer to another computer.

Speaker:

those are the types of living off the land.

Speaker:

techniques or tactics a bad guy would use.

Speaker:

So one of the first things that they will do, and this is the first phase in

Speaker:

just about any attack, is reconnaissance.

Speaker:

So I've got access to something.

Speaker:

What is it and what do I have access to?

Speaker:

what's here?

Speaker:

What tools do I have?

Speaker:

one of the things that they will look for are shadow

Speaker:

copies, for a couple of reasons.

Speaker:

One is that potentially leverage?

Speaker:

for extortion or something else.

Speaker:

and two, can I collect that data and potentially use that as information,

Speaker:

to further understand the target?

Speaker:

maybe it's, I've found out that this shadow copy is of, QuickBooks.

Speaker:

so now there might be some financial information there, or, maybe there's

Speaker:

a company name or a project name.

Speaker:

something that's gonna make, the light bulb go off and help me determine

Speaker:

if this is a valuable target.

Speaker:

and then whatever my next steps might be.

Speaker:

Go ahead.

Speaker:

When you're talking about that, I'm actually not sure what's done today, but

Speaker:

when an application is reading from a shadow copy, does that get logged in all

Speaker:

the other normal processes or is this sort of something that's hidden away and so

Speaker:

the normal mechanisms don't necessarily apply and so it might be harder to figure

Speaker:

out if someone is exfiltrating data.

Speaker:

No, you can actually monitor for, not only the store, access to where the

Speaker:

files, the data is, but you can also, monitor for the processes that would.

Speaker:

interact with the shadow copies, the, vs admin as an example.

Speaker:

so yeah, those are all things that you can write detections for.

Speaker:

and whether that's like anti-malware on the device or, much broader

Speaker:

like network monitoring, cybersecurity monitoring platform.

Speaker:

Yeah.

Speaker:

the challenge though, and again, this goes back to the, to the, the nature of living

Speaker:

off the land is that the tools in question are commonly used by other tools, right?

Speaker:

VSS admin and w and WMIC, which is a, the Windows management,

Speaker:

interface, on the command line.

Speaker:

that's one of the challenges of the, you can monitor for it, but you might

Speaker:

end up getting a lot of false positives.

Speaker:

the, sorry.

Speaker:

just looking at here.

Speaker:

Ask another question while you're looking.

Speaker:

Mike, sorry, I'm, I've never actually dug into, is it vs.

Speaker:

Admin or VSS Admin, by the way.

Speaker:

Two S's.

Speaker:

Okay.

Speaker:

For VSS admin, I've never dug into it.

Speaker:

Can you access control VSS admin?

Speaker:

I know you talked about you can monitor it for what files are being accessed, what

Speaker:

processes can you also lock down who has access to it from a process perspective,

Speaker:

or is that not really possible?

Speaker:

That's a good question.

Speaker:

I know you have to have a, you have to be a local admin.

Speaker:

but I don't know if.

Speaker:

a scripted like PowerShell, could do it, remotely or, that's a good question,

Speaker:

Okay,

Speaker:

but

Speaker:

but Okay.

Speaker:

But yeah, once you have local admin, then all bets are off right at that point.

Speaker:

Yep.

Speaker:

So there are ways to use, again, just so we're not anti snapshot, I

Speaker:

am definitely anti any kind of backup system that is easily deletable.

Speaker:

And this is one of them.

Speaker:

There are ways to use snapshots and there are ways to use snapshots,

Speaker:

perhaps storage array, snapshots, for example, that can be done in such

Speaker:

a way that they're not deletable.

Speaker:

basically if you get it to a place, if you get it to a version of the

Speaker:

system that is, that is where you have worm turned on, that would be one.

Speaker:

Write once read many, right?

Speaker:

There are, again, not anti snapshot.

Speaker:

Just this particular snapshot being used as a backup method.

Speaker:

go ahead.

Speaker:

I wanna challenge you is based on your definition, Curtis, of what we've been

Speaker:

talking about for the last six years, is that shadow copy, even a backup,

Speaker:

It's not,

Speaker:

okay?

Speaker:

that,

Speaker:

Yeah.

Speaker:

So what and why do you think it's, why do you think, I think it's not a backup.

Speaker:

it doesn't follow your favorite rule in the world.

Speaker:

it does not follow my favorite rule in the world.

Speaker:

which of course would be the 3, 2, 1, which is now the 3, 2, 1, 1 0, but

Speaker:

basically three copies of the data on two different media, one of which is offsite.

Speaker:

Arguably the way I understand Peter Krogh, the person who originally coined

Speaker:

this term, is the idea, he would not think of three versions as three copies.

Speaker:

He would think of three actual copies, right?

Speaker:

And then he would want, and those need to be on separate pieces of media.

Speaker:

The two, the idea there is that you have it on different types of media.

Speaker:

So that you have a disc drive and an SSD or an SSD and a tape or a disc, whatever.

Speaker:

Cloud.

Speaker:

that's what the two was meant to.

Speaker:

Basically different risk profiles.

Speaker:

And then the one definitely needs to be somewhere else offsite would be the term

Speaker:

that we would typically be used, right?

Speaker:

It's a difficult term in the cloud, but, but.

Speaker:

The, yeah, you're right.

Speaker:

this isn't any of the, this doesn't meet any of the three, parts of

Speaker:

the definition of the 3, 2, 1 rule.

Speaker:

So yeah, I would not consider this a backup, but based

Speaker:

system can absolutely do that.

Speaker:

yeah, so again, not anti snapshot, just anti using them like this.

Speaker:

All right, Mike, you go ahead.

Speaker:

What.

Speaker:

But even if you are using a snapshot system, so I know as some of my

Speaker:

past employers, they would integrate with the VSS framework to then

Speaker:

trigger a hardware based snapshot because that was better performant.

Speaker:

And then they would do their backup off of the hardware based snapshot.

Speaker:

Yes,

Speaker:

question though, is.

Speaker:

I guess it depends how that snapshot is created.

Speaker:

And is it possible to use the same VSS framework to manage those snapshots?

Speaker:

Because I'm sure that those backup software has a retention

Speaker:

life cycle that it manages.

Speaker:

And so is there a risk that those snapshots could be also deleted if

Speaker:

someone gains access to that Windows box?

Speaker:

I don't think the way VSS admin work is it's designed to

Speaker:

control third party snapshots.

Speaker:

I think it's only the designed to work the other way.

Speaker:

I could be wrong.

Speaker:

but regardless of how off host snapshots are created, they need to

Speaker:

eventually be somewhere that is worm.

Speaker:

And if they're not, then I'm not a fan.

Speaker:

and again, I'm not, it's not, I don't have a problem with VSS.

Speaker:

I think VSS is a great tool.

Speaker:

Use it for what it was designed for, which is to give you a stable,

Speaker:

consistent image to create a backup from.

Speaker:

so let's talk about.

Speaker:

What people can do to, protect against this.

Speaker:

And I, Mike, you've already alluded to this one, is this idea of,

Speaker:

complaining, have your EDR tools look at.

Speaker:

Things like vss, admin so let's talk about the first thing here,

Speaker:

this idea of, give us a little bit more information on the configuring

Speaker:

your EDR to trigger on this stuff.

Speaker:

It's a process.

Speaker:

And what I mean by that is, you've gotta your tools and your people to

Speaker:

understand what's normal, so that.

Speaker:

A normal process like this running in an abnormal fashion would

Speaker:

be what you're detecting on.

Speaker:

so with any good security tool or security service, security

Speaker:

person, there's gonna be a lot of getting to know you the beginning.

Speaker:

so you know, the VSS admin, kicking off.

Speaker:

Twice a week, once a day, whatever the case is, those are gonna show

Speaker:

up, as a privileged, activity.

Speaker:

and so when we're getting to know our clients, we want to be very strict at

Speaker:

first so that we can understand all of those things and then we start to

Speaker:

flag those, or, build an understanding.

Speaker:

we build a wiki, that says these things happen on this.

Speaker:

on this frequency or in this manner.

Speaker:

so that going forward as the, as we detect those things, we can determine

Speaker:

if those are normal or not, or normal with some context like this will

Speaker:

happen, but so will these other things.

Speaker:

but if any of those normal things happen on their own, that in

Speaker:

itself is an anomaly or abnormal.

Speaker:

so creating a baseline and getting to know your, your environment is important.

Speaker:

because again, living, living off the land would indicate that it's stuff that's

Speaker:

already there and that should be okay.

Speaker:

Mike, then, I know you've brought this up, in previous episodes as well, about

Speaker:

okay, you will need to have someone come in, understand the environment, Kind of

Speaker:

fine tune it for your particular use case.

Speaker:

It's not like a. outta the box, everything is good to go.

Speaker:

For you, how long does that process normally take?

Speaker:

And I'm sure it's gonna vary significantly depending on the size of

Speaker:

the organization, the number of devices, the applications, all the rest of that.

Speaker:

But is this like something that'll take three years?

Speaker:

It depends.

Speaker:

Yeah.

Speaker:

what's interesting about that is, identifying the deficiencies or

Speaker:

opportunities for improvement, actually happens fairly quickly.

Speaker:

It's the remediation and the follow up that tends to take time.

Speaker:

as an example, and I'll preface all of this with the adoption

Speaker:

of security in most companies.

Speaker:

Has to be aligned with the culture or the compliance requirements.

Speaker:

other words, if I'm a, if I'm a company that doesn't like security, but I

Speaker:

want to do business in a regulated environment, I have to adopt security.

Speaker:

how we adopt it, how we implement it is a little different.

Speaker:

and may, it may take some time, but there are some companies out there,

Speaker:

organizations that don't like security or their culture is security averse.

Speaker:

We're too friendly to have security type of thing.

Speaker:

or, it's keeping me from doing my job, so we're not gonna do it.

Speaker:

there are a lot of, if not every organization has one or two or

Speaker:

many, Very blatant, observable, deficiencies, even with just best

Speaker:

practice, basic cyber hygiene.

Speaker:

One of those is allowing users to be local admins on their computer.

Speaker:

back in the day was very prevalent because it needed to be in order

Speaker:

to, keep the overhead on already constrained IT support staff you have.

Speaker:

let users do some of the basic stuff.

Speaker:

I remember when one of the new versions of Windows came out and you had to be

Speaker:

a local admin to change your wallpaper.

Speaker:

that was ridiculous.

Speaker:

And, but it was culturally people were causing a big stink because

Speaker:

they couldn't customize their computers and they were unhappy.

Speaker:

And so it got overburdened with these local ad, these,

Speaker:

basic local admin requests.

Speaker:

So everybody got local admin.

Speaker:

today, we still see that to a large degree, people, users have local admin,

Speaker:

just because, because they, there's an exception because they're an engineer or

Speaker:

a designer and their tools require it.

Speaker:

And there are better ways of going about that.

Speaker:

Sure.

Speaker:

Give them access to the local account, but don't let them run

Speaker:

day to day as a local admin.

Speaker:

run as administrator, not, Click on the thing and do run as administrator.

Speaker:

don't log in as an administrator.

Speaker:

So there's things, the more things you implement, the more burden

Speaker:

you're introducing into people's workflows and how they do their job.

Speaker:

And so the easy thing to do is just.

Speaker:

Give them local admin.

Speaker:

So there's that.

Speaker:

but even in the, in those cases as a security, cybersecurity practitioner,

Speaker:

so whether I'm teaching a tool to tell me what's weird or I'm monitoring an

Speaker:

environment for what's weird, if I knew that about an organization, if I knew

Speaker:

that somebody was a local admin, I could still, with that understanding detect

Speaker:

anomalies, occurring as local admin.

Speaker:

There are things that computers do, does bad guys do, that just

Speaker:

are not normal user behavior.

Speaker:

Yeah, the challenge I did a quick, quick look and it doesn't appear

Speaker:

that with VSS admin that basically anyone with administrator rights can

Speaker:

run it, and that I can't say, oh.

Speaker:

Only Steve.

Speaker:

Only Steve.

Speaker:

Or only administrator.

Speaker:

Only you know the base administrator of account.

Speaker:

It doesn't appear that I'm able to restrict it by administrator, and so

Speaker:

let's go back to your comment, Mike.

Speaker:

I think we can agree that.

Speaker:

The proper thing is to have an administrator account and not have

Speaker:

everybody have admin privileges.

Speaker:

But to go back to your earlier, thing, and by the way, this is the

Speaker:

same in, in, in Unix world, right?

Speaker:

Is that you have root and then you have, we, and you use SDU

Speaker:

to, to run things as root.

Speaker:

The same thing here, you have administrator.

Speaker:

Then you have Curtis and Curtis maybe an administrator, but doesn't

Speaker:

have administrator rights, and so I need to run it as administrator.

Speaker:

but to go back to your earlier comment, I couldn't agree more.

Speaker:

And it's the same in the backup world.

Speaker:

If your culture is not one of cybersecurity.

Speaker:

they're just gonna fight you the whole time, and I'm sure you've

Speaker:

been in situations where the culture is the problem, right?

Speaker:

I can think of a company, I can think of a, oh, this is a great story.

Speaker:

There's this company.

Speaker:

on the East Coast and I used to work for a big consulting company and they had this

Speaker:

particular company, and this is a company that literally everybody would know this.

Speaker:

they actually make things.

Speaker:

they're the rarest of all companies is an American company

Speaker:

who actually makes things.

Speaker:

you have these things in your house, guaranteed.

Speaker:

And we went to, they were having all these problems with their backups and I, I was

Speaker:

asked to come in as the big guns and to come in and basically tell them, 'cause

Speaker:

they were trying to get things done.

Speaker:

And then the organization was fighting them and I came in to give

Speaker:

a, presentation, which basically the title of the presentation was,

Speaker:

are the 10 things that you can do to screw up your backup system.

Speaker:

And congratulations.

Speaker:

You guys have managed to do every single one of them, But there was a person,

Speaker:

there was a person that, that had power.

Speaker:

Who was 100% against everything we were trying to do.

Speaker:

She just, and when you have that, when you have a person, whether they're high

Speaker:

up in the organization or down low in the organization, if they've got influence

Speaker:

and they're actively fighting against you there, there's just not much you could do.

Speaker:

And I'm sure you've.

Speaker:

been in that situation, Mike, where you're trying to do something like this

Speaker:

where it's like you shouldn't be using your VSS snapshots as backups and you

Speaker:

really shouldn't, and you should also be monitoring who's running VSS admin.

Speaker:

What.

Speaker:

What, by the way, lemme just stop for a second.

Speaker:

If you ag, if you follow the suggestion from the first part of the call, which

Speaker:

is don't use VSS admin as a method of doing snapshots, this concern becomes

Speaker:

much, much less important, right?

Speaker:

Because the worst you could do with VSS admin, if you're not using it for backup,

Speaker:

if you're not using them, as a backup method, the worst thing you could do with

Speaker:

VSS admin is create a snapshot, right?

Speaker:

Um, so.

Speaker:

Um, no, but there's no snapshots to delete if you're not holding onto your

Speaker:

snapshots right then who's running VSS admin becomes much less important.

Speaker:

okay.

Speaker:

Let's move on to

Speaker:

so Mike, I know we talked about okay, running VSS admin

Speaker:

within the Windows box is bad.

Speaker:

Keeping it long term is bad.

Speaker:

we talked earlier about hardware snapshots are good.

Speaker:

Could you talk about from a cybersecurity perspective, how this kind of

Speaker:

might change with virtualization?

Speaker:

And if virtualization and VSS lead to less cybersecurity concerns.

Speaker:

I know you talked earlier about like auditing and looking for

Speaker:

anomalies and that sort of thing.

Speaker:

Huh?

Speaker:

Is that a loaded question?

Speaker:

Is that a hard.

Speaker:

no.

Speaker:

so on a virtual machine or a virtual host with multiple virtual

Speaker:

machines, Again, it depends.

Speaker:

It depends on what the bad guys were able to compromise.

Speaker:

If I compromise the host, then I've got access to all the virtual machines.

Speaker:

If all those virtual machines are doing their own snapshots, I now have

Speaker:

access to all of those snapshots, because if I've compromised the

Speaker:

host, I'm also privileged on this box across all those virtual machines.

Speaker:

So there's that.

Speaker:

one of the things I do want to add though, even though the majority of, VSS.

Speaker:

Process and, data require admin privileges as a normal user on the machine, I

Speaker:

can still read those shadow copies.

Speaker:

So even if I'm a bad guy and have, I'm not admin yet, I can still do recon into

Speaker:

those shadow copies to see what it is I have access to and potentially exfiltrate

Speaker:

that leverage in an extortion attempt.

Speaker:

Because it's already packaged and compressed, right?

Speaker:

I don't have to redo that.

Speaker:

it's a nice little nugget, that I found.

Speaker:

But in a virtual environment, just as virtualization makes things more

Speaker:

efficient for us, it also makes things more efficient for bad guys.

Speaker:

And back to a comment.

Speaker:

or a thought, thread that Curtis was on?

Speaker:

As far as culture goes, I've seen it both ways.

Speaker:

I've seen where majority of a user base drives culture.

Speaker:

so even the leadership team may say, we need security, but then

Speaker:

you've got this workforce that says, we will not use security.

Speaker:

especially in a service or organization, where they're out, your user base is out

Speaker:

delivering something and they're rebelling against you, you will often cave to them

Speaker:

unless it's truly regulatory required.

Speaker:

and then I've definitely seen the opposite where leadership always wants the,

Speaker:

they're gonna push it down to everybody else, but they've got the exception.

Speaker:

And then everybody sees that, or maybe they become aware of it and

Speaker:

they're like, Leadership's not doing it, I'm not gonna do it.

Speaker:

setting the bad example type of thing.

Speaker:

Yeah.

Speaker:

And this is one of those, I, I've been in environments where, I've

Speaker:

got people in power that are telling me that raid is backup, right?

Speaker:

We don't need backups.

Speaker:

We're we have raid, and then they're actively lobbying against you.

Speaker:

That's the situation where like you need to just walk away, and, let them leave.

Speaker:

Go ahead.

Speaker:

there, there is an example of raid that is acceptable, but you have

Speaker:

to unplug it and put it on a shelf.

Speaker:

Okay, that's not just saying, not what we're talking about here.

Speaker:

Mike.

Speaker:

You're killing me.

Speaker:

am just saying it.

Speaker:

I've seen it.

Speaker:

They've plugged in a storage array.

Speaker:

They've done a backup.

Speaker:

They unplugged it and put it back on the shelf.

Speaker:

That's not braid, that's, you know what I'm saying?

Speaker:

Dang it.

Speaker:

don't, you're trying to get my, trying to get my blood pressure up.

Speaker:

It's

Speaker:

so this is a little bit depressing of a podcast episode, but is the takeaway that

Speaker:

VSS is evil and no one should use VSS.

Speaker:

just don't use VSS as a backup.

Speaker:

That's literally, we could, this could this episode could have

Speaker:

been two minutes long if you're using VSS as a method of backup.

Speaker:

Not to be confused with using it to make your backup.

Speaker:

I, don't you, you understand what, I don't know you're using it to facilitate

Speaker:

whatever actual backups you're doing.

Speaker:

In other words, the way we just, where we talked about where we said, Where

Speaker:

you're creating a stable, consistent image that you're then creating an actual

Speaker:

snapshot of or an actual backup of.

Speaker:

I have no problem with VSS if you're using VSS to create multiple shadow copies

Speaker:

you're storing those on your system, it's a bad idea for multiple reasons.

Speaker:

We've talked about performance.

Speaker:

Mike talked about using them as a method to get historical information, and it's

Speaker:

also bad because all it takes is one.

Speaker:

Bad, one black hat with VSS admin rights, and then they just delete 'em all.

Speaker:

So it's just bad.

Speaker:

So forget all the cyber stuff.

Speaker:

We said, just stop.

Speaker:

Just don't do this.

Speaker:

get a real, go get a real, there, remember there, there was a, I used

Speaker:

to have this, I don't know, it's probably bigger in my life than either

Speaker:

of yours, but there used to be this Dilbert comic strip that we had, and

Speaker:

it was, it was the, where the guy says.

Speaker:

There's a guy with suspenders in the beard he is oh, I recognize you.

Speaker:

And he says, beard and whatever, and then smug expression on your face.

Speaker:

You're a Unix admin.

Speaker:

And then the guy flips him a quarter, and he says, here's a quarter kid.

Speaker:

Go buy yourself a real computer.

Speaker:

That's the way I feel about people that are using their VSS, as backups.

Speaker:

okay.

Speaker:

I don't know.

Speaker:

Do we have any final thoughts on this?

Speaker:

So

Speaker:

I think you just said it.

Speaker:

go ahead, Mike.

Speaker:

VSS is often necessary as the guys talked about earlier.

Speaker:

it, it helps you, quote unquote lock files so that they can be prepared for.

Speaker:

a backup.

Speaker:

but that backup is some other solution.

Speaker:

a lot of backup solutions today do that for you.

Speaker:

they put an agent on your machine.

Speaker:

It does the, it does this function, in the absence of that capability, you're use,

Speaker:

you're using some other backup solution.

Speaker:

VSS is still some, is still often necessary, to prepare

Speaker:

your data to be backed up.

Speaker:

and then the last thing I'll add from a detection perspective is again, if I

Speaker:

knew you were doing backups every night or certain days of the week or once a

Speaker:

week, and that's often scheduled, right?

Speaker:

It's either a task or, a task on the local machine or a scheduled,

Speaker:

activity across the network.

Speaker:

If I knew that from a cybersecurity perspective, and then I saw.

Speaker:

VSS or even some other privileged, backup related account, doing things

Speaker:

outside that schedule, that anomaly, would be what I would respond to.

Speaker:

Absolutely.

Speaker:

Yeah.

Speaker:

I like by the way, there was a, it was an earlier topic.

Speaker:

when you're in a new environment and you are trying to configure

Speaker:

their EDR tools or XDR tools, can't you just put it in and let it watch

Speaker:

for a while to see what normal is?

Speaker:

You can.

Speaker:

and during that period, you're gonna get a ton of alerts.

Speaker:

so there's your false positives.

Speaker:

Okay.

Speaker:

as you're monitoring, I found a new file that's not common across other

Speaker:

user bases or, Bob logs in at 2:00 AM is, I don't know if that's normal yet.

Speaker:

during that baselining period, you're teaching those tools,

Speaker:

tweaking them or whitelisting them.

Speaker:

there's a ton of things that, that happens during that onboarding period.

Speaker:

so that you're essentially creating the what's normal baseline?

Speaker:

You can't just say, just watch for a month and don't tell me anything.

Speaker:

you could, it's not gonna

Speaker:

you don't advise it.

Speaker:

Okay.

Speaker:

I see what you're saying.

Speaker:

no, I don't advise it.

Speaker:

Okay.

Speaker:

this is why you're the cybersecurity professional, and I'm Mr. Backup.

Speaker:

All right.

Speaker:

All right.

Speaker:

this has been fun.

Speaker:

somewhat depressing, but yeah, just don't do that, doctor.

Speaker:

It hurts when I, it hurts when I do this.

Speaker:

Don't do that.

Speaker:

Yeah, I'd like to, that's a new t-shirt idea.

Speaker:

Just don't do that.

Speaker:

do that.

Speaker:

the antithesis of, Nike.

Speaker:

Yeah, don't do it.

Speaker:

Thanks guys for being on the show.

Speaker:

Thanks, Prasanna,

Speaker:

It was fun and I'm glad we gotta talk about something that was near

Speaker:

and dear to my heart many years ago.

Speaker:

So.

Speaker:

and thanks again.

Speaker:

Mike, you're a little fuzzy all of a sudden.

Speaker:

What happened with your camera there?

Speaker:

it's like an auto focus thing and it's got its own mind.

Speaker:

So I'm not real sure, but,

Speaker:

Anointing

Speaker:

a few more of these with you guys, I would expect some logo.

Speaker:

I'll wear a logo at some point.

Speaker:

you'll get one when I get one.

Speaker:

That's what I'm saying.

Speaker:

on.

Speaker:

All right.

Speaker:

Okay.

Speaker:

thanks to our listeners, we're nothing without you.

Speaker:

That is a wrap.