Network Segmentation Best Practices: Shrink Your Ransomware Blast Radius
Network segmentation best practices can be the difference between ransomware taking out one laptop and ransomware taking out your whole company. Prasanna, my co-author Dr. Mike Saylor, and I talk about how to stop running a flat network, what VLANs and microsegmentation actually do, and where you should start.
Mike and I wrote Learning Ransomware Response and Recovery around one basic idea: assume breach. You’re probably going to get hit, so the real question is how big the blast radius will be. Is it a plop, or does it take out the whole building?
We start with the old hard, crunchy exterior and soft, chewy interior model, and why so many billion-dollar organizations still run flat networks. Spoiler: laziness, budget, and management decisions all play a part. Mike explains VLANs with a hotel key card analogy that will make sense to those who aren’t network people. Then we take it a step further into microsegmentation, where access is based on what you’re there to do instead of where you’re plugged in.
Then we talk about:
- Get buy-in from on high.
- Inventory everything.
- Tag your workloads and build policy around those tags.
- Start with your crown jewels and work your way out. That means the stuff with tight RTOs and RPOs, plus the stuff you really don’t want ending up on the internet.
- Roll it out in phases instead of trying to rebuild everything in a day.
- Document every single rule, or the next guy will tear it all down and go flat again.
We also talk about why IT shouldn’t be the one deciding what’s critical (that’s what a business impact analysis is for) and how to wall off your backup network. You’ll also hear about the night before Y2K when the security team blocked my NetBackup ports one too many times and I lost it. And Mike explains how he used network segmentation to win an argument with a teenager.
If you’re a backup person who’s never had to think much about the network, this one’s for you.
Chapters:
00:00 – Intro
01:26 – Welcome, Prasanna and Dr. Mike Saylor
02:41 – My first client with firewalls on the inside
04:43 – Assume breach and the blast radius
06:24 – Grenades, house locks, and panic rooms
08:41 – What a flat network looks like
09:42 – Why so many companies still run flat networks
10:18 – VLANs explained with a hotel analogy
13:22 – Going further with microsegmentation
14:51 – The management overhead problem
15:15 – Why undocumented segmentation falls apart
18:45 – Leadership buy-in and starting with inventory
20:07 – Rolling it out in phases
21:16 – Tags, policy, and allow rules
22:56 – Default deny (and Mike says “it depends”)
24:00 – Start with your crown jewels
28:18 – Why IT shouldn’t decide what’s critical
30:38 – The night I lost my mind over segmentation
33:00 – Monitor traffic and wall off your backup network
34:35 – Final thoughts, home networks, and the chicken coop
36:04 – Mike’s teenager gets her own segment
Odds are you’re going to get malware at some point.
Speaker:The only question is whether it takes out one computer or your whole site.
Speaker:In this episode, Prasanna, my co-author Dr. Mike Saylor, and I talk about how
Speaker:network segmentation best practices can minimize the impact of an attack.
Speaker:We cover VLANs and microsegmentation using a great analogy of a hotel.
Speaker:We talk about starting with your crown jewels first, and why you
Speaker:have to document every single rule.
Speaker:There’s also a great story about me losing my mind once at a client
Speaker:- caused by network segmentation.
Speaker:If this is your first time watching or listening to me, I’m
Speaker:W. Curtis Preston, AKA Mr. Backup.
Speaker:I’ve been obsessing over backup, recovery, and now cyber recovery for over 30 years.
Speaker:If that’s your bag, then I’m your guy.
Speaker:You’re not going to find anyone that cares about it more than me.
Speaker:Ever since 1993 when I had to tell my boss that there were no backups
Speaker:of the database we just lost.
Speaker:Now I’ve written Five O’Reilly books, a blog and a podcast.
Speaker:Here we turn unappreciated admins into cyber recovery heroes.
Speaker:This is the backup wrap up.
Speaker:Hi, and welcome to The Backup Wrap Up.
Speaker:I'm your host, W. Curtis Preston, AKA Mr. Backup.
Speaker:And first of all, I have with me my real estate non-advisor Prasanna Malaiyandi.
Speaker:How's it going, Prasanna?
Speaker:Good, Curtis.
Speaker:So I think, the typical Indian fashion is anytime you have any money, you should
Speaker:buy gold and you should buy real estate.
Speaker:Nothing else matters, but just buy those two things and you'll be set for life
Speaker:I see.
Speaker:I am looking at real estate in one of the most expensive places to buy real estate.
Speaker:I think I have an interesting take on it.
Speaker:I'm not going
Speaker:Yeah
Speaker:but I, I have a interesting MO, would you not agree?
Speaker:Oh yeah.
Speaker:No, totally.
Speaker:are very like, I don't, I… Creative I think.
Speaker:No, creative.
Speaker:Creative in like what you look for and all the rest.
Speaker:Entrepreneurial, so
Speaker:I try.
Speaker:And speaking of entrepreneurs, we have Dr. Mike Saylor, my co-author on the book.
Speaker:Welcome to the show again, Mike
Speaker:Hey everybody.
Speaker:Thanks for having me again
Speaker:We're gonna have to get you like a room, like a, a spot on the couch or something.
Speaker:all the rooms here, all the rooms here are gone.
Speaker:but we'll do that.
Speaker:we're gonna go back in time to, a hundred, hundred years ago, back
Speaker:when I was working at a particular client at, the turn of the century.
Speaker:And
Speaker:Back in the day
Speaker:back then that at the time I had never seen, and we're now going
Speaker:to talk about it in this episode.
Speaker:I think it's now becoming more commonplace, and for those of you that,
Speaker:you're like, "What's he talking about?" What I'm talking about is it was the
Speaker:first time I'd ever gone to a large company that had firewalls inside, right?
Speaker:E- everywhere I had ever worked, the firewall, it was a hard, crunchy
Speaker:exterior, soft, chewy interior, right?
Speaker:that, that was, once you were inside, you were good, right?
Speaker:That was the way… I worked my, I cut my teeth at a $35 billion credit card
Speaker:corporation not that long before this, and there was nothing on the inside, right?
Speaker:Other than, usernames and passwords, but there was nothing
Speaker:that was blocking traffic.
Speaker:Was that before PCI compliance and all the rest?
Speaker:What?
Speaker:Yeah.
Speaker:no, I don't think so.
Speaker:Okay, just checking.
Speaker:I don't
Speaker:Just checking
Speaker:we'd have to Google
Speaker:Yeah.
Speaker:PCI didn't come out until just after, 2000,
Speaker:Yeah, yeah.
Speaker:So the answer is yes, that was before PCI compliance.
Speaker:Okay
Speaker:But, this particular company had a lot of, a lot of personal information and
Speaker:they were basically saying… They were a company that I think of companies
Speaker:would say this today, but they were the first company that I worked with
Speaker:where they basically said, "IT is our product." "IT is not the thing that we do
Speaker:to make our product with, the security.
Speaker:If our security fails, the type of data that we store, our business is out of
Speaker:business." And so they were like, "That just can't happen," and so they did the
Speaker:kind of thing that we're talking about.
Speaker:And of course, what we're talking about is network segmentation
Speaker:As you were talking, I was trying to be like, "What is he going to be talking
Speaker:about?" I was like, "That was not on my bingo card networks segmentation
Speaker:So basically, the whole point here, and when Mike and I wrote the book, I
Speaker:think the basic premise of the book was
Speaker:you're probably gonna get ransomware.
Speaker:That would be Learning Ransomware Response and Recovery over this
Speaker:shoulder over here, and for those of you watching on YouTube, available
Speaker:at, bookstores, available near you.
Speaker:My latest book, fifth book actually, and, the first one where I brought
Speaker:on a co-author, and, and that's how Mike and I got to know each other.
Speaker:and interestingly enough, we have never actually met, for those
Speaker:of you that don't know that.
Speaker:Mike and I have never physically met in person.
Speaker:We have always seen each other like this and during a bunch
Speaker:of Zoom meetings with O'Reilly.
Speaker:but we worked on a book, and the idea of the book was that
Speaker:you should assume breach, right?
Speaker:Assume that you're going to get ransomware, right?
Speaker:or some other type of malware.
Speaker:It doesn't have to be ransomware, but, we focused on ransomware.
Speaker:And if you're going to assume that, one of the things that we were saying was you
Speaker:want, if you're gonna assume it's going to get inside, you want to do whatever
Speaker:you can to reduce the blast radius.
Speaker:What do we mean by that, Prasanna?
Speaker:When something gets in, how many things does it take out?
Speaker:So if it gets in via an employee laptop, does it take out your servers, your main
Speaker:production applications, everything else?
Speaker:And that's kind of the blast radius
Speaker:Yeah.
Speaker:And of course that term comes from, bombs and such, right?
Speaker:Going boom
Speaker:yeah, going boom.
Speaker:And it's gonna go boom.
Speaker:The question is, does it, is it a plop or is it a, does it take out, company, right?
Speaker:so is it like what we're about to talk about is similar to, you know in the
Speaker:movies where a grenade gets tossed in and you have the one guy being like, "Oh,
Speaker:no," and then he jumps up on the grenade?
Speaker:Is what we're talking about similar to that?
Speaker:That's a very sad story, but yes.
Speaker:That would, that is one way to reduce the blast radius.
Speaker:I will say it's, it's effective.
Speaker:and there are people throughout history that have done that, and they deserve all
Speaker:of the credit and goes to those people.
Speaker:but what we're, I think what we're trying to say here is that you shouldn't
Speaker:have, like again, to continue with the grenade analogy, you shouldn't have
Speaker:your entire platoon in one building with no walls between them, right?
Speaker:And one grenade just takes them out, right?
Speaker:that, how's that for the analogy, Mike?
Speaker:Pretty good
Speaker:Bless you
Speaker:I know you, you talk a lot about, we d- you know, we often, we use
Speaker:analogies of of houses and things.
Speaker:how would you put this in a house analogy?
Speaker:it's very similar.
Speaker:y- you want walls, you want other doors.
Speaker:And to your point about having interior firewalls, you can
Speaker:have interior… we do, right?
Speaker:You have locks on your bathroom.
Speaker:Some of us have locks on our bedrooms and our closets.
Speaker:same principle.
Speaker:Limit access to different of your environment that not everybody needs
Speaker:or should have access to or can be walled off, when there's a threat
Speaker:Can I just mention something?
Speaker:And it's might seem like a non sequitur, but it's not.
Speaker:I was, randomly shown YouTube videos over the weekend, and it's about,
Speaker:houses that are supposed to be safe.
Speaker:And so they talk about, perimeter defenses, and, they were showing,
Speaker:an interior safe core, like you were talking about, Mike, like with 600
Speaker:pound solid steel doors that you can…
Speaker:with five bedrooms inside and supplies and everything else, so in case someone
Speaker:makes it past the outside perimeter, you're safe in the interior, and you
Speaker:don't have to worry about things.
Speaker:And I was like, "Wow, that's pretty impressive."
Speaker:And
Speaker:was also $15 million.
Speaker:that… Yeah, we're gonna bring that into the discussion.
Speaker:I'm glad you brought that up, Prasanna, 'cause we're gonna, gonna
Speaker:remind me, hopefully, by the end of this episode that I like that topic
Speaker:or I like that analogy, quite a bit.
Speaker:let me just look back here.
Speaker:so opposite of what we're talking about, Mike, is a flat network.
Speaker:You wanna talk about what that is?
Speaker:It's like a, imagine a, a bicycle wheel, a hub and spoke.
Speaker:So all of your core stuff is in the middle, and it connects
Speaker:to everything, printers and workstations, and they can all talk.
Speaker:And, it's easy to conceptualize that as just your office building.
Speaker:but I've assessed billion-dollar organizations that have offices in
Speaker:casinos and convention centers and high-rise buildings all over the
Speaker:world, and it's a flat network.
Speaker:From anywhere in the world, it could… you can ping the server
Speaker:in the data center, and from a conference hall, from wherever.
Speaker:It's completely flat, unrestricted,
Speaker:W- why is that, Mike?
Speaker:it seems to go against so many things we've talked about before and we will
Speaker:be talking about in the podcast, but is it just easier to do it that way?
Speaker:Absolutely easier, right?
Speaker:so there's so many things that can contribute to poor network architecture.
Speaker:One of them is laziness, or incompetence.
Speaker:Another one is budget.
Speaker:And so a lot of times we give the IT guys so much crap for, "Why did you build it
Speaker:this way?" And they're like, "Hey, I know.
Speaker:I know it's bad."
Speaker:But, they didn't give me budget to buy a new router or one that's
Speaker:capable of VLANs or whatever it was.
Speaker:Sometimes it's budget.
Speaker:Sometimes it was management that made that decision, and IT guys just get the brunt
Speaker:of it it looks bad and it's all technical
Speaker:Mike, you already brought up the term of VLANs, and we, I just, during, we took a
Speaker:little bit of a break during which we had a number of, episodes, and one of the ones
Speaker:that got the most traffic was the one that we talked about, network segmentation.
Speaker:And, we talked about VLANs, and, the… not sure everybody really
Speaker:understands if you're not a network person, again, re- realizing that we're
Speaker:reaching people that are maybe backup oriented, maybe cy- even cyber oriented,
Speaker:but they're not network oriented.
Speaker:do you wanna dive into, what we're talking about from a VLAN perspective?
Speaker:Sure.
Speaker:and I like to come up with analogies that, are relatable, and I'm trying to… I'm
Speaker:buying time here to come up with one.
Speaker:Hotels.
Speaker:So think of a, a hotel where you've had to use your room key in the elevator, right?
Speaker:So there's hotels where you can get in and push any button regardless of what
Speaker:floor your room is on, and then there's hotels where you get in and you have to
Speaker:use your room key, and the floor that you select is restricted to the room key.
Speaker:So that's similar to a VLAN.
Speaker:So physically, we've built this hotel with floors one through, let's just say
Speaker:one through 10, so that's the physical architecture, and part of that physical
Speaker:architecture is the security of each room.
Speaker:I have to have the key to get into my room as well.
Speaker:So if you think of the hotel where you can go to any floor without
Speaker:having to use your room key,
Speaker:You have the potential to segment that network so that based on the access in my
Speaker:room key, that dictates where I can go.
Speaker:You can then further segment or restrict access in this network, in this hotel
Speaker:by saying, "You can only go to the floor in this building or that part
Speaker:of this network based on the access on your card or your credentials, your
Speaker:login credentials," as an example.
Speaker:So that's more virtual.
Speaker:So it's not physical restricted, although elevator, but
Speaker:elevator goes to every floor.
Speaker:Virtually, I'm segmenting this hotel by access privileges,
Speaker:and that's what a VLAN does.
Speaker:So on a… Imagine, imagine the network hardware or the switch as
Speaker:the building itself, and there's all these different places you can
Speaker:plug stuff into this network switch.
Speaker:those would be the different floors So when you log into the interface of this
Speaker:switch, you can draw a box around the, the places you plug stuff in and say,
Speaker:"I want this to be floor two." And only with access to floor two can access
Speaker:floor two, and possibly even things on floor two, like the janitor closet.
Speaker:Like I can get to my room on floor two, but my badge doesn't
Speaker:work on the janitor closet.
Speaker:So you can further restrict things inside of VLAN as well.
Speaker:But in essence, does that resonate with you guys as far
Speaker:We want to even evolve past, sort of basic VLANs, which are typically port-based,
Speaker:and then go to micro-segmentation, which is more, application-based, and
Speaker:purpose-based, which I think you were, again, going with that analogy, you
Speaker:were going with you have a guest in the hotel, you have a food person, right?
Speaker:You have a maintenance person.
Speaker:You have a, an administrative person.
Speaker:You may even have a vendor, right?
Speaker:That it's able to go to certain things at certain times, and I think that's
Speaker:closer to the idea of micro-segmentation, where we're able to say not just these
Speaker:ports, 'Cause I know we can say this VLAN are on these ports, and this VLAN
Speaker:are on these ports, but I like the idea, the, the key card analogy that you said.
Speaker:So imagine if literally every single door, and probably, like, when I'm thinking
Speaker:about this, this is probably very close to the way things are in a big hotel, that
Speaker:literally every single door has a key card entry, then you can dynamically allocate,
Speaker:people's in that segment of the hotel, and that, again, that segment is very virtual.
Speaker:It ha- it has to do with your purpose and not necessarily based on your location.
Speaker:although that's where it starts because we're gonna do this, this
Speaker:segmenting inside this hotel, right?
Speaker:but then you can further subdivide it out based on what you are here to do,
Speaker:I actually really like the analogy, Mike, 'cause it makes a lot of sense.
Speaker:and I'm gonna ask this every single time you bring up a technology thing,
Speaker:and please don't say it depends.
Speaker:But one question is, this is all great, especially when you get
Speaker:down to micro-segmentation, 'cause you can really secure devices.
Speaker:And with v- VLANs, you can also ensure, as an example, that floor two can't make
Speaker:it down to floor three, but it's o- or up to floor three, but it's okay for
Speaker:floor four to maybe go down to floor three because that's where the gym
Speaker:is, or for f- everyone to have access to the lobby, to the ground floor.
Speaker:This is all great, but when you start to get very complicated
Speaker:micro-segmentation and VLANs and everything else, it becomes a nightmare
Speaker:from, a management perspective.
Speaker:So at home I run multiple networks.
Speaker:I have a VLAN, multiple VLANs, and just, organizing devices, dealing
Speaker:with firewall rules, everything else like that across my environment,
Speaker:and mine's a very simple network.
Speaker:Of course, I make it complicated, but that's just how I am.
Speaker:I'm just imagining, Curtis's example he brought up at the beginning, right?
Speaker:That very large financial company he used to work at, right?
Speaker:And how they would go about managing VLANs and micro-segmentation across thousands,
Speaker:hundreds of thousands of devices
Speaker:It is a lot of overhead, and one of the reasons, another one of the reasons
Speaker:that you may see a flat network is that there used to be some level of comple-
Speaker:complexity and technical, confusion, and we'll call it that, when, a new IT
Speaker:engineer comes in, and that beautifully designed architecture of all these VLANs
Speaker:and micro-segmentation was not documented.
Speaker:And so the new guy comes in with zero turnover, zero ramp-up time,
Speaker:and n- things start breaking or he doesn't know how it's built.
Speaker:So he's gotta tear all that stuff down and start over.
Speaker:You can figure some of that out based on the type of technology you have,
Speaker:but it takes time, and, in a lot of organizations we don't have bandwidth,
Speaker:and we need bandwidth to do stuff like…
Speaker:So I have seen that, where it becomes flat in order to start over.
Speaker:a- and one of the critical points I'll make about this, and hopefully you
Speaker:both practice this in your segmented environment, is you have to document it.
Speaker:You have a diagram, you have ports, you've got screenshots of your VLAN
Speaker:interface, you've got, a printout of the configuration and your
Speaker:rules, most importantly your rules.
Speaker:a- so that if something does break, or if you need someone to come
Speaker:and help you, it's documented.
Speaker:It's easier to see on paper.
Speaker:I'll add to that then that, a- and one of the things I wanna add
Speaker:to the analogy in this discussion is that, a- and you mentioned it,
Speaker:it's dynamic or it can be dynamic.
Speaker:Even though I've set this floor or these rooms as part of this VLAN, because it's
Speaker:virtual, I don't have to go run new cable.
Speaker:I can just go into the interface and pretty much drag and
Speaker:drop, this port or that room.
Speaker:I want it now to belong to this other VLAN.
Speaker:m- having those, that virtual segmentation makes it much easier to ma- to manage.
Speaker:and then the last thing I'll mention is, there's multiple parts
Speaker:to make this work effectively.
Speaker:There's the architecture obviously.
Speaker:There's the documentation, then there's the acc- and very often network
Speaker:architecture and access control are different stakeholders, different SMEs.
Speaker:making sure that the communication is there so that when either one of
Speaker:them get, an objective to say, this VLAN's gotta change or this access has
Speaker:to change, they still have to be able to collaborate and make that work.
Speaker:So absolutely, it can get complicated pretty quick
Speaker:Yeah, and like anything else that we've talked about, and that we'll
Speaker:talk about on this show, the b- the bigger the environment is, the more,
Speaker:A, it has to start with from on high.
Speaker:It has to be empowered from on high, right?
Speaker:Because you're gonna, you're gonna trip over a few, electrical cords and unplug
Speaker:some stuff once in a while, right?
Speaker:you start doing this.
Speaker:But that's, I'd say, the number one.
Speaker:And then number two, it's got to start with an inventory, right?
Speaker:Because we first have to know everything that we have.
Speaker:and then, then we n- we start grouping that inventory together, right?
Speaker:And we start observing the inventory.
Speaker:We start observing how, these workloads interact with each other, and then we
Speaker:start tagging those workloads, right?
Speaker:now how you do tagging and then how you use those tags is gonna
Speaker:be incredibly different based on whether or not we're talking about
Speaker:a traditional on-prem workload.
Speaker:Are we talking about, containers?
Speaker:are we talking about a purely cloud-based workload?
Speaker:How you use, how you implement those tags and then how you use those tags
Speaker:are gonna be incredibly different.
Speaker:I think that's probably, once you have… Again, it, we say it like,
Speaker:"Oh, yeah, just go inventory." I understand that the bigger you
Speaker:are, the bigger of a pain this is.
Speaker:But as I've often said, any good security plan is usually a pain, right?
Speaker:I like, Curtis, how you approached that, like getting the
Speaker:inventory and walking through.
Speaker:But I think another thing to also mention is you don't have to deploy your
Speaker:entire VLAN infrastructure in one go.
Speaker:You could say, "Here are the critical applications," or maybe,
Speaker:"Here are the risky devices.
Speaker:Let me segment those off first, and then I can slowly roll out everything
Speaker:else and have a prioritized plan for how to approach this," rather than
Speaker:saying, "I need to destroy everything and rebuild everything in a day."
Speaker:There's a lot of variety in segmentation, and it comes
Speaker:down to what's your objective?
Speaker:so a lot of organizations simply segment, public areas, conference rooms, lobby from
Speaker:production areas or normal user areas.
Speaker:They segment production from test so that things that happen
Speaker:in a test environment don't…
Speaker:So it's being done to some degree.
Speaker:just how do we take that, somebody's doing it, somebody's capable and knowledgeable
Speaker:and skilled in creating these segments.
Speaker:It just takes a strategy to roll that out in more meaningful ways
Speaker:Yeah, and also, you talk about the way that this is already being done.
Speaker:I'm sure that it's very common for when b- people are given a laptop,
Speaker:for example, that automatically gets put into a segment, right?
Speaker:And then you're… And the same thing is, and I think the more modern of
Speaker:an environment you have, the more virtualized, the more cloudy, if I
Speaker:could- the cloudier you are and the more virtualized you are, the more used to
Speaker:tags you, you are at this point, right?
Speaker:We didn't really do tags, at least not when I was doing physical servers, right?
Speaker:and I- it doesn't, it's not so strongly encouraged as it is in the
Speaker:virtualized world and the cloud world.
Speaker:It's oh, what is the tag for this environment?
Speaker:And you can automatically create those tags, right?
Speaker:You automatically create an Oracle tag, you automatically create an application
Speaker:tag, a laptop tag, a whatever tag.
Speaker:And so you could end up having these segments already, you're just not, you
Speaker:haven't yet gone to the next level, which is policy, And we say, "Okay, now that
Speaker:we have everything tagged, or at least we think we have everything tagged," or to
Speaker:go with you, Prasanna, I completely agree.
Speaker:We think we have the important critical, y- workloads tagged.
Speaker:Let's see if we can enact a policy where that, let's cl- let's cr-
Speaker:start creating the allow rules.
Speaker:Okay, the, all of the ones with this tag can all talk to each other, but
Speaker:they're not allowed to talk to the ones with this tag, the laptop tag, right?
Speaker:and then you go, the big thing is when you start doing the deny, and, but
Speaker:Mike, I just wanna get your take on that.
Speaker:My understanding is typically with firewalls and VLAN segmentation, by
Speaker:default usually things are denied unless you allow explicit access.
Speaker:I think it changes from vendor to vendor, but I think that's
Speaker:usually the best practice, right?
Speaker:So in North America it's denied by default.
Speaker:In South America it's allowed, but I'm
Speaker:A dum-
Speaker:It's 'cause, things…
Speaker:You almost had me there, Mike.
Speaker:wherever it flows the other way when you flush the toilet.
Speaker:it truly, I'm, I have to say it this time, it depends.
Speaker:It does.
Speaker:But once you,
Speaker:firewalls will
Speaker:if you have decided to have a flat network for whatever reason, you've
Speaker:turned off that default deny.
Speaker:Yeah.
Speaker:You have.
Speaker:you've got a rule that sits on top of it,
Speaker:Yeah.
Speaker:Deny, deny, deny.
Speaker:It's a double negative, which is a positive
Speaker:Allow all, and if that doesn't work, then deny.
Speaker:But yeah.
Speaker:Yeah.
Speaker:It's the chmod 777 for the old, Unix, people on, on the thing.
Speaker:the I do wanna go back to the thing that you said earlier, the
Speaker:idea of the, this panic room.
Speaker:I do think that, and it's very similar, it's related to what you said just a
Speaker:few minutes ago, and that is we really need to figure out the workloads that
Speaker:either, A, are super critical from a business operating perspective, right?
Speaker:Meaning what… these are typically gonna be workloads that we're going to
Speaker:have very tight RPOs and RTOs, right?
Speaker:That's recovery point objective and recovery time objective.
Speaker:If you're not familiar with those terms, go search on our episodes, that
Speaker:we fully define those terms, right?
Speaker:So they have very tight RPOs and RTOs, and so these are ones where the
Speaker:business can't d- can't do without them for very long periods of times.
Speaker:These would be the ones where we start putting them in the panic room.
Speaker:These are the ones that we start saying, "Okay, we really have to wall these
Speaker:off, from, as much as possible." And then to that, I would add not so much
Speaker:business critical from a money-making standpoint, but business critical
Speaker:from a, it would re- it would be really bad if this information got out
Speaker:Was it emails?
Speaker:email Y-
Speaker:I was going to the Sony hack from years ago.
Speaker:That's literally what went in my mind.
Speaker:right?
Speaker:yeah, the informa- this is your, your corporate secrets, your… F- both
Speaker:corporate secrets from a, meetings you really would not rather have public,
Speaker:like what happened with the Sony hack, and then, also just your corporate IP,
Speaker:your intellectual property, getting out.
Speaker:So starting there, you do have to be careful, right?
Speaker:Because you're also messing with the crown jewels, so you don't wanna mess those up.
Speaker:but what do you think about that idea, Mike, of starting at the core there?
Speaker:I think it's a great idea.
Speaker:so especially in a large environment.
Speaker:In a smaller environment, it's very easy to go this and this, can and
Speaker:shouldn't and, I can s- I can see the important stuff from here, right?
Speaker:So but in a larger environment, especially multiple locations,
Speaker:multiple floors, multiple departments, you're gonna start fighting politics
Speaker:with, for access, defining access.
Speaker:So it's much easier in my experience, to start with the
Speaker:crown jewels and work your way out.
Speaker:is it that you care about?
Speaker:And it could be m- obviously more than one thing.
Speaker:It could… Is it your enterprise, your ERP, your financial system?
Speaker:Is it, We have a client that does, a lot of crash testing, and their intellectual
Speaker:property is all the video and telemetry from the sensors in the crash test.
Speaker:They manufacture other stuff, but this is their crown jewel.
Speaker:This is what makes them who they are.
Speaker:so it's this, maybe priority one.
Speaker:Some manufacturing, priority two.
Speaker:something else, intellectual… design drawings, intellectual
Speaker:property type stuff is three.
Speaker:now you've identified all your really important stuff that you
Speaker:care about that's gonna keep the lights on and, the business moving.
Speaker:For each one of those, who needs acs- access to it, daytime, normal
Speaker:behavior, let's baseline all that.
Speaker:How's it being used?
Speaker:What's normal?
Speaker:it being backed up?
Speaker:'Cause it's not just who's accessing it day to day.
Speaker:Where's that data going it's getting archived?
Speaker:what's our disaster recovery plan for that thing and all the stuff that supports it?
Speaker:'Cause that's gonna go into segmentation.
Speaker:'Cause if you have a, a, a disaster and we've gotta rebuild that somewhere
Speaker:else, you also have to rebuild all that access stuff somewhere else, that goes
Speaker:back to that stuff being documented.
Speaker:But absolutely, start at the core, work your way out, and then fight those
Speaker:battles and they're political battles.
Speaker:On, on… As you do that, as, as your circles, your concentric circles of
Speaker:security grow, your onion, you're gonna, you're gonna run into these
Speaker:political battles of, "I needed access to that." Really, why?
Speaker:'Cause it doesn't support any of the use cases that we've identified or defined.
Speaker:And so do we need an exception for that?
Speaker:Do we need to pivot or redefine?
Speaker:and I- in my opinion, that's a, an amazing way to start.
Speaker:Mike, one of the things you mentioned is you said, I think you said
Speaker:something along the lines of, yeah, identify like what's important
Speaker:and then start figuring that out.
Speaker:But as an IT guy, I- or network guy, I'm not the one who's
Speaker:figuring that out, right?
Speaker:I… that's not my decision, right?
Speaker:It should not be your decision
Speaker:guy's decision of what the RTO and RPO are, right?
Speaker:Abso- or the, yeah, absolutely.
Speaker:Everything's data and architecture to us.
Speaker:we can tell you the value of that thing.
Speaker:We can tell you the value of a DLT tape.
Speaker:We can value the value of a firewall or how much it's gonna cost to
Speaker:run 100 yards of ethernet cable.
Speaker:we can tell you all those things.
Speaker:We cannot tell you the value of the stuff that traverses it or is stored
Speaker:on it or is being used by the business.
Speaker:That we're not that, I'm not gonna say we're not that smart, but we, our f-
Speaker:we don't have the bandwidth to consume that or to be concerned with that.
Speaker:It's,
Speaker:the business.
Speaker:it's not your role, right?
Speaker:nope,
Speaker:Yeah
Speaker:it's not your lane.
Speaker:And there's an exercise called a business impact analysis that can be done.
Speaker:it's fairly concise and it's impact.
Speaker:it's similar to a risk assessment where we just go talk to the,
Speaker:y- the key stakeholders that are driving critical business functions.
Speaker:"Tell me what you do here. Tell me what data and information, your role requires."
Speaker:We talk about the technology that, that's required to support that role,
Speaker:but we also talk about contingencies.
Speaker:Like what would you do if your computer's not available?
Speaker:How would you continue doing your job?
Speaker:Anyway, long story short, that BIA would identify key stakeholders, critical
Speaker:processes and procedures, all the stuff that those jobs require or lean on.
Speaker:gonna point you, if not identifying it directly what your crown jewels are, it's
Speaker:gonna really point you in that direction.
Speaker:You're gonna see a lot of commonality, that could help you identify those things
Speaker:Mike, I, I didn't hear anything you said after DLT tape
Speaker:You started droo- I saw some drool coming out of your mouth
Speaker:you… I was like, "Man, Mike is dating himself there."
Speaker:Time Track,
Speaker:It…
Speaker:I didn't wanna go that far back
Speaker:gracious, DLT.
Speaker:haven't thought about DLT in a minute, as the kids say.
Speaker:So Curtis, for you, I'm surprised… So I know we're
Speaker:talking about network segmentation.
Speaker:I'm surprised you have not brought up the story where you wanted
Speaker:to bash your head into the wall because of network segmentation
Speaker:You mean from that same client?
Speaker:Yep.
Speaker:that's the same one, yeah.
Speaker:Yep.
Speaker:y- it, it can… th- this is, this is like a, this is like a how
Speaker:not to do kind of thing, right?
Speaker:So the, the, this company that I talked about was very segmented off,
Speaker:and, one of the only application that needed to talk to every other
Speaker:application was my application, right?
Speaker:Backup.
Speaker:so I broke every segmentation, system that they had because I need… the,
Speaker:the idea of having a single server that could talk to every other server
Speaker:just basically blew their minds.
Speaker:And so they kept trying to, wall me off, and they kept doing it via ports.
Speaker:They kept seeing if they could figure out what ports, And a-
Speaker:at this point it was NetBackup.
Speaker:What ports was NetBackup using?
Speaker:And they'd say, "We figured it out." And then they would… And then I'd
Speaker:come to work the next day, and I'd find out that the backups all failed
Speaker:because they had blocked it off, right?
Speaker:and then it w- it was the, the final night of the fin- it was Y2K.
Speaker:Y2K was coming.
Speaker:And, it was the, it was after code, the code, freeze.
Speaker:So I was like, "Okay, so no more stuff," and everything.
Speaker:And I was there at 10 o'clock at night, 'cause that's when I
Speaker:was doing a lot of this work.
Speaker:And, I went to go run the backups, and they hung, and I lost my mind.
Speaker:And I started screaming out a blue streak, because I thought I was alone
Speaker:in the building, and thought I could get away with it, and, turns out I was not.
Speaker:And the guy that I was reporting to was there, and he stuck his head out
Speaker:and asked me if I was, like losing it.
Speaker:And I was just like, the network, the security with the
Speaker:and the da- da-da." And he's like, "I got it."
Speaker:Yeah.
Speaker:So you can step on some people.
Speaker:Try not to do that, or try to do that as little as possible,
Speaker:but, leave that up to Mike.
Speaker:Let Mike step on people I just, something tells me he could do that.
Speaker:the, I- so here's the thing.
Speaker:What we're basically saying is first off, the idea is, is monitor traffic,
Speaker:look at traffic that's going from VLAN to VLAN, see what's, happening.
Speaker:I do… If you have a VLAN for the backup, especially if you have a separate
Speaker:network for the backup system, quarantine that off as much as possible, right?
Speaker:especially things like, authentication and authorization so that you don't
Speaker:have, we talk about this with, with, active directory, right?
Speaker:where you don't want that to, if your primary active directory is compromised,
Speaker:you don't want that to bleed over into the, into the backup system.
Speaker:And then as we've already talked, l- look at the high value targets, look at
Speaker:the, the ones that are both h- valuable from a business perspective in terms
Speaker:of keeping the, the lights on, and then also in terms of the data that you
Speaker:really wouldn't wanna get out, and that's where you should start this process.
Speaker:And then Google VLANs and micro-segmentation for whatever
Speaker:type of environment you have, right?
Speaker:The three big ones being traditional infrastructure, say highly virtualized,
Speaker:but maybe I'll say f- I say four.
Speaker:So a highly virtualized but physical infrastructure, then we start
Speaker:talking about, containers and modern workloads, and then we talk about
Speaker:a highly cloud, based environment.
Speaker:And then, how do I do micro-segmentation in a cloud-based environment or
Speaker:whatever it is you have, and learn that.
Speaker:'Cause we can't cover all of that in, in an episode like this.
Speaker:We're just trying to, we're just trying to seed you, give you ideas.
Speaker:any final thoughts on that?
Speaker:I'll start with you, Prasanna.
Speaker:No, I think people should be using VLANs.
Speaker:the
Speaker:Just be careful when people start complaining because your firewall rules
Speaker:blocked off access that they used to have
Speaker:Yeah.
Speaker:How about you, Mike?
Speaker:Agreed.
Speaker:VLANing is, it's a pretty simple skill to learn.
Speaker:a lot of the te- technologies out there today, the interface is very intuitive,
Speaker:but it does take some strategy.
Speaker:don't go log in and just start doing stuff.
Speaker:it out on paper first.
Speaker:Really think through how I want to segment stuff, even at home.
Speaker:seg- create a, a guest wireless network.
Speaker:That's a great way to start.
Speaker:I have camera- I have, internet-connected cameras that are on a different segment
Speaker:in my home, including… I've got two.
Speaker:I've got the house, and I've got the chicken coop.
Speaker:so the chickens are on their own segment because it's, much
Speaker:further away from the house.
Speaker:If it gets compromised, then it's on its own segment.
Speaker:a- and it doesn't hurt anything to play with it, at home, 'cause you can undo it.
Speaker:Just put some thought to it before you start messing with it,
Speaker:you could,
Speaker:easy to do
Speaker:you can undo it, Mike, until someone in the household complains that
Speaker:the internet's down or they're not able to stream to the Apple TV
Speaker:from their phone or whatever it is.
Speaker:So just be careful with that too.
Speaker:Yeah.
Speaker:All right,
Speaker:tell you a, a
Speaker:Go ahead
Speaker:real quick practical story.
Speaker:So my, my oldest, this was years ago when she was in high school.
Speaker:she would wait till 10, 10 o'clock midnight to get on
Speaker:the internet with her friends.
Speaker:And though she was upstairs, it was right above our room, and so we could hear her.
Speaker:and so I put her on her own network segment, and then I assigned,
Speaker:time restrictions to that segment.
Speaker:So just letting you know.
Speaker:that's one
Speaker:Herc?
Speaker:Me- mean daddy.
Speaker:all right.
Speaker:And with that, I will say thank you for listening and thank
Speaker:you for you two coming on.
Speaker:that is a wrap
Speaker:The Backup Wrap Up is written, recorded, and produced by me, W. Curtis Preston.
Speaker:If you need backup or DR consulting, content generation, or expert witness
Speaker:work, check out backupcentral.com.
Speaker:You can also find links for my O'Reilly books on the same website.
Speaker:Remember, this is an independent podcast, and any opinions that
Speaker:you hear are those of the speaker and not necessarily an employer.
Speaker:Thanks for listening
Apple Podcasts
Spotify
Castro
RSS Feed