Network Segmentation to Prevent Ransomware: What the UCSF Attack Taught Us

Network segmentation to prevent ransomware isn't just a nice-to-have — the UCSF ransomware attack proves it's what separates a contained incident from a catastrophe. UCSF got hit. Their segmented network kept the damage from spreading across their entire operation. That's the difference we're talking about in this episode.
Dr. Mike Saylor — my co-author on Learning Ransomware Response and Recovery — joins me and Prasanna to break down exactly how network segmentation works, why it matters for ransomware defense, and how to start doing it without breaking everything in the process. (Not that I've ever done that. Much.)
We cover what segmentation actually is, how VLANs make it manageable, the "need to talk" principle, and where microsegmentation fits in — and when it becomes overkill. We also get into the complexity trap: more rules and more layers don't automatically mean more protection. Sometimes they mean nobody can troubleshoot anything when the house is on fire.
If you're an IT admin trying to make the case for better network architecture, or you just want to understand what would actually stop ransomware from ripping through your environment, this is the episode.
Chapters:
00:00:00 — Intro
00:01:40 — Welcome & Guest Introductions
00:05:17 — Case Study: UCSF Ransomware Attack
00:08:13 — What Is Network Segmentation?
00:12:32 — VLANs Explained
00:19:50 — The Need to Talk Principle
00:30:54 — Complexity vs. Security
00:31:09 — Microsegmentation
00:38:55 — Action Items: Where to Start
00:42:05 — Monitoring VLAN Traffic
You found the backup wrap up, your go-to podcast for all things
Speaker:backup recovery and cyber recovery.
Speaker:This week we look at network segmentation and how it can prevent ransomware from
Speaker:tearing through your organization.
Speaker:My co-host persona and I, uh, talked to Dr. Mike Saylor, my co-author on
Speaker:learning ransomware response and recovery.
Speaker:Uh, and we talk about, first off, we.
Speaker:Talk about what it actually is, uh, what VLANs are, how they figure
Speaker:into this, uh, the need to talk principle and also micro-segmentation.
Speaker:We started with the, the UCFS ransomware attack as a good case study.
Speaker:They did get hit, but they weren't destroyed basically because they got this.
Speaker:Right.
Speaker:Uh, we talk about how to, to do this without destroying everything.
Speaker:Um, not that I've ever done that.
Speaker:Anyway, just a quick note about me.
Speaker:I'm w Curtis Preston, AKA Mr. Backup, and I've been obsessing over backup recovery
Speaker:and now cyber recovery for over 30 years.
Speaker:If that's your bag, then I'm your guy.
Speaker:You're not gonna find anyone that cares about it more than me.
Speaker:Ever since 1993 when I had to tell my boss there were no backups of that
Speaker:production database that we lost.
Speaker:Now I've written five books, a blog and a podcast.
Speaker:Here we turn unappreciated admins into Cyber Recovery Heroes.
Speaker:This is the backup wrap up.
Speaker:Welcome to the backup wrap up.
Speaker:I'm your host, w Curtis Preston, AKA, Mr. Backup, and I have a guy who continues to
Speaker:surprise me at the stuff he knows about.
Speaker:I don't remember what it was, persona, but just like the other day we were
Speaker:talking and you were like, do you, do you, you started talking about
Speaker:something else where you're like, are you surprised that I know this?
Speaker:And then, it was something yet again.
Speaker:I'm like, of course you are aware of the, I don't know what it was,
Speaker:but something persona, Molly Yondi.
Speaker:How's it going?
Speaker:Persona,
Speaker:I am good.
Speaker:Curtis, you know what it might have been, was it our discussions around POE?
Speaker:probably was about P oe.
Speaker:Yeah.
Speaker:yeah.
Speaker:The, exactly the power over ethernet for those uninformed, I've been,
Speaker:I've been living POE quite a bit, lately because I've been, re, re.
Speaker:Basically reinstalling, you know, replacing all of the
Speaker:cameras in a, in a, yeah.
Speaker:Updating, thank you.
Speaker:Updating a, a p OE based security camera system for a preschool.
Speaker:And it's been, it's been a journey.
Speaker:and, lots of, lots of time on ladders and, you know, good times.
Speaker:I have yet to, you know, fall or anything.
Speaker:I have drawn blood at least once.
Speaker:But anyway, so.
Speaker:It is expected.
Speaker:but anyway.
Speaker:And then we have our Intrepid cybersecurity
Speaker:professional, Dr. Mike Saylor.
Speaker:How's it going, Mike?
Speaker:Well guys, thanks for having me.
Speaker:Yeah, I am, you know, no one at home knows how much effort we went
Speaker:into today to get you to sound
Speaker:we were having, we were having so much difficulty.
Speaker:microphone, troubleshooting.
Speaker:Yeah, that was, something I, yeah,
Speaker:But, but wait.
Speaker:Curtis, before you go on, I think
Speaker:yeah.
Speaker:You should probably sort of give a bit of background about Dr. Mike
Speaker:Yes, yes,
Speaker:I know we have, we've been having him on a bunch recently on the
Speaker:yes.
Speaker:I think it might be worthwhile.
Speaker:Yeah, so Mike is the, co-author on this book here, learning
Speaker:Ransomware Response and Recovery that was published just last month.
Speaker:Although he and I are still, it's, it's so new that he and I are still waiting
Speaker:for our own little copies and, it's gonna be like a competition, like a, like a,
Speaker:we need a. We need a, a pool to see who gets their, who gets their copies first.
Speaker:Mike, of course was the, he did the cyber part of this book, and I did
Speaker:the backup and Dr part of this book.
Speaker:And, honestly, I think it was a, it was an incredible, I was about to say marriage.
Speaker:let's not do that partnership.
Speaker:Incredible partnership.
Speaker:And that, Mike, I don't know if I told you, but the, but the, the.
Speaker:Our editor said that she had actually never seen a team that worked as
Speaker:well together as you and I did.
Speaker:because I mean, you, it was such a clear delineation of expertise.
Speaker:and you clearly know your stuff.
Speaker:I clearly know my stuff.
Speaker:And, this is one of those books where you really need stuff on both sides, right?
Speaker:And, many of our,
Speaker:like
Speaker:what's that?
Speaker:And it doesn't feel like two separate books.
Speaker:Right.
Speaker:we, we worked really hard to keep the voice consistent.
Speaker:and, you know, that was, you know, Mike is clearly way smarter than I am, so I just
Speaker:have to use lots of big words when I talk.
Speaker:So, but yeah, so that's, that's, why Mike has been visiting us so much
Speaker:on the podcast and will continue to do so for the foreseeable future.
Speaker:Today we're, gonna talk about another cyber related topic.
Speaker:And Mike, why don't you start us off with a story.
Speaker:there's, there's so many good stories, but one of the stories we'll talk about is, a
Speaker:healthcare, healthcare related, because.
Speaker:they are one of the high, highest, targeted industries,
Speaker:for a number of reasons.
Speaker:one, it's easy to leverage them if you can bring the hospital down and, you want to.
Speaker:You wanna get paid in return for restoring hospital operations.
Speaker:That's, that's good leverage.
Speaker:patient data is pretty valuable, both from a identity theft perspective,
Speaker:but then believe it or not, big pharma and some of these other less than
Speaker:reputable companies are willing to pay for patient like oncology reports
Speaker:and like, what could we sell this person, or what data could we use to.
Speaker:I.
Speaker:Promote our new drugs or, or procedures.
Speaker:So there's a lot of reasons for that.
Speaker:But nonetheless, university of, California, San Francisco, u
Speaker:U-C-S-F-U-C-S-F.
Speaker:you know, in an industry full of acronyms, sometimes I get 'em wrong.
Speaker:so University of California, San Francisco, got hit with ransomware.
Speaker:When was this?
Speaker:how it happened, but.
Speaker:when,
Speaker:Yeah.
Speaker:it's probably like five, five years ago,
Speaker:Yeah.
Speaker:maybe.
Speaker:which is, which is another, another.
Speaker:of the, of discussion is, you know, the, even, even though it's been
Speaker:five years, the, the way bad guys compromised them and, and how that
Speaker:whole thing went is still current.
Speaker:it's still stuff that happens today and yet, you know, tomorrow, you
Speaker:know, it's nothing, nothing that we've done in the last five years
Speaker:have has helped mitigate this.
Speaker:And in this particular case, it was, what we believe or they believe
Speaker:to be, compromised credentials.
Speaker:so it wasn't a phishing attack.
Speaker:It wasn't, you know, bad guys, you know, truly kicking down the door.
Speaker:it was, it was likely remote access or VPN credentials that were,
Speaker:were harvested some other way.
Speaker:whether they were easily guessed or, you know, former employee didn't
Speaker:get turned off, that kind of thing.
Speaker:so ransomware group got access, deployed their ransomware, and, you
Speaker:know, people started having a bad day.
Speaker:The good part of that though is, I think the topic of today,
Speaker:which is network segmentation.
Speaker:this environment was very well segmented for.
Speaker:Any number of reasons.
Speaker:One of them could just be compliance, hipaa, HIPAA compliance
Speaker:and healthcare, strongly advises.
Speaker:If not, in some cases, when with regard to confidential or sensitive
Speaker:information like healthcare records, requires, segmentation, this group.
Speaker:Did very well at segmenting their core IT environment from their,
Speaker:healthcare operating environment, from their lab environment.
Speaker:And so when, when one part of that network got infected, it
Speaker:was contained that segment.
Speaker:Mike,
Speaker:and that's one of the reasons why this DISEN environment
Speaker:didn't completely shut down.
Speaker:and Mike, for some of our listeners who may not be as familiar with the
Speaker:network side of things, could you provide a bit more context around
Speaker:like, what is network segmentation?
Speaker:Why is it important, that sort of thing as we walk through this.
Speaker:Sure.
Speaker:and there there's a, there's a number of analogies we can use in the,
Speaker:in the book we talk about kind of comparing it to a warehouse where
Speaker:you'd have different sections of the warehouse, even if it's just two.
Speaker:You've got the office air condition section, and then The warehouse
Speaker:on air conditioned section.
Speaker:in my case, when I worked at a warehouse, at least the bathroom
Speaker:was in, was air conditioned.
Speaker:So that's where I went to take my naps.
Speaker:But, in, in company environments, if you can, if you, and, and it's not just for
Speaker:security, it's also for management and maintenance and, and bandwidth control.
Speaker:I mean, there's a lot of benefits to network segmentation, backups, And
Speaker:there's different ways to approach it.
Speaker:So there's strategy behind this too.
Speaker:It's not just, you know, marketing has its own segment, or the first
Speaker:floor is a different segment than the second floor, or your, your
Speaker:production network is a different segment than your voiceover IP network.
Speaker:idea is how do we.
Speaker:Organize our environment from an operations and management perspective.
Speaker:So now we, we can truly measure and manage network traffic,
Speaker:anomalies, troubleshooting.
Speaker:from a. perspective whenever we create these segments.
Speaker:We can also apply access control rules depending on the hardware you have, the
Speaker:network hardware, so routers and switches.
Speaker:we can, we can control who can.
Speaker:Who can access that segment.
Speaker:so marketing or, or, you know, maybe, maybe we put a HR and
Speaker:legal in their own segment.
Speaker:So in payroll, you know, maybe accounting.
Speaker:So the rest of the company, you know, that doesn't need to worry themselves
Speaker:about paying the bills and, HR records and, you know, legal holds on, on data.
Speaker:all that stuff is, is truly protected.
Speaker:Or depending on your business, whatever your, your prize drill, your, your.
Speaker:Whatever your prized jewels are, you know, your data, your, logs, your fi,
Speaker:whatever it is, intellectual property.
Speaker:Put that in a secure segment or container, you can think of it that
Speaker:way too, and restrict who has access to it and what can happen, at, at
Speaker:a lot of different levels, right?
Speaker:It's not just, it's not just the user, it's also the type of network traffic,
Speaker:the services, the, the protocols, the, you can even limit bandwidth.
Speaker:there's a lot you can do, when, when you look at that
Speaker:type of, network architecture.
Speaker:Strategy.
Speaker:They think that
Speaker:probably easily, easily associate with sort of wifi
Speaker:networks, having like the guest
Speaker:the
Speaker:that you hand out to
Speaker:hand out to every client,
Speaker:they get.
Speaker:No
Speaker:get no access to.
Speaker:other than the internet versus like people who might be logging into
Speaker:like the corporate network or their home network where they have access
Speaker:to devices and they're streaming things and other things like that.
Speaker:And guess what?
Speaker:For those of you that're like, I will absolutely never do network segmentation.
Speaker:If you have a. Wireless network at home, and you've, you've separated, you know,
Speaker:you know when, when, when your neighbor's kids come over and they can't, they
Speaker:can't access your, your, your secret lab.
Speaker:You've only given them internet access.
Speaker:You've done segmentation.
Speaker:That's what I
Speaker:Interesting.
Speaker:it does cause issues sometimes and
Speaker:Yeah.
Speaker:we'll talk later about firewall and how do you connect these
Speaker:segments and all the rest of that.
Speaker:But yes, it does lead to some issues.
Speaker:Yeah, we have, we have tenants here at the house, like we have,
Speaker:we are renting rooms out and I created a guest network for them.
Speaker:And then of course they wanted to print and I was like, dang it,
Speaker:printer's on the wrong segment.
Speaker:which meant that either I have to give them access or I have to, I have to go on
Speaker:the guest network to print whichever, you know, one, one or the other is gonna work.
Speaker:so.
Speaker:And, and by the way, luckily this recording was segmented because I
Speaker:just got a call from my daughter and I was able to take that without,
Speaker:destroying the rest of the recording.
Speaker:You know, if you, you know, your, your adult daughter calls, you know, you
Speaker:got, you gotta take the call, right?
Speaker:That doesn't happen all the time.
Speaker:so let's talk about, hang on one second.
Speaker:Yeah.
Speaker:Okay.
Speaker:So am am I right in assuming Mike, that the key.
Speaker:tool in network segmentation are VLANs.
Speaker:The, the key tool is network, a network devices that would support segmentation.
Speaker:All right.
Speaker:Yeah,
Speaker:old school way is still just as effective.
Speaker:It's just not as easy.
Speaker:It's not a, you know, it's not an interface that I can log into and
Speaker:just drag and drop and click buttons.
Speaker:I've
Speaker:right.
Speaker:line, right?
Speaker:So
Speaker:So let, well, so let's, let's go back.
Speaker:Let's go back to that old school way.
Speaker:The old school way.
Speaker:It was literally, you gotta switch for this, you gotta switch for this,
Speaker:and never the twain shall each.
Speaker:Is that what you're saying?
Speaker:And that, and that.
Speaker:Now we have, VLANs.
Speaker:What, what is a vlan?
Speaker:Well, a VLAN is a capability of a switch.
Speaker:So newer, newer switches, and I say new, but the, you know, the
Speaker:VAN capable switches have been around for 10 or more years.
Speaker:New to those of us with with gray in our beards.
Speaker:So the, the older switches still could do segmentation, but you would have to.
Speaker:You know, you, you'd plug a console cable into the back of the switch and
Speaker:you, you would bring up a, a command prompt and you would have to physically
Speaker:type and know what to type the commands and the, the configuration of, of
Speaker:the segments that you want to create.
Speaker:And, you know, heaven forbid, now we've gotta add security with
Speaker:access control and other things.
Speaker:That's more typing and more things you have to know and, and.
Speaker:You know, it, it prone to mistakes.
Speaker:and from a backup perspective, also something you would want to back up.
Speaker:So if that switch died and you need to put a new switch in, you've got a a, a backup
Speaker:of, of that config for all those segments.
Speaker:'cause if you don't, then your network is broken and everybody's upset.
Speaker:the newer switches, newer even, you know, being around 10, 15 years, it
Speaker:you, you still log into the switch, but you can do it over the, the network.
Speaker:You don't have to have a console cable.
Speaker:You just, you hit the IP address, you get a nice user interface like a webpage, and
Speaker:there are tabs and buttons and to fill in.
Speaker:And as you interact with this interface, it's writing all that code in the backend.
Speaker:To create these VLANs, and now it's, now it's a, it's a graphic, you know,
Speaker:dashboard that shows you all your VLANs.
Speaker:You can name them, you can apply restrictions, you can do reporting.
Speaker:I mean, it's, it's, it's pretty, it's pretty easy to use.
Speaker:And so with VLANs, right, you're basically creating those segments that you have.
Speaker:Talked about earlier, right Mike?
Speaker:So you might have a say a VLAN N ID of 10, which is your production network.
Speaker:Maybe a VLAN ID of 20, which is your guest network, maybe a VLAN n
Speaker:ID of 40, which is your HR network.
Speaker:And in most cases, once you configure it, VLANs should not cross
Speaker:unless you give access for them to communicate with each other.
Speaker:Now, I know there are some systems out there where once you create a
Speaker:vlan, it allows traffic by default, which isn't always the best approach.
Speaker:Right?
Speaker:It's, I know Curtis, we've talked a lot about sort of, okay, shut everything
Speaker:down and then sort of add things back, and so I think from a VLAN perspective,
Speaker:yeah, you want those isolated except for the things which should be
Speaker:allowed to talk across each other.
Speaker:You're right.
Speaker:And, I think, I think the, the majority of devices that, that support
Speaker:VLANs do, Unrestricted by default.
Speaker:So it it's gonna start with, everybody can talk to this segment.
Speaker:And so, and, and I don't think we've, you know, VAN stands
Speaker:for Virtual Local Area Network.
Speaker:and so when we talk about segmentation, you know, within this, this
Speaker:company network, got the local area network, the land, and when you
Speaker:create a segment by itself, it's not necessarily a different land.
Speaker:When you restrict it, it becomes kind of a lan.
Speaker:so because you've gotta have access to it, just like you would the
Speaker:normal, the normal company network.
Speaker:Well, very similarly when we talk about VLANs, because it's virtual,
Speaker:I'm not adding new hardware.
Speaker:I'm just adding.
Speaker:Or taking advantage of the capability of this new switch,
Speaker:this new router, this new gear that allows me to virtually configure.
Speaker:And the way that works is, you know, if, if you can think of a network device
Speaker:and it's got all the, the, the plugs, the jacks where you can, you know, plug
Speaker:in a network cable, you know, whether it's, you know, eight or 16 or 32.
Speaker:When you, when you go into the VLAN console, it shows you a picture of the
Speaker:front of this device or the back where all these jacks are, and you just, you click
Speaker:on the ones that you want in this vlan.
Speaker:So it's both physical and virtual.
Speaker:Virtual from the perspective that it's applying logic to the traffic within the
Speaker:device, and then physical on the front end where, know, if I, if I want something
Speaker:on VLAN N one, I just need to plug it into whichever JAKs on the front of
Speaker:this device I've assigned to VLAN N one.
Speaker:Yeah, and there are also the capabilities, I know you were talking about the
Speaker:ports, Mike, where you can have actually multiple VLANs assigned to a single port.
Speaker:So you could imagine the case where you have a wifi access point, which is.
Speaker:broadcasting your guest network, your production
Speaker:network, your HR network, right?
Speaker:You could basically have all three VLANs come in on one physical
Speaker:port, but it, that port would support all three of those VLANs.
Speaker:So it's not always sort of a one-to-one physical to VLAN mapping, but you could
Speaker:have multiple sharing, a same, port.
Speaker:Yeah.
Speaker:And I, I, I was trying to think of, of a scenario where that was, you
Speaker:remember you were trying to talk about the, the, the, the wireless one there,
Speaker:but I was trying to think of a, of a different scenario where I would want
Speaker:an individual port, you know, which is gonna talk to another device, right.
Speaker:to be on more than one vlan.
Speaker:Can you think of a, of another scenario besides wireless?
Speaker:Well, you can think of the case where you're doing an uplink from one port
Speaker:to another, or sorry, sorry, from one switch to another, or from one
Speaker:switch to like an aggregate switch.
Speaker:So you need to be able to transfer all of those VLANs from that need to be.
Speaker:Transmitted from one switch to the other.
Speaker:And so you would have multiple VLANs on a single port.
Speaker:Yeah, that actually that's a, yeah.
Speaker:Yeah, that's a really good, that's a really good analogy.
Speaker:Thanks.
Speaker:That because of, because of vlan, and again, for those that not familiar
Speaker:with the VA VLAN isn't, like you said, it's not limited to a port, but
Speaker:it's also not limited to a switch.
Speaker:Right?
Speaker:So a VLAN could be across switches, so then you've gotta have inter
Speaker:intercommunication, but then you've gotta allow that communication to go.
Speaker:So, yeah.
Speaker:So that's, and then you might, like you said, you might
Speaker:trunk multiple ports together.
Speaker:When you're doing, inter switch communication, the, so what's, let's
Speaker:talk about, so that I, so I still go back to my original statement, that
Speaker:VLANs or the principle manner in which we're gonna implement this, but.
Speaker:It's like virtualization.
Speaker:Yeah.
Speaker:Yeah.
Speaker:I get it.
Speaker:I get it.
Speaker:I just, you know, anyway, so let's talk about, the need to talk principle.
Speaker:You wanna talk about that, Mike?
Speaker:what do, what do we mean when we talk about the, the need to talk?
Speaker:So, yeah, the need to talk principle is, is similar to at least privilege.
Speaker:is just a, it's a good security, strategy.
Speaker:The need to talk or the, the need to access, however, however you may have
Speaker:heard it, or, or you can think of it, is the whether or not it's appropriate
Speaker:for an end user device like a laptop.
Speaker:to be able to connect directly to a, a production server.
Speaker:and there's a variety of, of reasons to determine whether that's appropriate.
Speaker:but it's also an exercise you have to go through, which a
Speaker:lot of organizations don't.
Speaker:so identifying your critical assets, determining and based on the, that.
Speaker:That criticality, you can classify them confidential, public, et
Speaker:cetera, and then determine what's appropriate from an access perspective.
Speaker:In this case though, we, we were talking, we opened the, the segment
Speaker:with a, an example of ransomware.
Speaker:Imagine a, a user's computer.
Speaker:Got ransomware and they had a, a network, you know, there was a, a map drive.
Speaker:you know, when whenever they turn their computer on, they look at,
Speaker:you know, my, you know, the file explorer or my computer, and there's
Speaker:your S drive or your U drive.
Speaker:And that is an automatic, you know, scripted to a production server
Speaker:based on this user's credentials.
Speaker:Well, that ransomware now has.
Speaker:Those users credentials and would also, by association, have access
Speaker:to everything that this does, this device and that user is mapped
Speaker:to like those production servers.
Speaker:And so even though it's difficult and it's, it's a, it's a burden like security
Speaker:is automatically scripting access to production systems is, is frowned upon
Speaker:It's
Speaker:Say that again Automatically.
Speaker:Script automatically scripted, authentication to production
Speaker:Oh, okay.
Speaker:I understand what you're saying.
Speaker:Gotcha.
Speaker:Right.
Speaker:So, and, and I, I, I can see that because if, if we think about it, would you
Speaker:agree that end user, end user devices are probably the ones at most risk
Speaker:of being infected with ransomware?
Speaker:Right.
Speaker:Would, is that?
Speaker:Seem like a fair.
Speaker:Yeah.
Speaker:Okay.
Speaker:Yeah, because they're the ones, like they're taking their laptop and they're
Speaker:going to Starbucks and whatever.
Speaker:Right.
Speaker:And so they're, they're getting infected in that scenario.
Speaker:And so this is why you have this concept of not allowing them to
Speaker:directly communicate with servers unless there is a reason to do so.
Speaker:Yeah, server servers are not proactively or, or, you know, mindlessly clicking on
Speaker:links and opening emails with attachments.
Speaker:Yeah, exactly.
Speaker:I mean, we, we start deploying AI agents to do, to do more.
Speaker:You know, that may, that may, there may be a phase we go through.
Speaker:but currently, yeah, it's, it's
Speaker:Where ser, where servers get bored and they start browsing the web.
Speaker:So, so I was just, as you guys were talking through this, I was thinking
Speaker:about the episode, I think a couple weeks ago maybe we had, where we were talking
Speaker:about sort of users having admin access
Speaker:Mm-hmm.
Speaker:on their systems, right?
Speaker:And it's very similar in this case, right?
Speaker:It's like, hey, it's easier to just be like, yeah, users get
Speaker:access to production, whatever.
Speaker:I don't need to worry about.
Speaker:authorizing access every single time someone needs it.
Speaker:Maybe my IT shop is a little lazier, doesn't want to deal with
Speaker:these issues, or the end users want certain, privileges, I guess.
Speaker:And so it's sort of a bad design to say, Hey, all the users can run as.
Speaker:route or admin on their personal laptops.
Speaker:I think in the same way here, it's sort of like, okay, if you let them
Speaker:run with full access to production, that may be a risky maneuver.
Speaker:Same thing.
Speaker:It's less work.
Speaker:Just like you're saying, Mike, it's less, it's more work to do what we're saying.
Speaker:It's more work to segment, it's more work to say servers can't talk to
Speaker:laptops unless there is a particular use reason for them to do so, and they
Speaker:can't talk to, you know, mobile phones and, you know, all of that kind of stuff.
Speaker:but yeah, go ahead.
Speaker:Well, I just, well, but it's a good idea to do so, just like everything that we
Speaker:talk about, literally, there's nothing that we say in any of this, any part
Speaker:of this book where it's like, you know.
Speaker:Here's the thing that you could do that has no effort and great
Speaker:level of reward right there.
Speaker:There's just, there's everything we say.
Speaker:It's like, you should probably do this.
Speaker:There is a great amount of reward, right?
Speaker:a great reduction in risk.
Speaker:Imagine, you know, just like, again, another previous episode,
Speaker:not that long ago we talked about.
Speaker:If you don't have business with China and Russia and other similar
Speaker:countries, then just don't let servers in anyone in, in Russia or
Speaker:China communicate with your servers.
Speaker:Just turn it off and boom.
Speaker:Takes a little bit of effort.
Speaker:Huge amount of reward in terms of making sure you're not gonna get stuff from that.
Speaker:Of course, what those guys are gonna do, just hop on A-A-A-V-P-N
Speaker:and pretend like they're in the us.
Speaker:But you know,
Speaker:Yeah,
Speaker:we're, we're, we're trying to stop the stupid bad guys, not the.
Speaker:So,
Speaker:Not the smart ones.
Speaker:about the effort, right, Curtis, and we understand that managing, managing
Speaker:these environments, especially as new applications are spun up, right?
Speaker:New systems are brought on board.
Speaker:It's complicated, right?
Speaker:You have a network admin who's probably very overwhelmed or an IT generalist, and
Speaker:yes, the tools may be easier to configure it and versus what they used to be,
Speaker:but it's still work, it's still effort.
Speaker:You still need to monitor all the rest of that.
Speaker:And so.
Speaker:are managing these rules in order to allow those access.
Speaker:And Mike, in some of these environments, and I know you're just gonna say
Speaker:it depends, like how many firewall rules do you see sometimes in these?
Speaker:Like is it like tens to hundreds?
Speaker:Is it thousands?
Speaker:I'm sure it varies significantly, but like these are rules that someone
Speaker:created, someone has to manage right?
Speaker:You're right.
Speaker:I have seen quite a, quite a variety, if not the whole spectrum of like.
Speaker:Just default rules and because we don't know, and then just
Speaker:overly crazy cumbersome rules that actually cause problems.
Speaker:and then somewhere in between, you know, there there's also layers.
Speaker:So you've got your perimeter firewall and you've got, you know, internal firewalls.
Speaker:You've got firewalls for specific applications or servers or segments.
Speaker:there's a variety of strategy and architecture thought you
Speaker:can put to deployment of.
Speaker:Things at multiple layers, firewalls, one layer, segmentation's, another layer.
Speaker:yeah, there's, there's, I've seen, I've seen a lot.
Speaker:I've seen it.
Speaker:I've seen it.
Speaker:Good and the bad.
Speaker:And, and not to say one's better than the other, it depends on the,
Speaker:environment and, and the organization and,
Speaker:There's probably a point.
Speaker:Go ahead.
Speaker:Go ahead.
Speaker:Finish Mike.
Speaker:And the people and skills that, that you need to manage it.
Speaker:There's probably a point of decreasing marginal returns where like, you know,
Speaker:with, you know, with 50 rules, you get, you get this much, but with 400
Speaker:rules, you get this, this much more.
Speaker:The, the one that I saw, the, an environment that I was at where
Speaker:they had all of their applications stored very sensitive information.
Speaker:Right.
Speaker:And they had ano and they were, they were publicly faced, publicly
Speaker:facing applications, like to the public via the internet.
Speaker:And what they did a really good job of is segment, you know, basically vertical
Speaker:segmentation within their environment.
Speaker:So if you were interfacing with this app, you got access, you and
Speaker:the, the application that you were talking to and everything that
Speaker:that application needed to talk to.
Speaker:Was all available to that application.
Speaker:But if you were right next door and a server literally in the
Speaker:next rack, and you were talking to that application, that server
Speaker:couldn't talk to this server, right?
Speaker:So that I, I really like that now.
Speaker:It didn't end up creating.
Speaker:An incredible pain in the butt when I, the crazy backup guy
Speaker:wanted to talk to all the servers.
Speaker:That was considered like really verboten at the time.
Speaker:and, it led to a lot of fun, which, I've told a story about.
Speaker:Go ahead.
Speaker:I thought that was a
Speaker:Yeah, it's the, it's the same story.
Speaker:It's the same story.
Speaker:The one that results in me losing my, you know what, and
Speaker:At late
Speaker:out obscenities late at night.
Speaker:Yeah.
Speaker:Yeah.
Speaker:Mm-hmm.
Speaker:and Mike, I want you brought up a very interesting point, in your last
Speaker:comment, which was the skills of people.
Speaker:Right.
Speaker:And I think what ends up happening is a lot of what we talked about has.
Speaker:So far been mainly about like on-premises networking infrastructure
Speaker:to the most part, right?
Speaker:When we talk about physical switches, everything else, once you start throwing
Speaker:in cloud and the various ways that they protect their networks, the perimeter,
Speaker:their virtual data centers, whatever you wanna call a vbcs and AWS's case, right?
Speaker:All of that now adds a layer of complexity, and
Speaker:now when you try to overlay.
Speaker:settings, those rules would say something that also runs on premises
Speaker:and communicates back and forth.
Speaker:Now it gets very, very complicated, very quickly.
Speaker:It does.
Speaker:I'll, I'll, I will add part of this conversation that virtualization has
Speaker:so many benefits and security is one of those, if something's compromised, you
Speaker:just blow it away and, you know, go back to the most recent snapshot if you're, if
Speaker:you've got a good strategy around that.
Speaker:but for sure, whenever you're, you're talking cloud, you know, someone else's
Speaker:data center, whether it's co-located or truly outsourced, You're relying on all
Speaker:the controls and capabilities and and skills of the people supporting that.
Speaker:That, that you don't have control over it, aside from your, your
Speaker:contract, and then all of the communications between you and them.
Speaker:So how are you syncing?
Speaker:How are you sending and receiving?
Speaker:how do we, how do we ensure that access is appropriate and how
Speaker:are we monitoring all of that?
Speaker:and so one of the things I was.
Speaker:Touch on just from our last, you know, just the last thread, was
Speaker:the more complexity you add to your environment, more overhead it's gonna
Speaker:take to make sure, well one, how do we troubleshoot what's going on?
Speaker:What is anomalous?
Speaker:How do we go through these different layers to figure out what happened?
Speaker:And then back to the skills, you've gotta have the right people.
Speaker:and, and sometimes those.
Speaker:Those people will tell you, you need all this stuff.
Speaker:It needs to be this complicated thing.
Speaker:which maybe that's because of where they came
Speaker:Shiny new toys.
Speaker:it in that other environment, right.
Speaker:Or, sometimes the, you know, selling you this overly complex thing is, is their
Speaker:way of, sometimes overselling themselves.
Speaker:So a word of caution there too.
Speaker:the more, the more layers and things you put in place does not
Speaker:necessarily mean that you're, you're more protected or that your
Speaker:operations are gonna be more reliable.
Speaker:in, in the majority of cases that I've been involved with,
Speaker:it's actually been the opposite.
Speaker:The more complicated something is, the, the more difficult it is to
Speaker:respond and truly analyze things.
Speaker:It's also more difficult to manage it and keep it, keep it, keep it up and running.
Speaker:Glad you brought that up because the next topic that I wanted to talk under
Speaker:this, and it's related to the thing that persona just mentioned, which is the
Speaker:concept of microsegmentation, right?
Speaker:Where we're not just limiting things to VLANs and things like that, but
Speaker:we're also saying, this application can only talk to this application, or
Speaker:this applica, this piece of storage can only talk not just to this server,
Speaker:but especially in a world of the cloud.
Speaker:the where I, for example, have seen this when, you know.
Speaker:Persona and I worked, used to work at a cloud backup company and they, they
Speaker:used S3 as their target and they had, they had S3 configured so that only
Speaker:their application could write to that S3 write and read to that S3 bucket,
Speaker:so that even if you somehow had managed to break through all of the levels
Speaker:of security, get to get to S3, it wasn't, it you, you wouldn't be able to
Speaker:actually, read or write because they had.
Speaker:Pre preview already configured it so that it could only talk
Speaker:to the appropriate application.
Speaker:So I, I like the idea of microsegmentation, but Mike, I, this
Speaker:idea, you know, what you're saying is, is so true, is we can configure
Speaker:it just like everything else.
Speaker:We can configure this till the cows come home.
Speaker:And we can configure it so that it's so good that A, nobody can understand it.
Speaker:And b, we can't troubleshoot it when things, you know, when we get a,
Speaker:when we get a trip, you know, that nobody understands why we, why we
Speaker:got it and why we keep getting it.
Speaker:the more complicated we make it.
Speaker:You, you know, even though we, we talk about this stuff a lot, right?
Speaker:We're, we're, we're always recommending you need to look into
Speaker:this, you need to look into that.
Speaker:But we still have to argue for simplicity, right?
Speaker:Simplicity or complexity.
Speaker:Equals risk, right?
Speaker:Doing nothing equals risk, but doing way too much equals risk.
Speaker:You, you have to find a balance between doing things that you can understand.
Speaker:And I do think, by the way, just that, you know, we haven't, it has been 30
Speaker:seconds since we've said the word ai.
Speaker:I, I do think this is an area where AI can help, where you can say,
Speaker:you know, potentially you can say, here are the goals that I want
Speaker:to have for this organization.
Speaker:And that potentially AI could make a much more complicated security,
Speaker:you know, security forward network and application segmentation that
Speaker:wouldn't be possible otherwise.
Speaker:And I'm really curious to know your thoughts on what I just said.
Speaker:The, the, that last part.
Speaker:I, I agree with it.
Speaker:I, I agree that AI should be a good.
Speaker:Resource, for organizations to assess not only what, what they,
Speaker:what they're thinking about doing, but also what they currently have.
Speaker:and don't forget, and I've said it a couple times,
Speaker:forget to include your people.
Speaker:Whether those are internal people or your contractors, don't, don't
Speaker:forget to include them in this analysis because that's critical,
Speaker:especially if it's something complex.
Speaker:One of the biggest risks to environments that have these complex, you know,
Speaker:segmentations and layers, it, it's usually one or two people that built
Speaker:that with almost no documentation.
Speaker:Hmm.
Speaker:and so if you lose those people, you, you're.
Speaker:You're in a, I don't wanna say world of hurt, but, that's, that's a lot of
Speaker:risk to bear for someone new to come in.
Speaker:And I've seen this a lot, unfortunately, or coincidentally, the last couple
Speaker:of environments where I've seen a new person come in, into an
Speaker:environment that's overly complex.
Speaker:It was usually, pretty close to an incident, ransomware or something else,
Speaker:and they're trying to figure stuff out, out while the house is on fire.
Speaker:And they're figuring it out, but they also don't have time to document any of it.
Speaker:So it's still not documented.
Speaker:But,
Speaker:Yeah.
Speaker:a lot of environments where all these great things are in place.
Speaker:more than you need, and that person, you know, walks off the job or isn't available
Speaker:anymore and there's people have no idea.
Speaker:One other thing I'm gonna a add real quick about microsegmentation and, and
Speaker:you touched on it briefly, Curtis, you know, this application can only talk to
Speaker:that backend database or that bucket.
Speaker:of times that authentication is hard coded somewhere.
Speaker:You know, it's, it's, it's saved credentials or cashed credentials, or it's
Speaker:in a script or, it's a service account.
Speaker:again, years later, know, we're doing some other kind of audit or
Speaker:assessment and we find these things and we're like, what's this for?
Speaker:And nobody knows.
Speaker:So we turn it off and things break, or we delete that account
Speaker:or, you know, we update the code.
Speaker:It's not documented.
Speaker:It's hard coded, it's bad practice.
Speaker:And in a lot of cases it's, it's really not necessary.
Speaker:There's other ways to do this than, than some of these one-to-one
Speaker:authentication, approaches.
Speaker:you say though, Mike, that that example you just gave where you're using
Speaker:some sort of credential or service account or whatever else, so talk to
Speaker:another between two services that.
Speaker:Network microsegmentation is an additional layer, like it's not an either or, right?
Speaker:It's just an additional benefit to the authentication such that if someone
Speaker:stole those credentials and say, try to access that server from a different
Speaker:machine or something else, that the microsegmentation can help at least
Speaker:isolate and protect that endpoint rather than leave it completely wide open.
Speaker:Sure.
Speaker:and, you know, I, I got, I got pretty granular with, with credentials,
Speaker:but a step up from that is just, you know, the trust relationship.
Speaker:Is it one way trust, is it bilateral trust?
Speaker:I mean, that's been around forever.
Speaker:and those are appropriate.
Speaker:'cause that's just, that's a setting, you know?
Speaker:Right.
Speaker:So it that, that's not credentials.
Speaker:That's a, that's, you know, tokens or, Berros ticket or whatever.
Speaker:that's a setting.
Speaker:And, and those are appropriate.
Speaker:And those, those are, you know, very prevalent and.
Speaker:And, you know, good best practice if you can put the time into making sure it's
Speaker:set up right and main and documenting it.
Speaker:but no, you're right.
Speaker:I think, if, if you can start with network segmentation in general, put your
Speaker:production environment its own segment.
Speaker:Determine how to restrict or what's appropriate for other devices and
Speaker:users, to access that segment.
Speaker:And the, the behavior, like what does the network traffic look like?
Speaker:What do the trusts look like?
Speaker:in a lot of cases, you know, the majority of cases, that approach is gonna satisfy.
Speaker:Your objectives, your security objectives, your resilience, your backups, your,
Speaker:you know, protection from ransomware.
Speaker:that's gonna do the majority of the, of the heavy lifting.
Speaker:If you, if you really have sensitive systems, you know, maybe you're
Speaker:A-A-A-C-P, a firm and you, you've got all this tax data, you can do more, you
Speaker:can, I, I don't know that, you know.
Speaker:The, the very technical approach to microsegmentation that we've been
Speaker:talking about is necessary, but there's other security approaches, you know,
Speaker:with, dongles or tokens, you know, the UB keys and some other things,
Speaker:that could probably be more user friendly and manageable than
Speaker:the very technical approach to microsegmentation that, that would
Speaker:take some skill and, and some strategy.
Speaker:And it's
Speaker:I. Yeah, I, I think I like this idea.
Speaker:Again, you know, we're in sort of the action items part of, of the, of the
Speaker:recording here, and I like this idea first just sort of segmenting production from
Speaker:test dev and from end user devices, right?
Speaker:If we could create those three segments and then, I'd say the next step past
Speaker:that is when you look at your production environment, most applications are multi.
Speaker:They're, they're, you know, you start with like a web server in the front end.
Speaker:Then you have a database server, an application server behind that,
Speaker:and whatev whatever you've got.
Speaker:You might, you might have a whole number of things behind, but you could
Speaker:say, well, I, my, my devices only need to be able to talk to the web server.
Speaker:They don't need to be able to talk to all these other service.
Speaker:This server needs to be able to talk to those servers.
Speaker:So you can start with, with sort of a basic thing like that.
Speaker:Like you said, I like the idea of separating the, the dev and test.
Speaker:There's no reason that the dev and test need to talk to production.
Speaker:well, there's one reason I can think of, but again, that's a,
Speaker:it's a, it's an occasional reason.
Speaker:and I like the idea of restricting the end user devices, in their own
Speaker:little, you know, land ghetto, right?
Speaker:So that they're not allowed to, you know, talk to things.
Speaker:Again, anytime you do this, you just have to.
Speaker:you know, we've said this before.
Speaker:You need a, you need a, it, it's an on high thing.
Speaker:You need support from the people above you, because once you start doing
Speaker:this, you're gonna step on some toes.
Speaker:You're gonna, you're gonna hurt some apps and you're gonna, and, and so what you
Speaker:do, you is the thing of like, we turn on the new feature and then we wait.
Speaker:Right.
Speaker:We wait for, we wait for people to call the, to call the thing.
Speaker:Okay.
Speaker:It's been 15 minutes and nobody's called.
Speaker:Right?
Speaker:again, I, I'll give you a story from, from back in the day.
Speaker:There was this server, it was called Snazzy.
Speaker:I still remember the day.
Speaker:It was a little HP UX box, this little, little tiny thing.
Speaker:And, I had been told a lot of times backups would break on this
Speaker:box, and I, I didn't know why I, I was a brand new Unix guy.
Speaker:Right.
Speaker:And I was told, that when that happens, you just reboot the server.
Speaker:Right.
Speaker:And so one day this was happening, and so I rebooted the server and then all
Speaker:of a sudden, whoop, whoop, whoop, whoop.
Speaker:You know, people are literally running into the server room.
Speaker:They're like, what's going on with snazzy?
Speaker:I'm like, it's, it's rebooting.
Speaker:They're like, that's our communication.
Speaker:That's, that, that is the server through which we communicate to the
Speaker:mainframe in Dallas, you know, the, the mainframe that has all our money
Speaker:and we're a credit card company.
Speaker:Yeah.
Speaker:So you, you know, you, you're gonna break things, but you need to be prepared.
Speaker:The, the management needs to prepare and they need to support you.
Speaker:We're gonna do this, we're doing this for a reason, and we're doing this, you know,
Speaker:in a way that, we're trying, we're trying not to step on as many toes as we can.
Speaker:But, anyway, I'll step down on my soapbox.
Speaker:like you said, you do it in a phased approach, right?
Speaker:Mm-hmm.
Speaker:those four groups, right?
Speaker:Production, right test and dev end users.
Speaker:And then you could honestly just leave everything open on your
Speaker:production and then over time, just start locking it down,
Speaker:Yeah.
Speaker:And you monitor right?
Speaker:Leave it open and monitor.
Speaker:Right?
Speaker:Mike, can you talk about, about that a little bit?
Speaker:How do we monitor I vlan traffic.
Speaker:How do we figure that out?
Speaker:Well, there's an interface for that.
Speaker:Yeah.
Speaker:and then there's, there's, there's an, there's a variety of tools,
Speaker:both free, like Spiceworks or SolarWinds From a network operations
Speaker:perspective, you know, you can, you can monitor a, a number of attributes
Speaker:of your network across those VLANs.
Speaker:'cause it's, it's essentially.
Speaker:at traffic and whether you have that inside a, a segment or enterprise wide
Speaker:firewall all the way out to the perimeter.
Speaker:but that's really just network attributes.
Speaker:so bandwidth, thresholds on packets, users, machines, health.
Speaker:Someone trying to print, across different, segments.
Speaker:you can see all those protocols, all that good stuff from a
Speaker:cybersecurity perspective.
Speaker:Very similarly, you would put a a, a collector in each, within
Speaker:each segment that doesn't have.
Speaker:Trust.
Speaker:So any, any truly segmented, part of your network, it would have to
Speaker:have its own We call 'em sensors.
Speaker:And then you would push all the cis log and agent data to that collector.
Speaker:That collector would then meet up with other collectors and consolidate
Speaker:before it goes out through the firewall to a, a cloud data lake.
Speaker:And that's where
Speaker:That,
Speaker:run all this analysis
Speaker:that be a collector of collectors?
Speaker:It would be a collect a collector collection.
Speaker:A
Speaker:Electric collection.
Speaker:Yes.
Speaker:All right.
Speaker:Well, hey, I think that, I think this been good recording.
Speaker:we got, you know, we get in the weeds on, on how to do this and obviously
Speaker:the implementation is gonna vary from, environment to environment right.
Speaker:And how you actually do the dragging and dropping.
Speaker:but and I think Mike is recommending that everybody just go back to
Speaker:old school lands, not VLANs, just.
Speaker:Sports on a switch?
Speaker:no ai, no v anything.
Speaker:just,
Speaker:not recommending that.
Speaker:back in the day.
Speaker:I'll tell you, I'll tell you, I'll tell you the new, the new generation of hack.
Speaker:You know, the older guys are, are, you know, they've, they've, they've
Speaker:finally filled their nest egg and they're on a beach somewhere.
Speaker:or they moved outta, they finally moved outta mom's house.
Speaker:But, You know, a lot of the newer guys don't know how to hack some of the legacy
Speaker:stuff, you know, back, you know, old IBM, you know, big blue hardware, Novell,
Speaker:they, they have no idea how to hack a Novell, even though there's, there's
Speaker:good stuff out there for how to do it.
Speaker:so I'm not, I'm not, I'm not saying, you know, simple network, flat network
Speaker:is bad as long as it's appropriate for your environment, and your risk profile.
Speaker:But yeah.
Speaker:any number of things, you know, segmentation, load balancers,
Speaker:firewalls at the application network, even operating system level.
Speaker:there's a ton of resources out there.
Speaker:You just have to do the, an analysis.
Speaker:You have to know yourself and what you're trying to do, what your objectives
Speaker:are, and where everything is that inventory, and then figure out the
Speaker:best approach to protecting it all.
Speaker:Well, persona, thanks again.
Speaker:you got to, got to show off your VLAN expertise there.
Speaker:That this is what I've been doing at home the last couple years.
Speaker:Curtis
Speaker:gotta try things out for once.
Speaker:All right, and once again, thanks again, Mike for, for being on the pod.
Speaker:You are welcome.
Speaker:All right.
Speaker:And thanks to our listeners, you know, you are why we do this.
Speaker:I hope you're enjoying this.
Speaker:you know, give us a comment, give us a shout out, recommend the pod to
Speaker:other people, and, tell 'em to buy the dang book while they're at it.
Speaker:There you go.
Speaker:The Learning Ransomware Response and Recovery by w Curtis
Speaker:Preston and Dr. Mike Saylor.
Speaker:That is a wrap.







