Enterprise LAN Segmentation that works

If you've been around the enterprise security environment for any length of time, or even if you've just accidentally brushed up against someone who knows someone in the security world, you heard that you must segment everything! Flat networks are bad! Segment, segment, segment!

In this article I hope to share some strategies for creating segments in a sane and controlled way. Before we begin let me describe what I mean by segment. A segment is really a region of control. All devices in a segment have the same level of (dis)trust. Within a segment traffic is routed as normal (for the most part. I'll get into other options later). Between segments, traffic is passed to a Policy Enforcement Point (PEP) (aka a firewall).

The Setup

Now let's define some segments that we will probably see in an enterprise environment.

Corporate devices (we should see a bunch of those)
BYOD devices (this is really the only reason why we build wireless networks... right?)
Guest/Vendor (straight to the Internet with you!)
Building control (you have these. If you think you don't, then no one told you about these and their on the corporate network)
OT, manufacturing, etc.

This is not an exhaustive list, but most decent size companies will at least have these. Now for each of these segments a unique supernet should be allocated. This will help with route summarization and firewall rules. Separate VLANs will need to be created on every access layer switch stack and one IP range for each of these segments assigned. I recommend you avoid trying to size the subnet to the number of devices of each type. You'll end up just driving yourself nuts! Instead develop a standard subnet allocation per access layer switch/rack/closet and assign one network per segment per stack. Next you will need to create a separate VRF for each segment (if you are not sure what a VRF is and how to create and route with a VRF stop here, read up, practice, and then come back because VRF's are the heart of this design)

Now that we have VRF's created we need to route the traffic to somewhere. In this case that some where will be the PEP. You'll now have several choices on how to route the traffic. In order for least favorite to most...

The worst case you can static route the other/non-local/extra VRF's IP supernets (see, told you that summarization would help you) to the firewall. This option virtually eliminates any redundancy.
Utilize the default route. Assuming that this isn't also a static route, you can use the default route learned from the firewall to "pull" the extra-VRF traffic to the PEP.
Best case. The firewall and router are routing peers and the routes from the extra-VRF are learned via the firewall (for bonus credit the firewall should inject summary routes of the segment supernets).

Voila. You now have segments between different type of devices! The rest is up to security. You can get a cold one and... what, asymmetric routing? The security team wants multiple PEP devices in different locations! They are never happy are they!

Devil in the details...

Asymmetric routing

Nothing is free. This is doubly so with segmentation. Unless you can convince the security group to disable stateful inspection on the firewalls (thus turning them into routers) we will need to prevent asymmetric routing. Unfortunately the only practical way to do this is to designate one PEP as active and the other PEP(s) as backup. Utilizing routing metrics we will need to make the backup PEPs look less preferred than the active PEP.

Intra-segment routing

Though I don't recommend doing it most routers and access layer switch can implement an ACL to stop intra-segment routing. If this is done there are a few guidelines to follow.

keep the ACL's small. Remember that there is a limited amount of TCAM space on the switches and routers. When that space exhausted unexpected and truly bizarre things happen.
Updates to ACL's can stop the flow of all traffic as the TCAM's get programmed. This delay can increase as the ACL's get longer.
ACL's are stateless so pick a direction (IN the port or OUT the port) and write the ACL from that perspective.
There are a number of situation where an ACL will cause a packet to be punted to the CPU for processing which can cause high CPU and even dropped packets. Know what these situations are for your specific hardware and try to avoid them. It may be necessary to implement some sort of control plane policing (CPP) to prevent a DOS of the switch due to high CPU.

What about...

Private VLANs

In general I do not use Private VLANs (PVLAN) on client LAN segments. Mostly because there are side effects between PVLAN and NAC. On server segments where NAC is not used, I do use PVLAN to limit the blast radius of "at risk" servers.

VXLAN

VXLAN is a layer 2 over layer 3 technology (aka tunneling). It allows for keeping traffic separate based upon a VXLAN tag. In the same vein as GRE or mGRE. In and of itself there is nothing wrong with either technologies. However there is still a need to separate the L3 routing domain and have the traffic pass thru a PEP for policy evaluation and enforcement. There are many good reasons to run overlays, but it is not a substitute.

SGTs

Security Group Tags (or what ever Cisco marketing is calling today) seem like they can be a useful addition to further micro segment and environment. But they have a few draw backs. 1) they operate within the layer 2 ethernet packet so each device that moves the packet forward must support the tags. (or enable SXP which is a whole other article in itself) 2) when utilizing non-Cisco gear you're forced down the SXP route. But mostly I've never seen it work in a real production environment. (if you know of one where SGT's are deployed and working I'd love to discuss how it was done)

Wrapping Up...

There are obviously other ways to approach network segmentation. A somewhat extreme example might be Google's Beyond Corp model for a border-less network environment. It provides an interesting perspective and for companies whose primary services are all Internet facing, this may make a lot of sense.