The month of March has been an absolute blur. I'm still processing what just happened. With half the planet confined to their homes by COVID-19, IT departments are facing the greatest logistical challenge in history: sending the entire office workforce to telecommute from a Friday to a Monday.
We've emptied the laptop inventories of all suppliers and wrestled with software licenses, but the real technical drama has occurred at the heart of the corporate network: the VPN concentrators.
The bottleneck of the Hub and Spoke model
Historically, enterprise security has been designed like a medieval castle. You trust everything inside the walls (the office intranet) and block everything outside. So an employee from home could access a file server, we used a VPN (Virtual Private Network) that created a secure tunnel through the wall.
This model (known as Hub and Spoke) assumes that only 10% of the workforce is traveling at the same time. When 100% of the company connected their VPN clients on Monday, March 16 at 9:00 AM, the physical firewall boxes literally drowned.
The biggest design flaw is forced routing. VPNs were configured so that all the user's laptop traffic passed through the company's central servers. Thus, when an employee at home joined a Microsoft Teams video call or downloaded a PDF from an external SaaS, that immense traffic traveled to the office, saturated the corporate firewall's bandwidth, and went back out to the internet. We were strangling our own network.
# Quick fix configuration example (Split Tunneling)
# We tell the VPN client to only send traffic heading to our internal subnet (10.0.0.0/8)
# through the corporate tunnel, and let the rest of the internet go straight out
# through their home router.
route add 10.0.0.0 mask 255.0.0.0 192.168.100.1
# Everything else (Teams, Netflix, YouTube) bypasses the corporate firewall
Implementing emergency Split Tunneling saved our servers from melting down, but it has opened a security hole that will take us months to audit.
Reflection: The death of the perimeter
The pandemic has compressed ten years of digital transformation into ten days. Just as we realized years ago that the physical server was dead compared to the cloud, today the physical office has died as a security perimeter.
If your applications are in AWS and your employees are in their living rooms, passing through an appliance in an empty office makes no technical sense. The imminent future is the Zero Trust model (inspired by Google's BeyondCorp). Zero trust in the network you connect from, basing access purely on the user's cryptographic identity and the health of their device. The medieval castle has fallen; now every employee is their own fortress.