Planning out a Secure and Sandboxed KVM Hypervisor Network (KVM Network Tutorial, Intro)

But first, a quick background

Earlier this summer I received some unexpected news that left me with a bit more free time than I had anticipated. Of course, this post isn’t about something I didn’t do this summer, so I won’t get into the details of it. Since I had this extra time, though, I decided to look a little into home server options. I’ve tinkered around with hosting on a Raspberry Pi, and I felt like a bump in speed & memory would be nice.

As usual, this mild curiosity quickly spiralled into a thorough scouring of ebay & various classifieds, an hours-long research session on the small performance differences of various Xeon chips, and a snap-decision purchase. The model of choice was an IBM x3650 M3 with dual 3.47Ghz X5690 chips and 64 Gigs of RAM–it was a seriously good bargain for $150.

By the time the server arrived, I had some ambitious plans for it–hosting multiple websites with a TCP proxy, provisioning a Plex server for streaming movies, and setting up a few sandboxed VMs for malware analysis were but of a few of the projects I was planning to dive into. I was certainly planning on using its hypervisor capabilities to their fullest extent. Before all that, though, I wanted to make sure to have my network configured to minimize damage in case of a virus. Exposing ports to the internet at large always has its risks, and I didn’t want any infection spreading to my personal devices.

Tutorial Scope

In this series of posts, I’m going to cover exactly how I went about doing that–designing and configuring a Linux KVM hypervisor and router so that servers on the hypervisor can be fully sandboxed. Beyond that, I’ll also show how to set up a management network that allows access to each of these sandboxed networks while blocking access in the other direction.

This setup was accomplished using only an OpenWRT-supported home router, an IBM server running Ubuntu 20.04, and a few network cables. The Ubuntu server used KVM to run as a host hypervisor, though, so this series can realistically be applied to any Debian-based distro of Linux. Redhat-based Linux distributions will also work, though the network configuration files are formatted differently.

Network Concepts

Before we can begin to plan out our network, we need to first understand what constraints we’re working under, as well as what network tools we have at our disposal.

OSI Layers

Planning out the Network Map

First, we need to determine what devices will be on our network. Here’s a list of what I was looking at:

  • Miscellaneous personal devices (Home network)
  • The phones of various guests who want Wifi (Guest network)
  • The host KVM hypervisor
  • One or more servers for running various websites
  • One server running a HTTP/HTTPS reverse proxy (to redirect traffic to various websites based on hostname)
  • A Plex media server
  • A few throwaway servers for testing malware on
  • A few workstations I could remote into
  • A Wireguard server that allows for me to remotely access my network securely

With this list in mind, I started dividing devices into zones, following a least privilege methodology–if one device doesn’t need to access another, it shouldn’t be able to. I started by segmenting servers I would be running on my VM:

(image of segmented servers: loadbalancer + webservers plex workstations malware analysis )

None of these servers should need to access each other, so they’ll each be placed in a different subnet/VLAN. The router’s firewall rules will ensure that communication between these subnets are blocked.

Next, I needed to plan out a network that was capable of accessing the various servers listed above. SSH was my preferred method of management, and I wanted to be able to work on each of these servers without having to constantly switch networks. In order to facilitate this, I planned out a “command” network–one that had one-way access to these other networks. I placed the Wireguard server in this network, as well as a wireless Access Point that was separate from both my home and guest network.

I also decided to put my hypervisor host on this network. This may seem to go contrary to the principle of least privilege I was following, but since my Wireguard instance (which ran on top of the hypervisor) was on this command network, it wouldn’t make sense from a security perspective to put the KVM on another network.

The result of this command network was as follows:

(image of segmented servers, plus command network)

If we add in my home and guest networks, which remain completely separate from the other networks, we get the full picture:

(image of all the things)

There was one final configuration I had yet to plan: router access. In the end, I decided to configure my router so that only my home network could access it. This may seem counterintuitive (shouldn’t the command network have access to the router?), but I’ll soon get into why I chose this setup.

Security Considerations

Like I had mentioned before, I wanted my network layout to make it as hard as possible for a compromised server to affect my day-to-day online life. To visualize this goal better, we can rearrange the network map as follows:

(Show another image of the network map, but from the perspective of attacks)

If we look back to the rearranged network map above, there are essentially three layers to the network:

  1. The Home & guest networks

  2. The KVM hypervisor & associated access

  3. Various segmented servers running on the hypervisor

By default, any compromised hosts in the 3rd layer are incapable of accessing the layers above it. There are a few ways this barrier could be broken though–VM escape exploits or VLAN hopping mixed with another vulnerability come to mind. Because of this, the 2nd layer ‘command’ network is additionally firewalled off from my home and guest networks.

There is one big stipulation to this layering, however: it is ultimately the home router that draws the firewall boundaries. If it gets compromised, then the home and guest networks are about as toast as you can get. So, it makes sense to limit access to the router to the most protected layer–the home network.

Conclusion

As we can see, there are plenty of security benefits that can be had simply by taking the time to plan out an effective network map. It may take a little more effort than throwing everything together on one subnet, but segmenting a home network can definitely be worth it for the peace of mind.

Now that we have our network mapped out, we need to properly configure our router and server to implement it. The next post in this series will go over how to configure these network boundaries for our router using OpenWRT–I hope you find it helpful!

Next up: Configuring multiple VLANs/subnets for one server in OpenWRT (KVM Network Tutorial, Part 1)