VirtNet

network stack virtualization for FreeBSD

FreeBSD network stack virtualization

FreeBSD jail [1] is a widely accepted framework for application environment isolation. Processes running inside jails have a restricted view of resources provided by the operating system, most notably, they are unable to directly interact with other processes outside the scope of their own jailed environment. Combined with restricting jail's network visibility to a single system IP address while confining the file system access to a private directory tree, the jail model provides isolation capabilities sufficient to allow system administrators to host tens or hundreds of such environments on a single physical machine while delegating per-jail superuser authorities to other parties. In other words, a jail can be thought of as an isolated lightweight virtual host with its own (potentially untrusted) system administrator, users and applications; while sharing the base OS kernel and physical system resources with other such environments. This concept, first introduced in FreeBSD 4.0, proved so successful that not only it become a platform of choice for many application hosting providers, but it lead to introduction of similar features in operating systems other than FreeBSD, such as zones in Sun's Solaris or the Linux Vserver project. While providing less rigid levels of isolation compared to traditional hardware virtualization architectures such as IBM's z-Series platform or more recent paravirtualization models such as Xen [2], the main attraction and strength of FreeBSD's jail concept lies in its scalability and efficient usage of hardware resources.

One shortcoming of the original jail model is that it exposes a very restricted set of networking facilities to jailed applications. Network stack settings such as IP addresses, routes or firewall rules can be administered only from the global OS context, not from within the jails themselves. Delegating the authority to manage the network stack settings to jailed super-users would require independent copies of network stack state variables to be kept on a per-jail basis. Precisely such a model was experimentally implemented as an extension to FreeBSD 4.7 kernel [3]: the existing networking kernel code was modified to operate on multiple clonable structures where most of the networking-related state is kept.

Such a new virtualized network stack model turned out to provide a great flexibility to perjail local administrators. Each jail-style environment could control multiple private network interfaces with multiple IP addresses, could maintain and control its own firewall ruleset, routing tables, address translators, traffic shapers etc. In short, looking from the networking perspective, the network stack cloning model blurred the line between the traditional hardware (or para-) virtualization architectures and jail-style lightweight virtual machines. Most importantly, the experimental implementation proved that the overhead of the network stack virtualization was neglectable, so that the performance advantage of jails over traditional server virtualization models was preserved. Besides for virtual hosting applications, the clonable network stack model enabled the OS kernel to be used as a highly scalable and efficient network topology emulator, by constructing arbitrarily complex kernel-level virtual topologies composed of network stack instances (nodes) and netgraph-based explicit links [4].

References:

1. Poul-Henning Kamp, Robert N. M. Watson, Jails: Confining the omnipotent root, in Proceedings 2nd SANE Conference, May 2000.
2. Barham, P. et. al., XEN and the art of virtualization, in Proceedings of the ACM Symposium on Operating Systems Principles, 2003.
3. Zec, M., Implementing a Clonable Network Stack in the FreeBSD Kernel, in Proceedings of the 2003. USENIX Annual Technical Conference, San Antonio, Texas, June 2003.
4. Zec, M., Mikuc, M., Operating System Support for Integrated Network Emulation in IMUNES, 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure/ ASPLOS-XI, Boston, October 2004.