VIRTUALB.EU VMWARE 6.2 UPDATE SERIES 2: From vSphere Virtual Switch to NSX 6.2 Part 1

A little bit of VMware virtual networking history

Virtual networking has come a long way. Started off with VMware workstation 1.0 to GSX and ESX a brief interlude on ESXi and finally landing on a control plane on the vCenter. With the coming of NSX VMware has taken this even further and distributed the components out of the vCenter and out of the virtual and physical datacenter. But before we go to the enhancements in NSX 6.2 let’s rewind back to the good old days and start with the birth of the virtual switch aka the vSwitch.

The first iteration: the vSphere Standard vSwitch (vSS)

The first virtual switch was just a port of a normal network switch as we know with the control plane placed on the ESX(i) server. Most of the intelligence as we know it of normal switches was taken out of it (port QoS, etc) and a simplified switch was created. It’s main function was to create a virtual network with a dual function:

  1. Interconnect Virtual Machines (VM’s) with eachother so they could exchange packets as if they were on a real network using virtualized network cards or vNICS
  2. Connect VM’s to the actual network through the means of uplinks with physical Network Interface Cards (NICS) or pNICS. VMware developed redundant uplink policies so multiple pNICS could be bundled to avoid outages when a port or pNIC failed
  3. Another function the vSwitch offered was access to the kernel with a special kind of vNIC: vmkNIC . These NIC’s were used for a number of functions: NFS or iSCSI traffic, vMotion traffic, management traffic to the service console at the time (depreciated in ESXi)

In short it was making the a virtual representation of the physical network in it’s most basic form. It still exists today on standalone ESXi servers.

This is how it looks like:

sw

 

The second iteration: the vSphere Distributed Virtual Switch (vDS)

Configuration of the vSphere Standard vSwitch was on a per host basis. This is great for a 3 node cluster and with some scripting skills relatively scalable. But imagine the amount of work going into reconfiguring 1000+ hosts. It just wasn’t scalable enough so entry the vSphere Distributed Virtual Switch. (There was a small interlude with a port of a virtualized and distributed Cisco Nexus 2000 to the vSphere platform. I will not mention that in this article as it died a silent death after the introduction of the vDS)

What was the big difference with the vSS? The switching management plane moved from the ESXi server to the vCenter, the switching data plane (handling actual packet switching, tagging, etc) remains on the ESXi. The vCenter now distributes the configuration of the ESXi local data switching from one configuration point. So what were once local portgroups will now be distributed port groups and what were once local uplinks will now be distributed portgroups. This will ensure all hosts are configured exactly the same and eases the network (re)configuration off multiple hosts at the same time.

dvs

Advantages:

  1. Quick network configuration and interconnect of new and existing ESXi servers in a vDatacenter
  2. New, more advanced, features introduced per version upgrade (FlowControl, Ingress/Egress filtering,…)
  3. Simple backup / restore features

Disadvantages:

  1. Makes vCenter a massive Single Point of Failure (SPOF)
  2. Circular dependency when your vCenter is virtualized (see chicken or egg syndrome link below)
  3. Comes only with Enterprise Plus licensing so at a cost. (Same as host profiles)

VMware has done a lot of work since the first release of the vDS and the latest version basically addresses all above issues by protecting deleting the management network when migrating to vDS from vSS, advanced backup and restore functionality and the coolest advanced NIC teaming policies including enhanced support for Link Aggregation! These are the currently supported LAG NIC teaming policies available to attach your pNIC’s to your physical switch:

  1. Destination IP address
  2. Destination IP address and TCP/UDP port
  3. Destination IP address and VLAN
  4. Destination IP address, TCP/UDP port and VLAN
  5. Destination MAC address
  6. Destination TCP/UDP port
  7. Source IP address
  8. Source IP address and TCP/UDP port
  9. Source IP address and VLAN
  10. Source IP address, TCP/UDP port and VLAN
  11. Source MAC address
  12. Source TCP/UDP port
  13. Source and destination IP address
  14. Source and destination IP address and TCP/UDP port
  15. Source and destination IP address and VLAN
  16. Source and destination IP address, TCP/UDP port and VLAN
  17. Source and destination MAC address
  18. Source and destination TCP/UDP port
  19. Source port ID
  20. VLAN

Just to give you an idea how the above NIC teaming policies are calculated I will give an example how IP hash is calculated. IP hash is the default policy when using Etherchannel (or LACP):

With IP Hash the host will take the source and destination IP address in a HEX format, perform an Xor operation on them, and then run the result through another calculation based on the number of uplinks in the team. The final result of the calculation is a number between 0 and the number of uplinks, minus one. So if you had four uplinks in an IP hash team, the result of each calculation would be a number between 0 and 3 – four possible outcomes that are each associated with one NIC in the team. This end result of the calculation is commonly referred to as the interface index number.

This is an example of the use of LAGs with vDS:

LACP2

As you can see VMware has worked hard to improve the initial vSS to come to great product like the vDS with LAG support! The vDS combined with host profiles has greatly simplified deploying huge vSphere farms in short time frames and with exact the same configuration settings, ruling out human error. When managing 1000’s of ESXi hosts the combination of these two tools is extremely powerfull and saves time, both in initial engineering and later in support.

More information in these previous articles on Virtualb.eu

vSphere-5-1-dvs-networking-improvements

vCenter-on-dvs-chicken-or-the-egg-syndrome

VMware KB’s

NIC teaming in ESXi and ESX (1004088)

Configuring LACP on an Uplink Port Group using the VMware vSphere Web Client (2034277)

Enhanced LACP Support on a vSphere 5.5 Distributed Switch (2051826)

Troubleshooting-network-teaming-problems-with-ip-hash

The third iteration: The coming of NSX and complete network independence, set the VM free!

NSX is the logical follow up from the vDS but takes things a bit further and completely virtualizes the network stack from L2 upwards. So it basically takes the intelligence that used to live up your network stack (on the switching, routing , QoS, load balancing, firewall, etc) and pulls it into the vCenter.

NSX overview 

(Image courtesy NSX configuration guide – see link below)

VMware compares this with with server virtualization as you can see in the above diagram. Basically NSX puts a virtualization layer between the networking hardware just the same way ESXi does for X86 hardware when virtualizing servers. The only requirement NSX has is an IP transport layer between point A and point B. So we still have the vDS on the vCenter and we are virtualizing the end points we normally connect to from our ESXi servers and VM’s. This allows to deploy VM’s within certain network zones without intervention on the physical hardware and fully automate the network configuration. One step closer to the fully automated datacenter.

All of this kicked off a few years ago with requirement to create a network overlay between phycical datacenters to allow VM transportation (vMotion) without the requirement to change the IP address of the VM. Virtual Extensible Local Area Network aka VXLAN was born. VXLAN was an effort to create an internet standard (RFC) to create a L4 (OSI model Layer 4) network overlay for L2  (OS model Layer 2) network packets. (Quote Wikipedia: “A VLAN-like encapsulation technique to encapsulate MAC-based OSI layer 2 Ethernet frames within layer 4 UDP packets, using 4789 as the default IANA-assigned destination UDP port number.) What does this mean? You can “stretch” your L2 (data link) network by means of encapsulating and routing it in L4 (TCP in this case). This makes all devices on the network think they are connected to the same physical network so a stretched subnet/VLAN can be created and the same IP address space can exist in 2 different datacenters enabling true VM portability.

At the time VXLAN was conceived there were limited use cases for it:

  • Biggest use case probably was the discontinued VMware vCloud Director which required VM mobility between different tenant zones
  • Another use case was stretched vSphere clusters where synchronous storage replication was needed to synchronize VMDK files between datacenters. This enabled DA (Disaster Avoidence) but turned out to be a complex and expensive solution for a limited use case.

Allthough I like the technology neither of both complex solutions ever made me go: “Wow, this is the answer to the problem I’ve been dealing with for the last few years.” When creating a technical design I always live by the rule “Keep it simple, stupid.” There is a simple reason for this: As an architect/designer it is quite easy to go about and deliver complex solutions. Lots of times this not only comes to the cost of the solution but also comes to a cost for L2 and L3 support teams: sometomes new hires have to be made to support the complex technologies or current resources need to be retrained to be able to support it. I am not saying new technologies should be avoided, what I am saying is that during the initial discovery phase gaps in support teams should be identified and added to the total cost of the project. I have seen many IT projects go haywire because support teams were not being kept in the loop on what was in the pipeline which resulted in support not accepting the new solution, downtime or extra costs down the road which were not accounted for in the initial project costing.

Fast forward a few years and VXLAN has been integrated in a full product suite called VMware NSX. VMware NSX is the VMware’s Software Defined Networking (SDN) product. There are a few competitors on the markets but VMware is ofcourse ideally positioined to pitch NSX as they already are in 99% of the enterprise datacenters. Competitors in the same market segment are Cisco, Juniper, Open Daylight, Big Switch Networks and numerous open source products.

I will go more into depth on VMware NSX Use cases and scenarios in part 2 of this particular blog post on Virtual Networking. That post will also highlight the newest innovations in NSX 6.2 and

More information

NSX 6.2 Configuration and Installation Guide

Virtually Speaking Podcast – Episode 10: Intro to NSX and use cases

RFC 7348: Virtual eXtensible Local Area Network (VXLAN)

Speak Your Mind

*

%d bloggers like this: