VMware IPFIX support and our IPFIX collector, Scrutinizer, give you visibility into the ‘cloud’. In this blog, I will show an example of a communication between two hosts (called tenants) on separate Virtual Machines. Read below, there’s nothing but blue skies.

As virtual deployments grow and become more prevalent in our infrastructure, this overlay network needs monitoring in the same matter as your typical physical network layout. VMware has embraced this with their IPFIX implementation in monitoring the Virtual Distributed Switch (VDS).

Through a tunneling mechanism like VXLAN, an overlay network is created, allowing Virtual Machines (VM), located on separate physical servers to communicate as if they have a direct Layer2 connection to one another. Packets pass between VMs through these tunnels without the VMs having any awareness of the network infrastructure in between, or the tunneling mechanism in place. Let’s see how this works.

Here is a communication between Host-1 (Tenant-1), located on ESX-1 is communicating with Host-2 (Tenant-2) on ESX-2.

Tenant-1 to Tenant-2 Conversation
Host to host conversation via VXLAN

With traditional NetFlow support, our collector would receive flows from the underlay network that ESX-1 is communicating with ESX-2, completely unaware of the Tenant communication that is happening with VXLAN encapsulation. This happens because the ESX is also a VXLAN Tunneling End Point (VTEP). Thus, we rely on VMware IPFIX support, to receive the encapsulation elements.

Here is the breakdown of the packet header:

The outer Layer2 header is dependent on Layer3 hops of the underlay network.

The outer Layer3 header has the IP address for the destination VTEP-2 server and the source IP of VTEP-1 server.

Layer 4 UDP header has a destination port 4789. Port 4789 is reserved by IANA for VXLAN. The source UDP is calculated at random. This creates variability in source UDP ports for packets between pairs of VMs

VXLAN header contains the VXLAN Network Identifier (VNI). This is how VTEPs distinguish between various hosts (tenants) on a VM. The VNI has 24 bits reserve, which allows for over 16 million VXLAN identifiers, take that VLAN.

Here is what it looks like in our collector:

pairs with tenants
VMware > Pairs with Tenants Report

Above you can see the Source Tenant-1 communicating with Destination Tenant-2 through the Source VTEP-1 and the Destination VTEP-2 via VNI 1.

As you can imagine, with more tenants on a particular VM, the visibility into their communications proves itself vital to an organization.

Here is another example of this communication, this time singling out the tenant conversation with source/destination ports and the protocol used:

tenant conversations
VMware > Tenants Conversations

There you have it, just a taste of the rich IPFIX export available for your virtual environment. Configure VMWare IPFIX support.

Contact our support team and we’ll help you configure custom VMWare reports.

Jennifer Maher

Jennifer Maher

Jennifer is a Software Engineer at Plixer. She strives to create a product that provides customers the optimal experience. Jennifer loves working with data visualizations and finding new ways to make that information easier to understand. When she’s not working she enjoys spending time with her children, video games, dancing and theater.

Related