Blog :: Netflow

NetFlow Generators: Enabling NetFlow Without NetFlow Support (Part #2)

Continued from NetFlow Generators: Enabling NetFlow Without NetFlow Support (Part #1)

Last week we covered NetFlow Generator basics including many of the more common deployment options. Now let’s take a look at some of the NetFlow generators available and what characteristics to look for in a best-of-breed NetFlow generator.

Features to Look for in a NetFlow Generator

While all NetFlow generators generate NetFlow, not all NetFlow is the same. NetFlow v9 and IPFIX are state of the art in terms of flow export format. Both NetFlow v9 and IPFIX make use of a self-defining record format that allows the exporter to send flow fields of any type in any order. Almost anything can be sent over NetFlow v9 as long as the NetFlow collector knows how to process the fields (more on this later). The list below provides an overview of the features to look for in your NetFlow Generator:

> At least 2 capture ports and 1 management port

The generator should come equipped with at least (2) capture ports and (1) management port. Traditional Ethernet taps require one capture port for send and an additional capture port for receive. In addition to the capture port a management interface is required for accessing the generator and exporting flows.

> Single or multi-cache option

Some NetFlow Generators such as our FlowPro Defender and nProbe allow for either a single NetFlow cache across all interfaces or a unique cache per interface*. Multi-cache deployments are used when the collector supports NetFlow deduplication. In a single-cache scenario double counting can occur when a packet is seen by more than one interface on a single generator as two different flows will be created in the same cache based on a single flow.

* This feature can be somewhat difficult to grasp at first so drop us an email if you’re interested in discussing cache options in greater detail.

> Application awareness

High-end NetFlow Generators will usually offer an application awareness option. Application awareness allows the generator to peer into the packet payload itself to identify the underlying layer-7 application without regard for port number. An “application-id” field is exported that tells the collector the true nature of the traffic being monitored. Given the migration of most network traffic to HTTP (port 80) this feature can be a life saver. Check out this blog on application aware flows for more detail.

> Latency, loss, and other network performance calculations

Another feature that’s crucial for network performance monitoring is latency exports – specifically RTT and SRT. RTT is “round trip time”. This is a measurement of the time it takes a packet to travel from the client to the server and back. RTT measures the network’s latency and is very similar to the results you get when you issue a “ping”. SRT is “server response time”. SRT is a measurement of the time it takes the server side of a flow to respond to queries such as HTTP GET or POST.

In addition to SRT and RTT, some NetFlow Generators such as nProbe will also include statistics on voice and video. Cisco’s MediaNet NetFlow features are one example.

> HTTP information

Given the wide-spread use of HTTP for everything from P2P to instant messaging, it’s often very useful to see the actual URL or host-header information for a given HTTP flow. nProbe and our FlowPro Defender both provide this information.

> Deep cache

One of the primary benefits a NetFlow Generator provides is the ability to maintain a very large NetFlow cache. In a normal NetFlow-enabled router or switch the NetFlow process must steal memory from the rest of the features enabled within the router. Memory is often at a premium and the size of the cache is usually quite small (128K flows or less). If the cache is too small and fills up under heavy network load, the exporter will drop flows and the collector will under report bandwidth levels. A robust NetFlow Generator should offer the ability to scale its cache upwards of 1 million flows or larger.

> Multi-destination export

The NetFlow Generator should allow export to multiple collectors on varying ports.

> IPFIX support

All NetFlow exporters, including NetFlow Generators, should be equipped with IPFIX support. Over time NetFlow v9 will be deprecated in favor of the IETF standard of IPFIX.

> Flow filtering

Some NetFlow Generators such as Cisco’s NGA allow the user to filter on certain types of traffic. This option can be useful in high volume environments where you only want to see a subset of the network traffic.

 

NetFlow Generators to Know

NetFlow GeneratornProbe

The original NetFlow Generator developed by Luca Deri (creator of nTop). nProbe is one of the most sophisticated NetFlow generators available. With support for latency, URL export, and application awareness, nProbe should be one of the first NetFlow generators you should try out. It’s software-based so it can be deployed directly on a server for host-level NetFlow export.

nBox

A high-performance, appliance-based, turn-key implementation of nProbe. Available from Plixer.

Plixer FlowPro Defender

Plixer’s FlowPro Defender is a full featured, enterprise-ready NetFlow or IPFIX generator. The insight it provides into DNS traffic for identifying malware place this among the elite of NetFlow generation appliances. The FlowPro sports deep packet inspection, TCP latency statistics, and URL exports. Its rich set of NetFlow elements make it an exceptional probe.

Endace NetFlow Generator

A high-performance, hardware-based NetFlow generator. Endace is known for its high-speed products and this device is no different. If extreme speeds are your concern this generator would be worth a look. Unfortunately the Endace NetFlow Generator lacks some of the more sophisticated features such as applications awareness.

Cisco NetFlow Generation Appliance (NGA)

The NGA is a recent addition to Cisco’s vast product portfolio. Based on Cisco’s own Flexible NetFlow subsystem, the NGA is a full-featured generator that supports such advanced capabilities as export filtering and applications awareness (NBAR2). <<< This needs to be confirmed.

 

Now Go Get a NetFlow Collector!

Regardless of which NetFlow generator you select you’ll need a NetFlow collector to see the value of network flows. Given the wide variety of advanced NetFlow fields exported by NetFlow generators such as nProbe, you’ll need a collector that supports any NetFlow v9 or IPFIX field sent by the exporter. Many flow collectors only support the basic flow fields such as source IP, destination port, or byte count. Flow collectors that limit the fields processed should be avoided as they do not take full advantage of the NetFlow Generator’s capabilities. One such collector is Plixer’s Scrutinizer NetFlow Analyzer. Scrutinizer can process all the elements from any of the netflow generators listed above. Scrutinizer’s Flow Analytics subsystem detects network threats while the Advanced Reporting module satisfies the needs of the most demanding network operator.