Sampled NetFlow accuracy is often a concern when network administrators find that sampling is their only NetFlow network traffic monitoring option.
High volume flow exports can consume too much bandwidth when sent across the network, and/or drastically impact performance of the switch or router. As a result many high end network device vendors are either forcing, or optionally recommending, that you sample packets for flow exports.
A sampling rate of 1 out of 100 packets may reduce the export of NetFlow data by as much as 50 percent. But you have to keep in mind that when sampling, a NetFlow or IPFIX collector is only receiving a small percentage of the traffic and will not properly represent total throughput or traffic details.
Vendors supporting sFlow exports get around this by using interface traffic counters in conjunction with the IP samples. Poll Counters export accurate general interface use statistics much like using SNMP. The traffic details seen in reports still fall short because we are only reporting on what got sampled, 1 in every X packet headers, not all of them.
So you can see that using flow sampling for network traffic monitoring can leave some holes in what is being reported.
Another very important side note as it relates to sampled NetFlow accuracy is network security.
What is the impact when using sampled flows for Network security and detecting advanced persistent threats?
Today, network traffic monitoring using flow technologies to monitor communication behaviors, maintaining baselines, and detecting advanced persistent threats is becoming more relevant.
By collecting flows representing all of the conversations traversing the network, you gain visibility into suspect conversations coming in and out of your network as well as moving laterally inside. When the signatures in the IDS/IPS fail to catch malware, NetFlow and IPFIX can sometimes recognize enough odd behaviors to identify an infection.
When security professionals need to go back in time and view a communication pattern, they find that sampled flows seldom contain the flows that they want to investigate.
So what options do you have when it comes to sampled NetFlow Accuracy?
- Don’t use Sampling.
- Offload the NetFlow function to a Flow Probe.
- If available, use the new flexible aggregation capabilities of true flow technologies that allow the user to specify the key fields they want to receive. In lieu of sampling packets, hardware can create flows based on specified key fields. We can reduce the volume of flows exported if we simplify the matching key fields.
- Apply a multiplier to the report data.
Option Four is why we are talking about sampled NetFlow Accuracy today.
In most sampled flow configurations the sample rate is globally applied across all of the monitored interfaces. We can use the sample rate as an option to apply a multiplier across the details seen in the reports.
There are couple of things to keep in mind when applying a sample multiplier.
How consistent is the traffic traversing the interface or device?
If the traffic is fairly uniform, applying a multiplier can probably yield pretty accurate reporting results.
What about a mixed, non-uniform, traffic flow?
Can we assume that a packet sampled as part of an ICMP or SNMP request carries the same traffic weight as a sampled packet from a file download?
I would say, NO.
Simply applying the multiplier to the sampled ICMP flow would most likely overstate the amount of traffic seen in that particular host to host conversation.
The other part of the equation is – what did get sampled?
In many cases conversations may not show up at all in report details because they did not get sampled. This is very much the case on high volume devices where minimum sample rates are in thousands.
So you are really only applying the sample multiplier to the sampled traffic, not all of the traffic.
Sampled flows are great for understanding trends. If you really need sampled NetFlow accuracy and traffic visibility, set the sample rate to 1. This way you will be collecting all of the flows.
Is sampled NetFlow your only configuration option? If you would like to explore other network traffic monitoring options, contact us.