Often these days, we’re asked for a general rule of thumb or guidelines that can help with calculating NetFlow Volume. How much disk space should an average NetFlow Deployment consume. One of the biggest concerns is that exporting NetFlow (or one of its cousins) for Network Performance Analysis will have a drastic impact on the available bandwidth, CPU overhead on devices, or on the hard drives storing it. This is simply not true.
It is important to note that one Network Flow Data export can contain records for up to 30+ conversations or flows. This is important because the average volume of NetFlow is directly proportional to the number of unique TCP/UDP sockets created by clients and servers on your network.
This aggregate nature of NetFlow, and the fact that NetFlow Packets are comprised only of IP Header information (i.e. not the actual packet payload itself), is why the export only consumes between 1-2% of interface throughput. Since 2004, Cisco NetFlow experts have maintained (NetFlow for Accounting, Analysis and Attack, slide 116) a rule of thumb that NetFlow will only create 1-1.5% of throughput on the interface it is exported on.
With that out of the way, we now want to know — what is the typical flow volume per PC? The answer is: it depends, but in our office the trend appears to be about 100 flows/minute per desktop, with a peak of 350. Let’s take a look at the guy next to me:
Let’s say for example that your company has 20,000 nodes on it and every node causes 200 flows per minute. This equates to about 4 million flows in one minute, or about 67,000 flows per second. You might ask, why so many flows?
Applications spawn lots of unique flows, especially web browers and most applications these days phone home to check for updates. Here are some typical applications that are very chatty:
- Java, Adobe, Anti-virus, web browsers
- Skype is very chatty and causes traffic to the DNS as well
- Web pages spawning flows for images, ads, etc.
- Email constantly checking the Inbox
- NetBios
In the following example, I selected a time when the flow volume was lightest, check this out:
After considering the above image, understand that every network is different and so is the flow volume. My goal here is to provide you with some guidelines when planning out your next enterprise NetFlow collection and reporting solution. A single copy of Scrutinizer is capable of collecting over 100,000 flows per second and even more when collectors are configured for distributed NetFlow collection.
Only capturing all the flows will allow flow data to be used as a part of IT Threat Management for network behavior analysis or as a Network Security Forensics tool. Don’t settle on a NetFlow solution that can’t talk about saving raw flows, or explicitly define how much flow data is being saved. Our NetFlow Analyzer will save raw data as long as disk space is available and archiving is user-configurable for the Top N conversations at each interval.