Our NetFlow and sFlow Analyzer receives data collected over a 1 minute time interval per flow, and can store up to 100 000 conversations (flows) per device. One limitation in NetFlow monitoring today is the amount of disk space needed to store the collected network traffic information. Especially, if one’s intent is to hold on to that information for a certain period of time. In this blog I will try to help you understand how Scrutinizer archives data. In addition I will talk about the NetFlow Calculator, which can be a helpful tool for estimating the disk space needed on your NetFlow analyzer server.
Scrutinizer Data Archiving System
The following blogs explain how it works:
Assuming we read the blogs I will now move on to another subtopic.
If you are using our NetFlow Analyzer, you may have seen the page illustrated in the following screen capture.
Seeing this page when trying to generate a report could mean that there is no data that fits the specified time frame. Both of the blogs I recommended reading give a clear description of DB tables that are created as a result of roll ups. There are 1 minute, 5 minute, 30 minute, 1 week and 1 day tables. Because of this organization, when a time frame is selected, Scrutinizer will choose tables that best fit the select time frame. For example, if you chose to view data from the last hour, it makes sense that Scrutinizer first tries to retrieve data from the 5 minutes and 1 minutes tables because they are smaller intervals.
What if the flow collector server has just started collecting, and the 1 min tables have not been rolled up into 5 min tables yet? In this case, If Scrutinizer first tried to retrieve data from the 5 min tables and did not find any data, it will show you a page such as the above screen capture. However, on this page “1m” will be a link to an alternative report that will be generated from 1 min tables.
Moreover, the appearing of this page could mean that the data in the Scrutinizer database is missing certain information that is necessary to constructing the requested report ; usually because a device was not configured to send such information. In our traffic analyzer for instance, in order to generate reports such as “Application NBAR” or “Conversation NBAR”, you will need Flexible NetFlow configured for NBAR export on your devices so that NBAR information is sent out as part of the flows.
To be continued in part 2