Most WAN Optimization appliances support NetFlow. Although the majority of our customers have Cisco WAAS, a significant amount of our customers have the Riverbed Steelhead appliance for WAN compression. NOTE: This is not going to be a Cisco WAAS vs Riverbed Steelhead post. I’m sure both products have their merits and we’re not about to pick on WAAS issues or point out all the Steelhead problems. Rather, I want to highlight some things to look out for in regard to the Riverbed NetFlow exports.
In August of 2006, Riverbed released NetFlow v5 support on the Steelhead appliance in RiOS 3.
In the Riverbed document Understanding NetFlow and Steelhead on page 2 it points out that “NetFlow statistics are collected on the ingress interfaces of the Steelhead appliance. Therefore, to see a complete flow or conversation between the server and client, it’s necessary to configure NetFlow on both the client-side Steelhead as well as the server-side Steelhead.” This is because NetFlow v5 only counted traffic going ‘in’ the interface. To display traffic going ‘out’ an interface, the destination interface of the flow is utilized by the NetFlow Analyzer to determine outgoing traffic on the interfaces of the hardware. For this to work correctly, ingress flows must be collected on all interfaces of the hardware. What goes in must go out.
The above strategy works with NetFlow v5 as outbound traffic is calculated by querying all of the inbound flows. NetFlow v5 works well, but not so great with WAN compression. A flow might come into the device with 200 bytes, be compressed and go out with 25 bytes. I think you can understand now why this caused problems in the NetFlow Report. What goes in goes out differently.
In the above Riverbed document referenced, it states “Many NetFlow analyzers incorrectly assume that the amount of traffic entering the router must be equivalent to the amount of traffic exiting the router.” I feel that this is not entirely correct. Many NetFlow monitor companies feel that using ingress flows to determine outbound or egress utilization is how NetFlow v5 was designed and intended. I don’t remember Cisco, the inventor of NetFlow, ever making a statement like this or supporting something like a ‘FakeIndex.’ I’ll digress on that below.
Regarding NetFlow, Riverbed issues also exists where “in a virtual in-path deployment, both the “InputInt” and “OutputInt” fields will be set to the WAN interface. Technically this is correct, as all traffic enter and exit the WAN interface in a virtual in-path deployment. Nonetheless, customers with virtual in-path deployments will not be able to differentiate between LAN-to-WAN and WAN-to-LAN traffic by using the “InputInt” or “OutputInt” fields. Customers can, however, still use the source IP and destination IP addresses to differentiate between LAN-to-WAN and WAN-to-LAN traffic. If this isn’t a feasible option, consider using the “FakeIndex” feature.” While I question whether “Technically this is correct” is actually correct, I do agree that this Riverbed problem needed to be addressed, and their “FakeIndex” solution is a decent work around although it adds more complexity to an issue that could have been avoided with proper IPFIX or NetFlow consulting.
Before I get back to how Riverbed eventually decided to support NetFlow v9 Egress NetFlow, I want to point out another one of the Riverbed problems that is also cited in the above document: “A side-effect of having the WAN router exporting NetFlow data is that the WAN router will also export the inner-channel traffic between the Steelhead appliances. This is a minor issue as the NetFlow analyzer should be able to ignore traffic on port 7800.” Perhaps a way to avoid this issue would have been for the Riverbed hardware to avoid using NetFlow to exchange proprietary information between the hardware. Maybe they could have used another technology or simply exported the data using completely different interface indexes. Asking the NetFlow analyzer to cover their NetFlow problem causes more work for all the NetFlow Collector vendors.
Anyway, back to ingress vs egress NetFlow. Riverbed complaints came in regarding the overstated outbound utilization issue. We pushed the Riverbed NetFlow developers to consider NetFlow v9 with egress support. With egress NetFlow, traffic is collected as it goes out an interface (i.e. after it has been compressed).
In time, Riverbed implemented NetFlow v9 with egress support. Notice in the Steelhead Central Management Console User’s Guide Version 6.0 Dated June 2010 on page 288 it states:
• NetFlow v5. Enables ingress flow records.
• NetFlow v9. Enables both ingress and egress flow records.
The next opportunity for the Riverbed NetFlow development team is to migrate to IPFIX which is the proposed standard for NetFlow. They may also want to consider some of the awesome new statistics that products like the nBox, Sonicwall and Cisco WAAS are implementing.
UPDATE: 12/19/2013 Riverbed Steelhead NetFlow Reports are now FREETags: egress NetFlow, riverbed problems, steelhead netflow problems