We had a customer call in that wasn’t seeing outbound traffic on their H3C NetStream capable routers in our NetFlow Analyzer. Normally outbound utilization is displayed using ingress flows from the other interfaces destined for the selected outbound interface. In this case, the customer said their Netstream export was providing both ingress and egress flows. NetStream is really NetFlow as is JFlow.
We requested a packet capture and found that they were definitely exporting egress flows in a separate template from the ingress flows. This is really very different from any NetFlow or NetStream export than we have ever seen. Why were we getting two different templates. Investigation was necessary!
Notice above that Wireshark displays “Post Octets: 80” which is what we call postOctetDeltaCount. Our NetStream reporting front end is always looking for ‘Octets’ or octetDeltaCount when displaying inbound or outbound data. We don’t look at postOctetDeltaCount because it is my understanding that this field is only used when a “middlebox function” is used. IMO, the NetStream engineers may have made a mistake. See Jon’s blog on octetDeltaCount Vs. postOctetDeltaCount.
The logic we use when NetFlow or IPFIX reporting on interfaces is as follows:
- if inbound is requested, use ingress flows for the selected interface
- else, use egress flows
- if outbound is requested, use egress flows for the selected interface
- else, use ingress flows
When the customer tried to look at inbound utilization on an interface of the HC3, our NetFlow analyzer displayed the data expected. Conversely, when the customer tried to look at outbound utilization on an interface of the HC3 our NetFlow interface displayed “no data” because all of the egress collected flows were missing octetDeltaCount. In other words, when our front end looked for egress flows, it found the template but, without any octetDeltaCount resulting in a report that didn’t work.
What was the solution?
The solution for the customer was to turn off egress collected flows. When the user reported on outbound utilization on an interface, the front end wouldn’t find egress flows and would resort to using ingress flows which solved the problem. The customer was happy.
One template is plenty?
Why is HC3 exporting two templates? Because the egress flows export postOctetDeltaCount and the ingress flows export octetDeltaCount and because of this difference, two templates were necessary. Normally, both ingress and egress export octetDeltaCount and the direction bit is used to indicate either ingress=0 or egress=1. We haven’t seen postOctetDeltaCount implemented by anyone. Not that it couldn’t be done correctly, we just haven’t seen it.
Lets summarize what we found:
- In the ingress flows template we found octetDeltaCount as expected. Inbound reports worked!
- In the egress flow template it was using the OUT_BYTES (i.e. Post Octets) info element. Outbound reports failed.
- All other vendors we have ever worked with are exporting octetDeltaCount for both ingress and egress flows
After our engineers determined the above, we realized that there were a couple of options…
- We suggest to the customer that they use ingress only and disable egress flows. Egress NetFlow is only used in a few cases.
- We could build special reports for H3C that would use postOctetDeltaCount for outbound utilization reports.
The customer went with option 1 and it is a good thing for us. The 2nd option would have been considerably more work us and H3C isn’t a vendor we see a lot of demand for. Since H3C shares roots with Huawei, I can only imagine that we could see the same issue with their hardware.
Huawei is making a big push in the Integrated Communications Technology market and hopefully IPFIX will be a part of their strategy. John Roese is their new VP and GM for N. American R&D. Prior to Huawei John was the CTO for Nortel and Enterasys both of which are companies that implemented IPFIX and NetFlow respectively.
If you are a vendor looking to implement NetFlow or IPFIX, feel free to run any engineering questions by our NetFlow and IPFIX Experts. We can save you a lot of headache. In the mean time, the NetStream configuration details can be found on their web site.
Founder and CEO
For a free 30 day trial of Scrutinizer, Download Now!