NetFlow version 9: egress vs. ingress

Posted in NetFlow, NetFlow Analyzer, Network Traffic Analysis, Scrutinizer on June 4th, 2009 by mike@plixer.com
NetFlow version 9: egress vs. ingress

I’m doing some more work lately with Wireshark and Scrutinizer v7. I thought that the topic of egress vs. ingress might be interesting to some readers.  NOTE: Egress is only available in Cisco NetFlow v9 and not NetFlow v5.

IPFIX or NetFlow v9?
In theory, ingress and egress should work the same in IPFIX, which is based on NetFlow v9, but they are certainly different. Although they are very similar, don’t let any company tell you they are exactly the same. Many collectors that work with NetFlow v9 will puke when they receive IPFIX. Scrutinizer handles both with ease. Nortel supports IPFIX, as does/did Avici, which is now Soapstone Networks, Inc. Other vendors, such as Adtran and Enterasys, support NetFlow v9.

One annoying area where IPFIX and NetFlow v9 differ is in the labeling of fields: NetFlow v9 has ‘IN_BYTES’ and IPFIX labels the same field ‘octetDeltaCount’.  IPFIX probably renamed it because when talking about egress flows, IN_BYTES is sort of misleading.

Ingress vs. egress differences
NetFlow v9 Ingress is collected on traffic going into (i.e. inBound) an interface.  This is how NetFlow v5 collects data. To figure out outBound traffic volume, ingress must be collected on all interfaces and the reporting software then displays outbound traffic. What goes in must go out, right?  Ya, usually.

NetFlow v9 Egress is collected on traffic going out (i.e. outBound) of an interface.  Generally, it is used in combination with Ingress, but it doesn’t have to be. I’ll dive into this a bit more.

Why collect with egress?
Why collect with egress, if ingress worked so well with NetFlow v5? Because hardware such as WAN optimizers compress data.  Traffic compression with Cisco NetFlow means that what comes in 100 bytes might go out as 50 bytes. If only using ingress flows, the NetFlow reporting software will show 100 bytes outbound, even if it was compressed to 50 bytes. GASP!!! This is because it was calculated using ingress flows.

Tell me the truth!
If the router is exporting both ingress and egress and the NetFlow monitor can report on both without overstating utilization, you can see how much of each flow is being compressed. It’s pretty slick, but it requires that the NetFlow collector understand what is known as the flow “Direction”. If the field in the NetFlow v9 packet is a 0, then it is an ingress collected flow.  If the field is a 1, then it is an egress collected flow.

Ingress Flow with IPv6 (the same with IPv4)

nfv9ingress

Egress Flow with IPv6 (the same with IPv4)

nfv9egress

The network traffic reports produced by the NetFlow analyzer need to be intelligent when dealing with ingress and egress flows. I feel that dynamically figuring out flow direction in mixed NetFlow v9 ingress egress environments is crucial, especially if the customer has hundreds of routers. If you are just setting up ingress, I would keep this blog in mind: “ip route-cache flow or ip flow ingress… Which do I use?”

Something else to think about
NetFlow traffic analysis is going to be taken to another level as Flexible NetFlow matures. Perhaps we’ll see it take advantage of what NetFlow v9 calls ‘OUT_BYTES’. (IPFIX, needing to be different, calls this same field ‘postOctetDeltaCount’.)

Now you might ask: how is it related to ingress or egress?  Stay tuned…

Michael Patterson
Founder and CEO

For a free 30 day trial of Scrutinizer, Download Now!

Sign up for Advanced NetFlow Training™ coming to a city near you!

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.
Tags: , , , ,

14 Responses to “NetFlow version 9: egress vs. ingress”

  1. ip route-cache flow or ip flow ingress… Which do I use? - NetFlow & sFlow Network Monitoring - Systrax Blog Says:

    [...] **NOTE** With NetFlow v5, we only had the option to monitor inbound statistics using the ip flow ingress command. However, with the release of NetFlow v9, we now have the option to monitor traffic leaving each interface via ip flow egress. Check out this blog which tackles the question: Which one is better to use? Ingress or Egress? [...]

  2. venkatesh Says:

    Hello,
    Some of your posts are awesome, and I really enjoy reading them. I have a very basic question regarding ingress and egress neflow data. How does CACTI or MRTG plot the interface in and out using the netflow v5 only. May be you would have explained this earlier but it would be great if you could explain in more detail if possible with an example.

    Thanks,
    Venkatesh

  3. Mike Patterson Says:

    Hello Venkatesh,

    Cacti and MRTG use SNMP to query the ifInOctets and ifOutOctets of a device to display in and out traffic. I don’t think they use NetFlow at all. The database of these tools is not practical for NetFlow storage. Storage and aggregation are paramount issues to consider for NetFlow analysis and reporting. Here is how we do it:
    http://www.plixer.com/blog/scrutinizer/unprecedented-netflow-and-sflow-archiving/

    I hope this helps.

    Mike

  4. NetFlow v9 vs. NetFlow v5: What are the differences? - NetFlow & sFlow Network Monitoring - Systrax Blog Says:

    [...] flow, which is something that NetFlow v5 can’t do. You need to read this blog on “NetFlow version 9: Egress Vs. Ingress” to understand the value of Egress flows. What I want to stress in this blog is that [...]

  5. Scrutinizer NetFlow Glossary - NetFlow & sFlow Network Monitoring - Systrax Says:

    [...] Flows Flexible NetFlow Ingress vs. Egress Interface 0 ip-flow timeout active 1 IPFIX ip route-cache flow vs. ip flow ingress NBAR NetFlow [...]

  6. Paul Says:

    What about performance? Cisco says egress flows puts a burden on the CPU, they make it sound dire. What is Plixer’s position on this?

    Cisco Link about egress flows:
    http://www.cisco.com/en/US/customer/docs/ios/netflow/configuration/guide/cfg_nflow_aggr_cache.html

    QUOTE from Link:

    “Performance Impact

    …Configuring Egress NetFlow accounting with the ip flow egress command might adversely affect network performance because of the additional accounting-related computation that occurs in the traffic-forwarding path of the router. “

  7. Mike Patterson Says:

    We found that earlier releases of NetFlow NBAR would drive up the CPU as well. This was fixed in IOS v15.1. Are you using Flexible NetFlow to export egress flows? I would try this. Also, monitor the CPU of the router with SNMP before and after.

  8. Anouar Says:

    Hi,

    I was surprised when I learned your post while we have in our network some routers interfaces configured with “ip flow egress” and we export Netflow v5 to the collector. Everything works fine with this configuration, so what I’m missing ?
    router : c2811 – 12.4(22)T5

  9. Mike Patterson Says:

    Sometimes the Cisco IOS allows you to type in commands that appear to be accepted when really they do nothing. Do you also have ingress enabled?
    Is the traffic ever over stated on the interfaces? My guess is that egress with NetFlow v5 isn’t really being exported. Have you captured the packets to confirm?

  10. Roland Says:

    Hi,

    Thanks for the great information, but still one question annoys me.
    How could it be that for v5 on routers I still can see Inbound and Outbound flows as well in Scrutinizer, but not for switches? On the routers I set up the ingress/egress on *only one* interface and the same thing is required for the switches. We do not want to monitor the traffic on other interfaces. I even tried v9 on switch, but still the same until I enabled the netflow (egress and ingress) on other interfaces.
    Thanks.

  11. mike@plixer.com Says:

    I’m trying to understand your question. If you have NetFlow v5 enabled and you are seeing outbound traffic then one of two things must be happening:
    * you have ingress enabled on 2 or more interfaces
    * you really have v9 enabled with egress

    Don’t be surprised if your catalyst switch lets you type in commands that don’t really work. In other words, you might not receive any errors from the console even if they don’t work. We have a catalyst 2900 and the CLI allows us to enable NetFlow, but it doesn’t work.

  12. Avaya IPFIX configuration - NetFlow & sFlow Network Monitoring - Systrax Says:

    [...] ingress only, therefore, make sure to enable flow exports on all the physical interfaces. In “NetFlow version 9: egress vs. ingress“, Mike Patterson explains why the practice is necessary in this [...]

  13. sk Says:

    Hi, some one can explain how to do sampling for egress packets

  14. mike@plixer.com Says:

    The answer is in this post: http://www.plixer.com/blog/netflow/beware-cisco-egress-netflow-with-flexible-netflow/

    Flexible NetFlow:
    ip flow monitor andrew -mon output

Leave a Reply

You must be logged in to post a comment.