Open source tools for Cisco admins

If you like testing out open source tools, you’ll probably like COSI, the Cisco-centric open source community. You’ll find tools such as Ciscocmd, which allows users to send a set of command to a large number of IOS target hosts and get a separate report for each; and Ciscoping, a Unix command-line utility that allows users to access the various options of ping on older versions of IOS. COSI encourages developers and customers to publish, discuss and share their open source tools, scripts and utilities for managing or operating Cisco gear.

If you missed last week’s announcement of Scrutinizer v7, here’s the deal: Scrutinizer NetFlow and sFlow Analyzer version 7.0 is absolutely free and includes several new features, interface enhancements and improved performance. You can read all about it  in the press release and view a webcast by Scrutinizer Product Manager Michael Patterson giving an overview of the new version.

You can see a list of the free network monitoring tools that Plixer uses and recommends here. Among the tools include Getif, an SNMP MIB browser; Qcheck, a utility for running response time, throughput and streaming tests; and Nmap, a network scanning utility.

* Successful VoIP Migration: This episode addresses jitter, latency, packet loss and security. (Cisco NetFlow supports all VoIP traffic, while Plixer’s WebNM monitors VoIP services from almost any vendor.)

* Invisible Attackers: Stop the Bot: This episode discusses tools and techniques used by hackers to infiltrate networks. (Also, read our two-part blog series on using NetFlow to tell if your network is part of a botnet.)

Transcripts of each TechWise TV episode are available on Cisco.com. Bite-size TechWise TV podcasts are also available for download. Users have to register at Cisco.com to view the videos but the podcasts can be downloaded via iTunes.

Cisco Press podcasts and videos
More educational videos and podcasts for Cisco network admins are available for free on Cisco Press’ InformIT site as part of its On Networking series. The videos and podcasts are interviews with well-known Cisco Press authors, such as Jeff Doyle, author of the Cisco networking bible Routing TCP/IP Volume 1 and Volume 2. Recent  topics include:

* Cisco ASA: All-in-One Security Plan: Omar Santos gives an overview of Cisco’s Adaptive Security Appliance. (Also, read here about how NetFlow is handled in the ASA.)

* Strategies for Learning IPv6: Chip Popoviciu discuss the importance of IPv6 training. (IPv6 is supported in the free Scrutinizer v7. Also, learn how to configure NetFlow on Cisco routers for IPv6.)

That’s it for Part 1 of this blog series. Tune in next week when will I tell you where you can get your hands on some open source tools and utilities for Cisco networking equipment, and share with you some favorite free tools as used by your friends at Plixer.

Update: Read part 2 here.

It’s not just home computers that could be part of botnets. Any work computer could be compromised if users unwittingly download malware or visit malicious Web sites, putting corporate networks at risk.  How can Cisco NetFlow be used to identify DDoS attacks?

Watch the flow behavior
Network traffic monitoring using Cisco NetFlow can help identify suspicious behavior.  Use the Scrutinizer Vitals to see if a recent spike in overall flow volume collectively from all your routers has occurred:

Once you identify the router kicking out massive amounts of flows, drill in to determine who is receiving the most flows:

Use flow analytics
Scanning for threats from external sources can be used to identify whether an internal computer is part of a botnet. The Flow Analytics module of Scrutinizer features an Internet Threats Monitor that monitors all connections in and out of the Internet for such behavior. Flow Analytics, when used with the RST/ACK Destination algorithm and SYN Violation algorithm can help catch network worms.

“Network Behavior Analysis with Flow Analytics is an important part of our NetFlow Analysis software,” says Michael Patterson, Scrutinizer product manager. Our solution looks for network threats across hundreds of routers and deduplicates flows to ensure an accurate Unique Index is compiled per host. DDoS attack behaviors can be identified with well engineered mathematical algorithms.”

Here are some interesting links for further reading on botnets:

How my computer became a zombie

How Can I Tell If My Computer Is Part of a Botnet?

Busting bots: Defending against botnets

Quick overview of DDoS

DDoS attacks are often caused by botnets flooding Web sites with requests thus bringing the site’s Web servers to their knees. A botnet is a collection of computers that have been compromised by viruses and worms so that they can be controlled by malicious individual(s). An example could be the collection of computers compromised by Conficker, however a Conficker botnet has yet to be leveraged to do harm.

In the case of Twitter, the irony is that it could have been the compromised computers of some of Twitter’s own users that caused the DDoS.

Are you part of a botnet?

So how do you know if your computer is part of a botnet? Here are some of the symptoms:

• Your Internet connection appears slower than usual, which could be a sign that the botnet is using your connection to send and receive data.
• Your computer seems slower than usual or crashes for no apparent reason.

Many worms have codes that change constantly making it difficult for antivirus software to detect them. DDoS attacks can be detected using Cisco NetFlow and Flow Analytics:

Learn more in part 2 of this blog series on how to identify a DDoS attack using NetFlow.

The Canadian Radio-Television and Telecommunications Commission is expected in November to issue guidelines for Canadian ISPs on how to manage Internet traffic and congestion. Cable companies and network management software providers issued their comments at the end of July for the CRTC to review.

Among the comments was a late submission from BitTorrent. You’ll recall that back in 2008, the then FCC Chairman Kevin Martin said he would recommend barring U.S. ISP Comcast from using P2P network traffic management technologies to slow down or block traffic to and from P2P sites such as BitTorrent. Comcast and BitTorrent later agreed to work together to solve rich media content and network capacity management problems.

Now, BitTorrent, which is based in San Francisco, is targeting ISPs in Canada. Some ISPs in Canada have been using various network traffic management practices to slow down BitTorrent performance. In its submission to Robert Morin, secretary general of the CRTC , BitTorrent claims there are “misconceptions” over the effect of its P2P application on the Internet, saying that there “has been an overstatement of the effect of such applications on network congestion.” It claims that its software and services are “devoted to making the Internet more efficient for consumers, publishers and network operators.”

BitTorrent criticizes “discriminatory network management” as having the “potential to stifle existing technologies in their infancy as well as new technology development.” It adds “network management need not run afoul of the principles of network neutrality and ISPs can effectively manage their users that induce congestion rather than discriminate against the general use of a specific application.”

It goes on to describe P2P as a cost-effective method to reach an audience, adding that the Canadian Broadcasting Corporation (CBC) in March 2008 distributed the final episode of its reality television program Canada’s Next Great Prime Minister for download via BitTorrent. However, BitTorrent says the download times for many Canadian users was longer due to the traffic management practices of several Canadian ISPs. It claims that it works with the ISP community to solve problems such as Internet congestion.

BitTorrent notes that moving large media files via the Internet can “overwhelm weak links in the network”, adding that the problem is driven by the popularity of consuming media on the Internet, and not a P2P problem. Rather, P2P “enables unreliable peers to be aggregated into a very reliable and efficient delivery system.”

BitTorrent explains that it is offering ISPs its technology called “uTP“, a transport service on top of UDP. It describes uTP as putting a “yield sign” in front of any traffic sent by the client and gives way to other applications that need the network. BitTorrent is trying to gain acceptance of uTP within the IETF and is co-chairing (alongside a Microsoft rep) a working group aimed at dealing with network congestion issues.

BitTorrent said it worked with Comcast to implement a network management policy that manages heavy users, rather than applications. And only in times of intense congestion.

As we’ve discussed before, when employees are at home, they can decide for themselves whether they want to use P2P services such as BitTorrent on their own networks. But when it’s on the company’s network, it becomes the company’s responsibility. Cisco NetFlow with Scrutinizer’s Flow Analytics can monitor Internet traffic and identify P2P activity.

We’ve also discussed how ISPs can fix congestion problems. Fixing the issue may not mean immediate intervention with traffic flows rather, sometimes a NetFlow billing solution for traffic above a threshold during peak hours is the best way to encourage customers to police themselves.

For in-depth reports about the CRTC hearings, go to CBCnews.ca, and in particular the story headlined “CRTC to decide on new rules for internet service providers“.

For instance, Alessandro Acquisti, a researcher at Carnegie Mellon University is going to show how information about an individual’s place and date of birth can be exploited to predict his or her Social Security Number. To cut a long story short, Acquisti says SSNs were designed to be simple identifiers and not for authentication purposes, and so businesses should stop using them as confidential passwords.

We know enterprise networks are big targets for cybercriminals. Here are some Black Hat Vegas briefing sessions by security professionals about new attacks that could be around the corner and how to protect against them. Slides from the presentations are expected to be available at the Black Hat site after the event. Slides from January’s Black Hat DC 2009 briefing sessions are here.

What happened to Conficker’s payload?
Security researcher Mikko Hypponen says the Conficker/Downadup worm infected several million Windows workstations and servers around the world. The worm uses several new techniques that have never been seen before, and can create a unique list of 250 random domain names everyday. The creators of Conficker had the power to seize control of all of the computers the worm had infected. Yet, nothing happened. There were no reported botnets, no spam and no data theft. Hypponen says his Black Hat Vegas talk will reveal the motive of Conficker’s developers and why they never pushed the payload button.

Could a large part of the reason be the industry’s coming together to fight the Conficker worm? As I mentioned in my last blog post, the security industry reacted swiftly by creating the Conficker Working Group, and companies such as Cisco ensured their products protected against the worm and educated customers on how to strengthen their protection against Conficker. And of course, Cisco NetFlow Analyzer and Flow Analytics can catch Conficker.

How fast are you closing security vulnerabilities compared to your competitors?
A group of security researchers have examined six vertical industries, including finance, healthcare and manufacturing, to determine the time-to-patch trends. The result is a “half-life period” — the period it takes each industry sector to patch 50% of the vulnerabilities discovered after the first security advisory. The researchers say organizations can use the findings to measure their time to patch against others in their industry.

What’s the value of your network?

Rob Beckstrom, former director of the National Cyber Security Center in the U.S. Department of Homeland Security will discuss Beckstrom’s Law at Black Hat Vegas. The law attempts to answer the question of “how valuable is a network?” (network could include social networks as well as electronic networks). According to Beckstrom, the value could be extrapolated by “looking from the edge of the network at all of the transactions conducted and the value added to each. It states that one way to contemplate the value the network adds to each transaction is to imagine the network being shut off and what the additional transactions costs or loss would be.”

Your network is compromised, but you can still keep you data safe
Security specialists Aaron LeMasters and Michael Murphy will share details of Codeword, their free tool that provides management and analysis during rapid enteprise triaging (RETRI) of compromised networks. The developers explain: “Rather than focusing on individual network segments or hosts, our approach prioritizes broad network isolation to contain the threat and ensures core business functions remain operable. The result is less strain on your IT staff and no downtime for your users.”

Details of all Black Hat Vegas sessions are here.

Cisco also warns that until social networking sites use “more robust protection”, cyber criminals will continue to target popular online communities to lure unsuspecting users to click to fraudulent sites or to download malware.

But it’s not all gloom and doom. Cisco says this year was a turning point in the ongoing battle against cybercrime as the security community and industry collaborated to fight the Conficker worm. Cisco praised the work of the Conficker Working Group.  At the Conficker Working Group website, you can check to see if you are infected with the Conficker worm, and access other resources.

Cisco recommends the following actions to ensure network security:

• Be proactive. Don’t wait to patch your systems.
• Understand the anatomy of an attack and use multiple types of security products and techniques that work well together to prevent the threat from moving to the next phase.
• Train end users to be security-aware. Ensure they understand the risks of using Web 2.0 collaborative tools, applications and mobile devices that you may not support.
• Know that older and unpatched machines could be compromised by hackers.
• Beware of the risk of insider attacks.
• Create policies that include antimalware, acceptable use policies, and data loss prevention.

Cisco NetFlow can be used to monitor your networks for malicious behavior. Here are a couple of interesting ways that NetFlow can be used for network traffic monitoring of suspect activities:

The Null Scan – You’re being watched
How using Cisco NetFlow with a behavior analysis tool can help identify Null scans, a type of TCP scan that hackers use to identify listening TCP ports.

How to detect spambots with Scrutinizer NetFlow Analyzer
How one company used Scrutinizer NetFlow to discover an authorized host sending mail through the corporate network.

Read Cisco 2009 Midyear Security Report here.

Google Chrome operating system

But the question that will be of most interest to you is whether and when you will need to monitor your enterprise network traffic for Google Chrome OS activity. Google says Chrome OS won’t be available until the second half of 2010. That timeframe is almost a whole year after the launch of Windows 7, slated in October.

Google says it is working with companies including Acer, Adobe, Hewlett-Packard, and Lenovo to design and build devices to support Chrome OS. Even though the first devices will be aimed at consumers, it might be worth analyzing your networks soon after such devices are launched to monitor for Chrome OS activity on your networks. You may find some users have connected their Chrome OS devices onto the network.

The other concern for enterprises is around security – how much of a security threat are Chrome OS devices going to present to enterprise networks? Much of Google Chrome OS will be based on open source technologies, including a Linux kernel. Google also says it is “going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with with viruses, malware and security updates. It should just work.” As we know, building a bullet-proof operating system is a tall order for Microsoft. What must it be like for OS upstart Google?

For consumers, Google promises that Chrome OS will be a modern operating system (Google says today’s operating systems were “designed in an era where there was no web”) that will be fast and lightweight and will start up and get you on the web in a few seconds. Undoubtedly, the OS will also help the movement toward cloud computing where applications, such as spreadsheets and word processing software, are easily available on the web, rather than tied to the desktop. Google’s promotion of cloud computing will also spur third-parties to develop all sorts of innovations, which should only be good for the consumer.

For enterprises, industry analyst Gartner says there won’t be a significant effect on IT operations for at least three years, and that organizations should not change plans to migrate to Windows 7 because of Google’s announcement. However network managers should prepare policies regarding the support of Chrome OS on their networks, says Gartner. Perhaps that should be in addition to any policies you may have now in place for Google Android devices?

Whatever new developments Google (and Microsoft in response) may bring to market, it’s certain that Cisco NetFlow will play a key role in analyzing and monitoring traffic from Chrome OS devices.

Cisco explains that the Cisco Certified Architect “recognizes the architectural expertise of network designers who can support the increasingly complex networks of global organizations and effectively translate business strategies into evolutionary technical strategies.”

Candidates must first already hold the CCDE certification and have approximately 10 years of industry experience before they can apply to enter the program. The certification doesn’t have the usual Cisco written or lab exam, but instead candidates are required to meet before an in-person review board made up of Cisco-appointed exam committee members. There, candidates defend their proposed network solutions and must be able to revise the proposals on-the-fly when challenged to by the review board.

The Cisco Certified Architect certification joins other similar programs that recognize IT professionals at this elite level. For example, The Open Group and Microsoft run similar programs named The Open Group IT Architect, and Microsoft Certified Architect, respectively. Both programs are aimed at IT professionals with about 10 years of industry experience and have designed IT infrastructures. Both also require candidates to interview with peer review boards as part of the rigorous certification process.

These programs do several important things: they help non-IT people recognize the strategic value of IT and that senior IT professionals should be viewed with the same level of importance as their chief finance people and corporate lawyers. These programs also give entry- to mid-level IT professionals a potential career path.

But even if you don’t see yourself becoming a Cisco Certified IT Architect, I think it would be a useful exercise to see what it takes to become one. That’s because many of the qualities of an IT architect (read senior IT professional) should be qualities that all IT pros should have.

The chief among these is an understanding of your employer’s business sector. Often employers lament that while their IT staff are great at what they do, they often lack an understanding of their business challenges. According to Cisco’s description of the Cisco Certified Architect, the individual “understands the impact of a company’s business on the design and operation of that company’s network,” and specifically “goes beyond technical decisions to consider how the network and business direction interplay.” Architects must also “ensure business requirements are incorporated into extensible, resilient, secure, and supportable architectures and designs.” They must also be able to “clearly communicate and advocate proposed network architectures to both executives and expert-level technical leaders.”

Indeed, hiring managers often prefer candidates that can demonstrate both technical skills and business knowledge.

The Cisco Certified Architect program is available January 2010 and cost $15,000.

Do you see the Cisco Certified Architect in your future?

And almost a year ago, city employee Terry Childs was arrested on four counts of computer tampering with the City of San Francisco’s multimillion-dollar FiberWAN, which holds much of San Francisco’s key records. Childs, who built and administered the network refused to hand over passwords to the network, effectively putting the city on lock-down.

The Texas and San Francisco cases are extreme examples of insider threats. Dong Chul Shin, the former Energy Future Holdings employee had been fired from the company the day he allegedly used his login details to access the corporate network, while San Francisco’s Childs had apparently placed sniffers on the network and discovered that his job was in jeopardy.

Such instances could happen again. A recent survey of 125 companies polled by SailPoint found that 57% of companies lack the transparency needed to prevent insider threats, and only 14% of organizations felt they have adequate controls in place to address the risk of insider threats. Some 17% said that they felt very concerned, and that “It was just a matter of time” before corporate security was compromised.

Although Shin was terminated from his employment he was still able to gain access to the corporate VPN later that day. This oversight is apparently all too common. A full 42% of respondents to the SailPoint survey said they do not have the ability to immediately remove access privileges after a massive layoff. (A full 40% of respondents said their organization had undergone a significant layoff in the last six months.)

So it appears that network monitoring could play a vital role here in mitigating any damages from insider tampering.