CLEANACCESS Archives

December 2009

CLEANACCESS@LISTSERV.MIAMIOH.EDU

Options: Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Isabelle Graham <[log in to unmask]>
Reply To:
Cisco Clean Access Users and Administrators <[log in to unmask]>
Date:
Tue, 1 Dec 2009 15:16:58 -0500
Content-Type:
text/plain
Parts/Attachments:
text/plain (32 lines)
We've recently run into a problem here that I wanted to share as a cautionary tale.
We're in the process of upgrading our network design to be more decentralized and fault tolerant. In 
planning for this we realized that converting CCA to out of band would compliment the new network 
design as well as ease some architectural challenges. So we've been migrating a building at a time 
to the new network model and OOB concurrently.
Recently, we started seeing two apparently separate problems. Some users would log in to CCA as per 
normal, but then get bumped back to the authentication VLAN and prompted to log in again. Other 
users would log in, receive errors and then be unable to receive any network traffic at all. Looking 
at the switch and CCA logs we were able to determine that the first issue was caused by an 
additional link-down trap being sent after successful login, in response to which CCA was resetting 
the port (as it should). The second issue was being caused by ports being set to disabled, but there 
was no clear reason why.
In troubleshooting this further with Cisco TAC, we found that on the few Catalyst 4000 model 
switches we had migrated, utilization was extremely high. This was causing them to be unresponsive 
to SNMP traffic and process SNMP commands out of order, or drop commands entirely (causing the port 
bouncing and disabling we were seeing). However, we also found out that when this happens a great 
deal with one or two switches, the CAM will get overwhelmed and will start dropping SNMP 
communication with other switches, causing the problem to cascade through-out the network (even on 
newer hardware).
The solution we have currently implemented is to switch to MAC based notification instead of link 
based notification, which is configured both in the switch settings in the CAM and in the switch 
configurations themselves. This should lighten the SNMP load enough that utilization comes down on 
both the 4000s and the CAM. Currently it seems to be working.
The takeaways that I wanted to pass on to the list are that MAC based notification seems to be a 
better option in terms of lower overhead (both network and CAM) and that the older 4000 switches, 
while supported, don't work well with OOB and should be upgraded if possible before deployment.

-- 
Isabelle Graham
Information Security
American University

ATOM RSS1 RSS2