Carol Sabbar and Jim Walsh Carthage College Towards a Better September: Controlling Residence Hall Computing

Surviving September? Can it really get better? Do we have any control at all? Who we are: Lowly IT people from a liberal arts college with a pretty limited budget and about 1,200 resident students whose computers are all infected…

Fall 2003 – Blaster! What happened? Blaster and Welchia infected nearly every student computer on campus DoS attacks shut down the core switch We shut down whole residence halls to protect the core We thought that students could help themselves… or not… We turned off ports for hundreds of rooms We cleaned and patched hundreds of student computers

We survived but… We resolved to never let that happen again We had to figure out something Identify what happened and why Figure out how to prevent it Figure out what we could afford

Determining the Causes Vlan110 is up, line protocol is up Hardware is Cat5k Virtual Ethernet, address is c6d.d800 (bia c6d.d800) Description: Hedberg User VLAN Internet address is /16 MTU 1500 bytes, BW Kbit, DLY 100 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters 2d00h Input queue: 1/75/1139/55 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue :0/40 (size/max) 5 minute input rate bits/sec, 37 packets/sec 5 minute output rate bits/sec, 25 packets/sec packets input, bytes, 0 no buffer Received broadcasts, 0 runts, 0 giants, 170 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored packets output, bytes, 0 underruns 0 output errors, 0 interface resets 0 output buffer failures, 0 output buffers swapped out

Identifying the Culprits Seeing the traffic Understanding how worms work Finding infected computers –How many are patched –How many have anti-virus The patch is not the fix, and anti-virus won’t clean these Finding the secondary problems: –Spyware –Peer-to-peer

Looking forward to 2004 Containing outbreaks –Can we isolate them to a building? –To a room? Preventing infections –Can we mandate patches? –SP2: good or bad? –Can we mandate anti-virus? Chasing down infected machines faster and easier –Can we identify infected machines?

What we couldn’t do… Some proven solutions just wouldn’t work for us. These included: Perfigo or Bradford software – too expensive Anything requiring an agent on a student computer – too many installations to do A Packeteer for ResNet users only – too expensive Broadcast storm control – anomaly related to our wiring plant in res halls

Isolating Outbreaks Subnets –Already in place but only the base ACLs (details on next screen) –Isolation to the building –Moving them out to the edge switches –Required new hardware –Required outside expertise Storm control on ports –Problematic in our environment –Maybe fall 2005

ACLs on Cisco Switches Extended IP access list 180 deny icmp any any (92 matches) deny udp any any eq tftp deny udp any any eq 135 deny tcp any any eq 135 deny udp any any eq netbios-ss deny tcp any any eq 139 deny tcp any any eq 445 deny udp any any eq 445 deny tcp any any eq 4444 permit ip host deny ip any (1410 matches) permit ip any any ( matches)

Preventing infections – Part 1 Distribution of Symantec anti-virus in summer 2004 –Changing our Symantec licensing to make it “free” –Mailing out the CDs, “Update before you get here!” –Handing them out at check-in –We do not yet check for its existence before network access

Preventing infections – Part 2 Patching –Is SP2 really recommended? –In November, we decided “yes” –Working on PatchLink for on-campus computers, but requires agent for student computers –We do not yet require any specific patches for network access

NetReg – a Tool for the Hunt Required registration in fall 2004 Decreases identifying infected machines by several steps Turned off rooms posted on our web site Need to have someone well-versed in Linux to set it up

Fall 2004 – Any Better? No dorms totally shut down in Sept We cleaned less than 20 computers in September and October Infections seldom traveled from building to building Infected machines were identified and ports unplugged the same day A different story in November… started in an administrative building with no ACLs

New Problems With the elimination of the bulk of virus-related outages, we experienced other problems: Rogue wireless/wired routers More spyware Issues with Windows settings like “connection bridging” and 802.1x Some education issues related to NetReg

Looking forward to fall 2005 Do all the same as last year Increase functionality of NetReg? Use a product like Cisco Network Access Control? Deploy more switches that can discard DHCP response packets Deploy our own wireless in res halls We’re open for suggestions…