1. Communications Infrastructure Committee
8/19/2009
Communications Infrastructure Committee
Update for LAN Managers
Friday, May 27, 2011

2. Initial Building List OT – Done* Clark - Done* Shepardson - Done*
ARBL - Done
A/Z - Done
NESB – In process
Engineering
VTH
Atmospheric Science
* Single GE for now
April 15, 2011
IAC Update

3. 10G Connectivity Proposals
One upgrade in process, as part of a funded NSF proposal
Evaluating 5 additional requests for high-speed connectivity
Goal is to make a funding decision very soon
April 15, 2011
IAC Update

4. Operational Management Discussion
Issues and Difficulties
Responding to problems/outages
Typical examples of problems
Edict from VPIT
Open Discussion
April 15, 2011
IAC Update

5. Issues and Difficulties
Switch Consistency (Type and Configuration)
Documentation (Change mgmt., ports, vlans)
Security (Physical and Virtual)
Resources
Hardware inventory
People – delegation of responsibilities (configurations, responding to outages, etc.)
Multi-departmental occupancy of buildings
New technologies
Radiation Safety Project
VoIP – exacerbated by all of the above
April 15, 2011
IAC Update

6. Issues and Difficulties (Cont’d)
900+ switches
120+ MDFs/IDFs
40,000+ systems
Video, Environmental Controls, Power Meters, Voice, Police/Fire, Security Panels, Servers, Wireless, Private College LANs, etc…
Main campus, West Campus, South Campus, Remote sites comprising
213 Subnets
60 Subnet managers

7. Response Issues w/ Current Model
Typically, 90% of time (2-4 hours, average) is spent figuring out what changed, then a quick fix (put it back the way it was)
10% of the time, problem is evident but very difficult to find the source and/or remedy
“Flying blind” is very difficult
We need to develop and maintain accurate cabling records
E911, CALEA Act Compliance, etc.
April 15, 2011
IAC Update

8. Small sample and examples of concerns
Environmental controls offline due to multicast storm, After hours response by Facilities, April 2011
Looped network takes down environmental controls requiring reconfiguration of switches and further monitoring after hours by Facilities, NOC, April 2011
Firewall not configured to be on network and thus not firewalling, March, 2011
Several building networks down along with environmental controls due to dual-homed server misbehaving. Discovered hubs and other devices as part of debug process. March, 2011
VOIP and Wireless down due to cables being moved to wrong switch ports. March, 2011
Compromised machines swamp entire LAN affecting police/fire, March 2011
Departmental servers down, nobody knew location of switches affecting servers nor configuration of switches thus requiring NOC to respond, find, fix, March, 2011
Unit carrying police/fire notifications moved to wrong port on switch. After hours response by NOC, Facilities, Feb. 2011
Lost communication on large campus network due to switches being plugged into each other incorrectly. Feb. 2011
Network device with duplicate IP takes down a College’s central server, Feb 2011
Emergency fiber run had to be made to bypass departmental network device incapable of handling vlans for cardkey access, Feb, 2011

9. Directive, from VPIT
Must ‘harden’ our network for future applications, including VoIP
April 15, 2011
IAC Update

10. Discussion
Operational management of switches is becoming more complex, and more critical
QoS, 802.1X, VoIP, documentation, life & safety, etc.
Need to “harden” the network, both physically and operationally
How can we do this, improving integrity of the network without impeding day-to-day functionality our users require?
Virtual vs. Physical Demarc
April 15, 2011
IAC Update