1 Information Security Vision Part II Network Planning Task Force 10/8/2003 Deke Kassabian and Dave Millar

2 Security: Defense in Depth ■Prevent ■Patch management tools & services ■Training & education ■Anti-virus scanning on mail servers ■Firewalls/Router filtering ■Virtual Private Network ■Personal firewalls ■Secure out-of-the box ■Campus-wide vulnerability scanning ■Detect ■Intrusion detection ■Respond ■Find a better way to locate compromised and vulnerable DHCP and wireless devices.

3 Simple Building Network Router switch Switch switch

4 Simple Building Network, Firewall for all of subnet Router switch Switch switch

5 Simple Building Network, Firewall for all of subnet Pros: ° More coverage from one FW device Cons: ° Blunt instrument, may subject too many things to one set of rules ° Problematic for network management Router switch Switch switch

6 Simple Building Network Router switch Switch switch

7 Simple Building Network, with firewall for servers Router Switch switch

8 Simple Building Network, with firewall for servers Pros: ° Excellent server- or service-specific protection possible Cons: ° May require server moves Router Switch switch

9 Simple Building Network Router switch Switch switch

10 Router switch Switch switch Simple Building Network, Firewall for one workgroup

11 Simple Building Network, Firewall for one workgroup Pros: ° Group-specific control and protection Cons: ° Can still be a blunt instrument ° Still problematic for network management Router switch Switch switch

12 Simple Building Network Router switch Switch switch

13 Simple Building Network, using VLAN Firewall Router switch Switch switch

14 Simple Building Network, using VLAN Firewall Pros: ° Very flexible in terms of participation ° Addresses net management problem Cons: ° Adds complexity and cost Router switch Switch switch

15 Perimeter Firewall: Current Situation Pros: ° Provides limited protection from common attacks Cons: ° Collateral damage ° No provision for legitimate access to risky services. Router switch Router Internet Router switch

16 Where to put a perimeter firewall? Router switch Router Internet Router switch

17 Router switch Router Internet Router switch Minimal perimeter filtering in edge routers

18 Router switch Router Internet Router switch Minimal filtering in campus routers

19 Campus VPN Service Router switch Router Internet Router switch VPN Gateway VPN Client

20 Campus firewall/VPN is not a panacea UniversityDate Netbios ports blocked # Windows machines # infected % infected Penn9/11/200311,0001,10010% Large state university 7/28/200312,0001,50013% Ivy League peer1/2/200218,0003,14617%

21 Campus VPN Service Pros: ° Allows us to block the most troublesome services and permit legitimate use. Cons: ° Complexity and cost ° Traffic is not encrypted on PennNet. ° Given the transient nature of PennNet this will at best stave off attacks for a few days Router switch Router Internet Router switch VPN Gateway VPN Client

22 Local VPN Service Router switch Router Internet Router switch VPN Gateway VPN Client

23 Local VPN Service Pros: ° Allows Schools and Centers to implement more restrictive firewall policies. ° Unencrypted traffic need not travel over PennNet. Cons: ° Complexity and cost VPN Client Router switch Router Internet Router switch VPN Gateway

24 Personal Firewalls (desktop & server software) Router switch Switch switch

25 Reviewing Terminology ProsConsEffective for ■ Filtering router – a relatively blunt tool that allows you to block services by port number and IP address on routers. ■ Can be economical if existing routers support filtering. ■ All or nothing. If a service is blocked inbound or outbound it is blocked completely ■ Can affect router performance ■ Can limit flexibility as new network services are created or requested by end users. ■ Temporary response to imminent or active threats. ■ Blocking services that are generally agreed by the campus community to pose excessive risk. ■ Firewall – a more robust security device that supports more complex security policies. ■ Greater flexibility: some allow you to inspect packets and block problematic traffic without blocking all traffic (e.g. block Code Red worm without blocking all web traffic). Other features allow you to permit inbound traffic if it is in response to a legitimate connection that was initiated internally (“stateful packet filtering”) ■ Expense, complexity ■ Can limit flexibility as new network services are created or requested by end users. ■ Departments or workgroups desiring more than only a basic level of security.

26 Reviewing Terminology ■ Security Policy – This term, when used in connection with firewalls and filtering routers, is generally taken to mean what kinds of network services you permit into and out of your network. A firewall or a filtering router is the physical device that enforces the security policy. These are the rules of what kinds of traffic are permitted and what kinds aren’t. ■ VPN - Security policies sometimes block services that users need to use from home or on the road (e.g. Outlook). A VPN, or Virtual Private Network is server software and (usually, but not always) client software that establishes a secure connection and permits authenticated remote access to services otherwise blocked by the firewall security policy. In other words, a VPN allows you to make exceptions to the broad policy, when necessary. ■ VLAN – A firewall or filtering router has to be placed on the “choke point” between the machines inside the firewall and the external insecure network. Without VLAN (Virtual Local Area Networks) technology, expensive wiring projects are often required to isolate the workgroup from other building occupants’ network connections. For example, in a shared building, VLAN technology allows us to isolate one or more workgroups from one another and establishes a virtual choke point so that a firewall can protect the workgroup without affecting others in the building. In summary, a VLAN removes internal building physical constraints, allowing a firewall to be established within a building regardless of individuals’ locations.

27 Time- frame TargetRecommendations Long- term Servers, desktops and workstations Evaluate a network design and migration strategy that better balances availability against security, and capable of supporting more flexible use of internal and external router filtering/firewall technology… under evaluation Near- term Servers, desktops and workstations Provide a basic level of network security by implementing router filter rules on external interfaces after campus consultation. Support department workgroup firewall requirements with firewalls and VLANs, or other topologies (see below) Near- term Desktops and workstations Evaluate and recommend a personal firewall software product for targeted users and do a pilot evaluation. Software license for users…………………..………$ $5000 for 3 years Firewalls Recommendations & Estimated Costs

28 Firewalls Recommendations & Estimated Costs* Time- frame TargetRecommendations Near- term Servers, desktops and workstations ■ Enable Schools and Centers to implement local security policies: ■ Select a campus firewall and VPN standard, avoiding a campus VPN “Tower of Babel”, buying time to patch systems, and enabling Schools and Centers to better implement local firewalls and VPN gateways………………………………………………………………………..……under evaluation ■ Design, implement and manage VLAN’s within buildings on request. This is the first step in allowing one or more workgroups in the same building to place their desktops, servers and workstations behind a firewall without affecting other workgroups in the same building. ■ Design and implementation costs…………………………………………….………………..…$1,300 ■ Annual, ongoing maintenance – $2.50 per port (16 ports) per month & $1,000 ………...$14.80/mo ■ After establishing a VLAN to isolate a workgroup from their building neighbors, the next step is to select, configure and manage a firewall. For workgroups on campus that do not want to do that themselves, create a new ISC Firewall and VPN management service: Firewall & VPN for under 25 users Firewall and VPN for workgroup of users & 2-5 workgroup servers Hardware and software$3,000 – 5,000 every 3 years $15,000 - $20,000 every 3 years Hardware/Software Maintenance $500 - $1,000/yr$3,000 - $4,000/yr Configuration and design (one-time) $500 - $2,000$1,000 - $2,500 Management and support$2,500 - $5,000/yr$5,000 - $15,000/yr *Note: Cost estimates assume internal staffing. For 3 rd party consulting service, add 20 – 30 %.

29 Security: Defense in Depth ■Prevent ■Patch management tools & services ■Training & education ■Anti-virus scanning on mail servers ■Firewalls/Router filtering ■Virtual Private Network ■Personal firewalls ■Secure out-of-the box ■Campus-wide vulnerability scanning ■Detect ■Intrusion detection ■Respond ■Find a better way to locate compromised and vulnerable DHCP and wireless devices.

30 Secure out-of-the box ■ When someone buys a new Dell or IBM machine in the Fall Truckload sale, it is very easy for them to complete the system install without creating an Administrator password. This has led to hundreds of compromised machines within days of going on PennNet. ■ Recommendation: Work with Dell, IBM and Microsoft to create more secure default images for newly purchased Penn machines ………………...…negotiated price < $25/image

31 Security: Defense in Depth ■Prevent ■Patch management tools & services ■Training & education ■Anti-virus scanning on mail servers ■Firewalls/Router filtering ■Virtual Private Network ■Personal firewalls ■Secure out-of-the box ■Campus-wide vulnerability scanning ■Detect ■Intrusion detection ■Respond ■Find a better way to locate compromised and vulnerable DHCP and wireless devices.

32 RPC DCOM Scan results

33 Campus-Wide Vulnerability Scanning ■ Vulnerability scanning has been a very useful tool in helping to manage the patching process University-wide. ■ Focused, campus-wide scans for single vulnerabilities campus- wide can be done quickly (i.e. in 45 minutes or so). This is very helpful to defend against a particular worm. ■ Broader, campus-wide scans for top twenty vulnerabilities take longer – more like 3-4 weeks. This is very helpful for “good hygiene” to ensure that we stay patched as new machines come onto PennNet over time. Goal: 1-3 weeks/campus-wide scan. ■ Finding contact information for the owners of thousands of DHCP (wired and wireless) is difficult and time-consuming. ■ The effectiveness of campus-wide scans is limited to the extent that laptops come and go at different locations on PennNet. We probably miss many vulnerable laptops if they’re “off-line” at the time we scan.

34 Vulnerability Scanning Recommendations ■ Develop more efficient methods of locating the owners of DHCP (wired and wireless) devices. See “Respond” section below.

35 Security: Defense in Depth ■Prevent ■Patch management tools & services ■Training & education ■Anti-virus scanning on mail servers ■Firewalls/Router filtering ■Virtual Private Network ■Personal firewalls ■Secure out-of-the box ■Campus-wide vulnerability scanning ■Detect ■Intrusion detection ■Respond ■Find a better way to locate compromised and vulnerable DHCP and wireless devices.

36 How do worms spread? Router switch Router Internet Router switch ■ 60% of the time attack Penn systems ■ 40% of the time: attack external systems

37 How did we learn about Blaster/Welchia infected machines? ■ Outsiders automatically send personal firewall logs to central services like MyNetWatchman, who in turn the report to us. ■ Penn people have automated extracts from their firewall logs and us the results. ■ We are automatically scanning our firewall logs and extracting the results every four hours. ■ Strengths: simple approach, inexpensive ■ Weaknesses: limited coverage, relies on Penn people’s willingness to continue to devote the time.

38 Improving detection Router switch Router Internet Router switch IDS Box

39 How could we improve our detection capability? OptionsProsCons IDS box connects to local switches Inexpensive Limited visibility IDS box connects to internal routers Broader visibility More expensive equipment – e.g. fiber taps. IDS box connects to edge routers Complete visibility of outbound attacks Technically challenging given our redundant internet connectivity. Most expensive Use edge router flow logs Limited visibility of outbound attacks Less expensive, challenging than IDS on edge routers.

40 Targeted Intrusion Detection Recommendations & Estimated Costs Near-termCreate policy establishing authority of ISC, Schools and Centers to conduct intrusion detection addressing privacy, log retention and related issues………… no incremental cost Near-termDeploy and manage six-ten switch-connected IDS boxes to quickly identify compromised campus systems. Hardware………………………$15,000-$20,000 every 2-3 years Staff to configure, manage, analyze IDS systems and follow up on intrusion reports………………….…$100,000/yr Long-termEvaluate and determine best method to provide router flow logs for intrusion detection……………………………..under evaluation Long-termEvaluate a network design and migration strategy that better balances availability against security, and capable of supporting broader intrusion detection ………… under evaluation

41 Security Vision: Defense in Depth ■Prevent ■Patch management tools & services ■Training & education ■Anti-virus scanning on mail servers ■Firewalls/Router filtering ■Virtual Private Network ■Personal firewalls ■Secure out-of-the box ■Campus-wide vulnerability scanning ■Detect ■Intrusion detection ■Respond ■Find a better way to locate compromised and vulnerable machines as well as targets of copyright complaints.

42 How do we find problem machines? ■ Problem machine is reported either by the RIAA, intrusion detection or campus vulnerability scan. ■ If static IP – look it up in assignments. ■ If DHCP – ask NOC for a port trace which translates the DHCP address to a physical location.

43 Current situation ■ Unable to respond to problems with wireless machines on parts of campus. Can’t very well disable the port :-( ■ Had to just drop cases of infected machines because of short DHCP lease lengths. ■ Requested 1149 port traces and 126 disconnects in calendar year 2003 (~$45,000). Trend is up (requested 270 in one week in September). ■ Had to hold off requesting some disconnects because it would have been unmanageable.

44 Incident Response Recommendations & Estimated Costs Near-termProvide tools to better support quick lookup of host and DNS contacts……………… …….under evaluation Near-termTargeted deployment of PennKey authenticated network access in, for example, College Houses, GreekNet, Library… $2,000 - $5,000/bldg Long-termFull deployment of PennKey authenticated network access on campus Hardware/Software (one-time)………… $1,000,000 Near-termResearch ways of ensuring security of newly connected machines: Vulnerability scan of machines as they connect to PennNet Ability to block infected/vulnerable machines based on MAC address Hardware/Software…………...………..under evaluation Staff………………………………………under evaluation

45 Next Steps & Estimated Costs Initiative FY 2004 ISC School/ Center FY 2005 ISC School/ Center FY 2006 ISC School/ Center Security patch policy………………………………………………………………? Create a new ISC Patch Management Service Staff……………………………………………………………………….$100,000/yr Hardware for campus SUS service…………………….…...$10,000 every 2-3 yr Software – 1000 seats……………… ……..…$6/seat/yr Virus scanning on pobox..………………….……..………….$5-$6/account/year Network design supporting internal and external router filtering/firewall technology………………………………………………………….under evaluation Support filter rules on external interfaces after campus consultation. Personal firewall software selection/pilot Software license for users……..……………$ $5000 for 3 years Select campus firewall and VPN standard……….……..……under evaluation Design, implement and manage VLAN’s within buildings on request Design and implementation ……..………………………………………………? Annual, ongoing maintenance……………………………………………….…..? Managed firewall service -- estimates per firewall, based on internal staff Hardware and software……………………...$3,000 - $20,000 every 3 years Maintenance …………………………………………………..$500 - $4,000/yr Set-up………………………………………………………..………$500-$2,500 Support…………………………………………………………$2,500 - $15,000

46 Next Steps & Estimated Costs Initiative FY 2004 ISC School/ Center FY 2005 ISC School/ Center FY 2006 ISC School/ Center More secure default images for newly purchased Penn machines.< $25/image Create Intrusion Detection policy no incremental cost Deploy targeted campus intrusion detection systems Hardware………………………………………$15,000-$20,000 every 2-3 year Staff……………………………………………………………………$100,000/yr Router flow logs for intrusion detection…………………………under evaluation Network design supporting broader intrusion detection…..….under evaluation Tools for fast Host and DNS contact lookup…………………..under evaluation PennKey authenticated access in targeted locations………2,000 - $5,000/bldg Full deployment of PennKey authenticated network access Hardware/Software (one-time)……………… $1,000,000 Implement two additional functions in PennKey network authentication of DHCP connections: Vulnerability scan of machines as they connect to PennNet Ability to block infected/vulnerable machines based on MAC address Hardware/Software…………...…………………………………under evaluation Staff……………………………………………………………….under evaluation