Herding Cats and Campuses: Addressing Distributed Security and Compliance Issues Educause Security Professionals Conference - April, 2007 Kathy Kimball and David Lindstrom The Pennsylvania State University

Outline Penn State Background Universities and Network Threats Legal and Regulatory Landscape The Challenge Facing Us The Information Privacy And Security (IPAS) Project Origin Sponsorship Administration Overview Staffing Phases Necessary Support

Penn State “One University Geographically Dispersed” 24 campuses statewide Also agricultural extension offices, recruitment centers and other distributed operating sites World Campus - provides distance learning opportunities globally VPN to allow remote connectivity to resources otherwise blocked by border router filters Fall 2006 Students: 83,721 (42,914 at University Park) Faculty/Staff: Full time: 22,478; Part time: 39,464 One backbone network supports almost all functions (Internet Connectivity goes back through University Park)

We Are…Very Large

We Also Deal With a Lot of Data

How Much??? One Terabit is roughly equivalent to 32 million two-hundred fifty page books By that measure, for the high month during the first six months of 2006, the data backbone transferred the equivalent of approximately 88,000,000,000 two-hundred fifty page books. (Or 2,838,709,677 of them per day rough average).

Penn State - More Numbers Typical Day: more than 100,000 individual computers are connected > 1.5 million authentication actions by 120,880 unique Access account users Doesn’t include all the College and Department logins 28 February: More than 54,000 systems (of the 100,000) communicated out to the Internet More than 2,900,000 separate systems attempted to “talk to” Penn State from the Internet 10% of the traffic coming from the Internet to Penn State that day was blocked by filtering at the border. (In other words, it was likely hostile activity subject to very simple blocks)

Universities and Network Threats “We’re Special…I Guess”

University Characteristics Certain Characteristics of Colleges and Universities Make the Security Problem More Difficult Distributed Governance Varying User Needs/User Populations Cultural Tradition of Independence Emphasis on committees and consensus Comparatively slow-moving process facing a fast- moving threat

Challenging Network Threat Climate Global network is a hostile place Constant probes Security is dependent on non-technical users Insecurity anywhere can affect the whole “Monoculture” intensifies attack effects If a new Windows flaw is discovered, it could enable rapid exploit spread due to Microsoft’s market dominance

Hostile Probes - 28 February (A Fairly Typical Day) Exploits against Penn State were attempted from multiple locations in the United States and abroad including: Korea, Japan, Brazil, United Kingdom, Russia, Chile, Austria, Uruguay, Turkey, Taiwan, Switzerland, Spain, Peru, Mexico, Kuwait, Italy, India, Hungary, Hong Kong, France, Argentina, Africa Top hostile probe award went to a single system in Spain with 948,708 hostile attempts (ssh brute force)

Trends: What’s Increasing? Sophistication level of network attacks (Bots, bots and more bots) Complexity of detecting and removing residual malicious software Number of vendor security updates Mobility Laptops and PDA’s connecting to uncontrolled networks and returning

Trends: What’s Decreasing? Amount of time for global spread (worms) Though less impetus to do so (rise in criminal exploitation that is profit motivated) Ability to prevent intrusions at the network border Amount of time available to install vendor security updates Amount of time to detect and defeat a network-based attack

Legal and Regulatory Landscape When in Doubt, Pass a Law (or Write a Policy) - Controlling the Uncontrollable

Privacy and Security Policy Overview Primary Penn State Policies related to Privacy and Security AD11 - University Policy on Confidentiality of Student Records AD19 - Use of Penn State Identifier and Social Security Number AD20 - Computer and Network Security AD22 - Health Insurance Portability and Accountability Act (HIPAA) AD23 - Use of Institutional Data AD35 - University Archives and Records Management AD53 - Privacy Statement ADG01 - Glossary of Computerized Data and System Terminology ADG02 - Computer Facility Security Guideline

Policy Overview - Continued We have an institutional duty to reasonably secure sensitive data entrusted to our care The network is distributed and so is security responsibility Deans and Administrative Officers are responsible for establishing security policies in their areas The local policies have the force of overall University Policy, and are intended to guide system administrators in the development of detailed procedures enabling secure operation of local networks

Network Policy In addition to overall University Policy and local policies/procedures, attachment to the network requires: a network administrative, technical and security contact Responsible for a designated range of network addresses The contacts are critical in incident notification Only a network address is generally known for university systems when response begins Accuracy of the contact list is a unit responsibility

Additional Policy Points Units handling administrative data have additional requirements as outlined in the Trusted Network Specifications ( Units with an exception to hold Social Security Numbers locally have even more requirements (under AD19) There is, however, a perceived gap between Policy and performance for a number of reasons

Legal Landscape Applicable Laws and Regulations (Partial): FERPA HIPAA Graham Leach Bliley The Pennsylvania Breach of Personal Information Notification Act [73 P.S. § 2301 et seq ] FACTA PCI-DSS (Credit card industry security standards) Undoubtedly more coming…Watch this space

The Challenge We MUST Do Better or What Part of “Comply” Don’t We Understand

Universities in General Have “Issues” we MUST Correct Two sources with slightly different numbers, but the news isn’t good: Educational institutions accounted for over 50 of the more than 300 major data breaches in 2006, according to the Privacy Rights Clearinghouse, exposing Social Security numbers, bank account information and other sensitive personal data According to the Treasury Institute for Higher Education “…of the 321 information security breaches nationwide reported in 2006, 84 – or 26% – were at education institutions. This 26% share for Education is particularly disproportionate when we consider that education represents only a small percent of total payment activity nationwide. As a result, financial institutions and card issuers increasingly view education institutions as risky merchants ”

Need to Improve Improving the state of privacy and network security practices is essential It’s a distributed problem; it requires a distributed solution We Must: Raise the bar with regard to security practices and policies Assure compliance with existing university policies and laws affecting Penn State Improve our ability to respond to new laws (And do this even in light of our distributed nature and management structure)

Information Privacy And Security (IPAS) Project Origin Joint Effort – two year project planned. Loosely based on the model used for Social Security Number conversion. Pushed strongly by: Information Technology Services Corporate Controller Planning began in July 2006 and was approved in November 2006 Planning documents were staffed via both chains (business/finance and IT) Various funding models explored. Ultimately central funding with a split between budgets/budget execs was adopted

IPAS Project Executive Sponsors Provost, Chief Financial Officer Jointly Oversight: University Controller Vice Provost for Information Technology Services

IPAS Project Administration Similarly, a joint effort between: Senior Director, Security Operations and Services, Information Technology Services – Kathleen Kimball Chief Privacy Officer, Corporate Controller – David Lindstrom (Advantage: Both business and academic sides are represented in the project administrative structure, as well as the senior executive management structure)

Project Overview IPAS is a large-scale, multi-year, multi-phase effort with University-wide scope Phase I - Evaluate (and remediate if necessary) PCI- DSS systems and networks Phase II - Take lessons learned and apply to systems and networks handling sensitive University information (There is overlap, with some Phase II tasks coinciding with Phase I. The Project Team has already begun to contact units)

IPAS Project Staffing Three project team members – temporarily assigned for the duration of the two-year project. (Project Manager, Senior Network Analyst, Project Technical Coordinator) Leadership of distributed units provided the staff resources for the project: ITS, Consulting and Support Services Student Affairs Research Information Systems

You’re Going to Make Us Do What? Initial Reaction by the Governed:

Phase I Very detailed requirements More than 100 merchant id’s University-wide Payment Card Industry Data Security Standard (Version 1.1) Qualified data security company is engaged (Ambiron Trustwave) Security scans required quarterly. Security Operations and Services also performs internal scans (ISS and AppScan) Bursar and eCommerce server evaluated and deemed compliant by the end of December 2006

Sample Requirement “Build and Maintain a Secure Network” The Devil is in the details. This objective breaks out to two main requirement sections with multiple subsections under each: Example -- Requirement 1: Install and maintain a firewall configuration to protect cardholder data  1.1 Establish firewall configuration standards that include the following:  A formal process for approving and testing all external network connections and changes to the firewall configuration  A current network diagram with all connections to cardholder data, including any wireless networks  Requirements for a firewall at each Internet connection and between any demilitarized zone (DMZ) and the internal network zone  …[through 1.1.9]

When in Doubt The twelve top level requirements and all of the detailed requirements are available through: We also have a brochure with all contact information

Incident Response Involving Credit Card Data Users or Distributed Contacts are instructed to contact immediately. Published 24/7 number There are significant University-level reporting requirements associated with PCI-DSS. Security will coordinate with all of the parties that must be notified (Privacy, Police Services, University Legal Counsel, University Relations, Audit, etc.) The level of protection/accountability associated with the compromised network will rise in the event of a breach. Independent forensic analysis and gap analysis may also be required Fines may apply

Phase II Overall privacy and network security improvement for University data (some of which is equally as sensitive as credit card data) Review and improve existing policy (beginning with overall data classification) Evaluate existing (and projected) law Consider the likely evolution of the threat

Selected Phase II Tasks Distributed risk assessment process definition/refinement Evaluate/improve security role in the software development life-cycle Examine current security organizational structure (University-wide) and recommend improvements Define and implement a more effective distributed compliance and enforcement strategy Define a more formal University-wide security and privacy training strategy for distributed IT staff to include mandatory initial courses and ongoing professional development courses thereafter

Selected Phase II Tasks (Continued) Examine and recommend changes to both central and distributed security staffing levels Examine and refine security and privacy related job descriptions to formalize qualifications for employees Examine performance based incentives within the Human Resource system such that staff attaining a defined level of security proficiency are rewarded Examine any architectural changes in the University backbone network architecture that would facilitate better unit security Examine and implement better log aggregation and network admission strategies Develop more focused end user training programs

Selected Phase II Tasks (Continued) Examine in depth existing University and distributed unit policies In short, we’re looking at the whole security infrastructure (people, policies and technologies) with no sacred cows (or cats as the case may be)

Project Implementation and Success Budget Executive support is crucial Other unit IT and financial personnel must be involved as designated by the Budget Executive

Required Support An overall project steering committee will exist. Some Budget Executives will be asked to serve and to advise their colleagues Each Budget Executive must assign the following staff to work with the IPAS Project Team for both Phases. All Contacts will be required to attend training on at least an annual basis. First session is April 13th: Technical Contact Financial Contact Administrative Contact

We CAN Make a Difference We can and must integrate more effective security while maintaining the openness essential to academic institutions IPAS will help define and implement solutions that accomplish these objectives

Where Are We Now? We are Busily Leading The Masses to Water -- And Some are Even Enjoying It…

The End… Questions? (Hiding is Futile; We Will Find You)