Module 8 ********* Data Validation & Security Workforce Information Database Training Last update November 2006

Providing Accurate Data Consistently Data validation allows us to provide data with confidence in its accuracy, and we can consistently provide this data by implementing thorough security. Module 8 Data Validation & Security

Data Integrity Validity, consistency, and accuracy of the data in a database. Table-level Field-level Relationship-level Business Rules Module 8 Data Validation & Security

Module 8 Data Validation & Security Data Validation The process of determining if an update to a value in a table’s data cell is within a preestablished range or is a member of a set of allowable values.

What are some common data quality problems that affect data integrity and validation? Module 8 Data Validation & Security

Common Data Quality Problems Illegal values Violated attribute dependencies Uniqueness violation Referential integrity violation Missing values Misspellings Cryptic values Embedded values Misfielded values Module 8 Data Validation & Security

Common Data Quality Problems continued…. Word transpositions Duplicate records Contradicting records Wrong references Overlapping data/matching records Name conflicts Structural conflicts Inconsistencies Module 8 Data Validation & Security

What are some methods for checking for data validity? Module 8 Data Validation & Security

Methods for checking for data validity Visual/manual Aggregation Reviewers guide Auto data checks Record counts Spell checks Have data provider review Module 8 Data Validation & Security

Time to Exercise!! Module 8 Data Validation & Security

Network permissions Physical security (including backup & restoration plan) Data Storage Access (Server or PC) Web security Application security User level ALMIS Database Security Considerations Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Security Concerns: Confidentiality Integrity Availability User level Application security RDBMS ODBC Suppression flags Data aggregation issues Network (LAN/WAN) security The contact in my state is:______________ Production vs. test Module 8 Data Validation & Security

Network permissions Physical security (including backup & restoration plan) Data Storage Access (Server or PC) Web security Application security User level ALMIS Database Security Considerations Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Security Concerns: Confidentiality Integrity Availability User level Application security RDBMS ODBC Suppression flags Data aggregation issues Network (LAN/WAN) security The contact in my state is:______________ Production vs. test Module 8 Data Validation & Security

Physical Security Questions to ask yourself: Where is your data actually stored? Are you responsible for physical security? If you are... Module 8 Data Validation & Security

Physical Security What physical security measures do you have in place? Do you have the right hardware? Are you using obsolete hardware that is prone to crashing or hacking? Do you have a choice? Module 8 Data Validation & Security

Physical Security Do you have a database backup plan? Do you have remote backup so if a fire burns the building you don’t lose both your primary and backup data at the same time? Do you have a data restoration plan? Does the backup plan allow for feasible restoration? Module 8 Data Validation & Security

Physical Security Restoration Plan Does your restoration plan allow for the restoration of individual tables or require the entire database? Do you maintain copies of the tables on your hard drive? Module 8 Data Validation & Security

Physical Security Who has access to the physical storage location? Is your database on a PC or server that someone could easily kick, spill coffee on, or accidentally reappropriate? If you are not responsible for physical security, ask these questions of those who are! Module 8 Data Validation & Security

Network permissions Physical security (including backup & restoration plan) Data Storage Access (Server or PC) Web security Application security User level ALMIS Database Security Considerations Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Security Concerns: Confidentiality Integrity Availability User level Application security RDBMS ODBC Suppression flags Data aggregation issues Network (LAN/WAN) security The contact in my state is:______________ Production vs. test Module 8 Data Validation & Security

Data Storage Access What software are you using to store the database? SQL Server, Oracle, FoxPro, Access? Survey says: SQL Server48% Oracle28% Other24% Module 8 Data Validation & Security

Data Storage Access What can be done at the server level to provide for security? Confidentiality, Integrity, Availability DO apply advanced security to the most sensitive data, but DON’T apply advanced security to non-sensitive data Module 8 Data Validation & Security

Data Storage Access Security at the server and/or database level… Permissions What kind of permissions exist? Read only, SA, etc. Who sets those permissions? Who has those permissions? Module 8 Data Validation & Security

ALMIS Database Security Security at the data level… Suppression Suppression flags Suppression can be handled at the database level or the application level Module 8 Data Validation & Security

ALMIS Database Data Security Two approaches: 1. ALMIS Database contains NO confidential data (all data available for use without restriction) 2. ALMIS Database contains confidential data (data access controlled by security and/or suppression) Module 8 Data Validation & Security

ALMIS Database Security Tables that have suppression flags... ces indprj industry iomatrix occprj oeswage stindprj stoccprj NOTE: tables stfirms and wage have no suppression flags but may contain confidential data Module 8 Data Validation & Security

ALMIS Database Data Security Issues to consider regarding suppression: If your database doesn’t contain suppressed records, detailed data won’t aggregate to totals (validity checks?) Without proper suppression, confidential data can be back-calculated Module 8 Data Validation & Security

ALMIS Database Data Security A note on confidential data: If you are checking 202 data (or any other potentially confidential data) to see that it loaded right, make sure your EQUI data files and/or printouts are always secured and shred all photocopies when finished. Check with your local BLS personnel or LMI administrator for specific confidentiality policies. Module 8 Data Validation & Security

Network permissions Physical security (including backup & restoration plan) Data Storage Access (Server or PC) Web security Application security User level ALMIS Database Security Considerations Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database. Security Concerns: Confidentiality Integrity Availability User level Application security RDBMS ODBC Suppression flags Data aggregation issues Network (LAN/WAN) security The contact in my state is:______________ Production vs. test Module 8 Data Validation & Security

User Access Three major types of user access: PC - direct to database (ODBC or RDBMS) Network (LAN/WAN) through application Web (passive or active through application) Module 8 Data Validation & Security

User Access via the Web How do Web users access the data from the database? Passive (static tables automatically updated to web pages) Active (query through application) Module 8 Data Validation & Security

User Access via the Web Questions to ask yourself: What kind of web server are you using? What are the security considerations with that choice? What kind of firewall do you have? Module 8 Data Validation & Security

User Access via the Web Does your web interface have security/confidentiality suppression? Do you display suppressed data to certain users? If so, how is access managed? Passwords? IP address? Module 8 Data Validation & Security

User Access via a Network (LAN/WAN) Questions to ask yourself: What type of network system do you have? Who has rights/access to your data via the network? Through what applications? Are confidential data (suppressed records) available? Module 8 Data Validation & Security

User Access via a Network (LAN/WAN) How do you control access to confidential data? Through user Ids? Through server access permissions? DO recognize the important distinction between network security and data security. Module 8 Data Validation & Security

User Access via direct connection Questions to ask yourself: Is local access machine password protected? Who has access to your machine? Is there a backup plan for your access machine? Is the source data for your database secure? Backup plan? Restorable? Module 8 Data Validation & Security

ALMIS Database Data Security Documentation It may prove beneficial to keep detailed records on… How security is done Where security exists Who is responsible for security Who has access/permissions to what Etc... Module 8 Data Validation & Security

Data Security DISCUSSION Module 8 Data Validation & Security