Data Validation & Security

Providing Accurate Data Consistently Module 8 Data Validation & Security Providing Accurate Data Consistently Data validation allows us to provide data with confidence in its accuracy, and we can consistently provide this data by implementing thorough security.

Module 8 Data Validation & Security Data Integrity Validity, consistency, and accuracy of the data in a database. Table-level Field-level Relationship-level Business Rules

Data Validation The process of determining if an Module 8 Data Validation & Security Data Validation The process of determining if an update to a value in a table’s data cell is within a preestablished range or is a member of a set of allowable values.

What are some common data Module 8 Data Validation & Security What are some common data quality problems that affect data integrity and validation?

Common Data Quality Problems Module 8 Data Validation & Security Common Data Quality Problems Illegal values Violated attribute dependencies Uniqueness violation Referential integrity violation Missing values Misspellings Cryptic values Embedded values Misfielded values

Common Data Quality Problems continued…. Module 8 Data Validation & Security Common Data Quality Problems continued…. Word transpositions Duplicate records Contradicting records Wrong references Overlapping data/matching records Name conflicts Structural conflicts Inconsistencies

What are some methods for checking for data validity? Module 8 Data Validation & Security What are some methods for checking for data validity?

Methods for checking for data validity Module 8 Data Validation & Security Methods for checking for data validity Visual/manual Aggregation Reviewers guide Auto data checks Record counts Spell checks Have data provider review

Module 8 Data Validation & Security Time to Exercise!!

ALMIS Database Security Considerations Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Physical security (including backup & restoration plan) Security Concerns: Confidentiality Integrity Availability Network (LAN/WAN) security Network permissions Web security Data Storage Access (Server or PC) Application security Application security RDBMS ODBC User level Production vs. test Production vs. test Suppression flags Data aggregation issues User level User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

ALMIS Database Security Considerations Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Physical security (including backup & restoration plan) Security Concerns: Confidentiality Integrity Availability Network (LAN/WAN) security Network permissions Web security Data Storage Access (Server or PC) Application security Application security RDBMS ODBC User level Production vs. test Production vs. test Suppression flags Data aggregation issues User level User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Physical Security Questions to ask yourself: Module 8 Data Validation & Security Physical Security Questions to ask yourself: Where is your data actually stored? Are you responsible for physical security? If you are...

Module 8 Data Validation & Security Physical Security What physical security measures do you have in place? Do you have the right hardware? Are you using obsolete hardware that is prone to crashing or hacking? Do you have a choice?

Physical Security Do you have a database backup plan? Module 8 Data Validation & Security Physical Security Do you have a database backup plan? Do you have remote backup so if a fire burns the building you don’t lose both your primary and backup data at the same time? Do you have a data restoration plan? Does the backup plan allow for feasible restoration?

Physical Security Restoration Plan Module 8 Data Validation & Security Physical Security Restoration Plan Does your restoration plan allow for the restoration of individual tables or require the entire database? Do you maintain copies of the tables on your hard drive?

Module 8 Data Validation & Security Physical Security Who has access to the physical storage location? Is your database on a PC or server that someone could easily kick, spill coffee on, or accidentally reappropriate? If you are not responsible for physical security, ask these questions of those who are!

ALMIS Database Security Considerations Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Physical security (including backup & restoration plan) Security Concerns: Confidentiality Integrity Availability Network (LAN/WAN) security Network permissions Web security Data Storage Access (Server or PC) Application security Application security RDBMS ODBC User level Production vs. test Production vs. test Suppression flags Data aggregation issues User level User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Data Storage Access What software are you using to store the database? Module 8 Data Validation & Security Data Storage Access What software are you using to store the database? SQL Server, Oracle, FoxPro, Access? Survey says: SQL Server 48% Oracle 28% Other 24%

Module 8 Data Validation & Security Data Storage Access What can be done at the server level to provide for security? Confidentiality, Integrity, Availability DO apply advanced security to the most sensitive data, but DON’T apply advanced security to non-sensitive data

Data Storage Access Security at the server and/or database level… Module 8 Data Validation & Security Data Storage Access Security at the server and/or database level… Permissions What kind of permissions exist? Read only, SA, etc. Who sets those permissions? Who has those permissions?

ALMIS Database Security Module 8 Data Validation & Security ALMIS Database Security Security at the data level… Suppression Suppression flags Suppression can be handled at the database level or the application level

ALMIS Database Data Security Module 8 Data Validation & Security ALMIS Database Data Security Two approaches: 1. ALMIS Database contains NO confidential data (all data available for use without restriction) 2. ALMIS Database contains confidential data (data access controlled by security and/or suppression)

ALMIS Database Security Module 8 Data Validation & Security ALMIS Database Security Tables that have suppression flags... ces indprj industry iomatrix occprj oeswage stindprj stoccprj NOTE: tables stfirms and wage have no suppression flags but may contain confidential data

ALMIS Database Data Security Module 8 Data Validation & Security ALMIS Database Data Security Issues to consider regarding suppression: If your database doesn’t contain suppressed records, detailed data won’t aggregate to totals (validity checks?) Without proper suppression, confidential data can be back-calculated

ALMIS Database Data Security Module 8 Data Validation & Security ALMIS Database Data Security A note on confidential data: If you are checking 202 data (or any other potentially confidential data) to see that it loaded right, make sure your EQUI data files and/or printouts are always secured and shred all photocopies when finished. Check with your local BLS personnel or LMI administrator for specific confidentiality policies.

ALMIS Database Security Considerations Module 8 Data Validation & Security ALMIS Database Security Considerations The contact in my state is:______________ Physical security (including backup & restoration plan) Security Concerns: Confidentiality Integrity Availability Network (LAN/WAN) security Network permissions Web security Data Storage Access (Server or PC) Application security Application security RDBMS ODBC User level Production vs. test Production vs. test Suppression flags Data aggregation issues User level User level Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

User Access Three major types of user access: Module 8 Data Validation & Security User Access Three major types of user access: PC - direct to database (ODBC or RDBMS) Network (LAN/WAN) through application Web (passive or active through application)

Module 8 Data Validation & Security User Access via the Web How do Web users access the data from the database? Passive (static tables automatically updated to web pages) Active (query through application)

User Access via the Web Questions to ask yourself: Module 8 Data Validation & Security User Access via the Web Questions to ask yourself: What kind of web server are you using? What are the security considerations with that choice? What kind of firewall do you have?

Module 8 Data Validation & Security User Access via the Web Does your web interface have security/confidentiality suppression? Do you display suppressed data to certain users? If so, how is access managed? Passwords? IP address?

User Access via a Network (LAN/WAN) Module 8 Data Validation & Security User Access via a Network (LAN/WAN) Questions to ask yourself: What type of network system do you have? Who has rights/access to your data via the network? Through what applications? Are confidential data (suppressed records) available?

User Access via a Network (LAN/WAN) Module 8 Data Validation & Security User Access via a Network (LAN/WAN) How do you control access to confidential data? Through user Ids? Through server access permissions? DO recognize the important distinction between network security and data security.

User Access via direct connection Module 8 Data Validation & Security User Access via direct connection Questions to ask yourself: Is local access machine password protected? Who has access to your machine? Is there a backup plan for your access machine? Is the source data for your database secure? Backup plan? Restorable?

ALMIS Database Data Security Module 8 Data Validation & Security ALMIS Database Data Security Documentation It may prove beneficial to keep detailed records on… How security is done Where security exists Who is responsible for security Who has access/permissions to what Etc...

Module 8 Data Validation & Security Data Security DISCUSSION