Data Management Managing Big Data Briefing 10/2012 Will Graves US-VISIT Chief Biometric engineer Chair of Biometric Domain

 Established in 2003, US ‑ VISIT was one of the initial programs at the Department of Homeland Security (DHS).  In 2007, US-VISIT became part of the DHS National Protection and Programs Directorate (NPPD). US-VISIT AT A GLANCE 2  Vision – A more secure Nation through advanced biometric and biographic identification, information sharing, and analysis.

GUIDING PRINCIPLES 3 FACILITATE legitimate travel and trade ENHANCE the security of our citizens and visitors ENSURE the integrity of our borders PROTECT the privacy of our visitors

INFORMATION REPOSITORIES 4

A DAY IN THE LIFE – BIOMETRICS* 5 *Data as of March 2011.

A D AY IN THE L IFE – B IOGRAPHIC O VERSTAY A CTIONS* 6 *Data as of March 2011.

Big Data  Big Data includes data sets with sizes beyond the ability of commonly-used software tools to capture, curate, manage, and process the data within a tolerable elapsed time  Big Data Datasets so large that they’re difficult to work with using conventional toolsets (terabytes, exabytes, ….)  Big Data as relates to Biometrics is the capture, collection, mapping, analysis, housing, and retrieval of biometrics related data containing properties such as privacy, availability, reliability, maintainability, security, usability, performance, prediction, prevention, and detection 7

Big Data - Factsheet  1996: A Teradata database becomes the world's largest database at 11 terabytes  1999: Teradata customer has world's largest database with 130 terabytes  2000: Oracle Claimed the largest warehouse at ~ 140 TB hosted on two massive Oracle instances … today 140TB is almost passe Cost of Big Data Storage Dropping In 2000 a GB avg $16.06, a 1 TB data warehouse was rare Today a GB avg $0.0621, a terabyte can be had for < $100 8

Big Data Challenges  Storing unprecedented volumes of data  Rate of biometric data in production is increasing  Size of biometric data is increasing (i.e. DNA)  Extracting and visualizing the data in ways that are helpful to users  Metadata and Semantics for describing content  Finding what is needed, in the context of the mission  semantically-enabled search engines that can use the context  Eliminating what is not needed  Either after the data usage or before deciding to store it  Governing data collections within the COI  Effective governance of data resources  Quality control structures 9

Big Data Taking Control  Employee a wide variety of data techniques and technologies in the areas of  Data aggregation  Query Languages  Data manipulation  ETL  Data analysis  Query Plans, Cost Estimation  Data visualization  Formatting  Data Organization  Data Modeling, Storing Data  Data Retrieval  Query Optimization  Data Integrity 10

Steps In Preparing To Handle Big Data  Requirements Analysis –User needs, policy mapping, what must database do  Conceptual Design –High level description (often done w/ER model)  Logical Design –Translate ER into DBMS data model  Schema Refinement –Consistency, normalization  Physical Design –Indexes, disk layout, conversions  Security Design –Who accesses what, and how 11

Harmonizing Big Data 12

Other Use Cases  Availability and Visualization  Maintainability  Reliability  Performance and Usability  Prediction and Prevention  Detection and Security  System and biometric algorithm improvement 13

C C ONTACT I NFORMATION William Graves Chief Biometric Engineer US-VISIT Program Information Sharing and Technical Assistance Branch Tel: