NIST BIG DATA WG Reference Architecture Subgroup Intermediate Report Co-chairs: Orit Levin (Microsoft) James Ketner (AT&T) Don Krapohl (Augmented Intelligence) August 21, 2013

Starting Point From the Previous Joint Call Transformation includes Processing functions Analytic functions Visualization functions Data Infrastructure includes Data stores In-memory DBs Analytic DBs Sources Transformation Usage Data Infrastructure Security Management Cloud Computing Network 8/15/2013NIST Big Data WG / Ref Arch Subgroup2

The Latest RA Diagram 8/15/2013NIST Big Data WG / Ref Arch Subgroup3

RA Current View The current RA view is technological, meaning that It doesn’t show topology or deployment models It doesn’t show business stakeholders or business relationships By using different sets of taxonomies, the same baseline diagram can be modified to show topologies or business relationships These different views would need to be shown separately (e.g., technologies vs. deployment models vs. business stakeholders) A proposal: Discuss these aspects in the context of the RoadMap scope and direction and sync on the direction and responsibilities 8/15/2013NIST Big Data WG / Ref Arch Subgroup4

Deliverables Delivery #1: White Paper describing different RA approaches The proposed Outline is in M0150 Call for review Call for volunteers to produce content for different sections Delivery #2: Common RA Draft The proposed Outline is in M123 Call for additional volunteers to produce content for different sections A number of participants volunteered already, more are welcome 8/15/2013NIST Big Data WG / Ref Arch Subgroup5

Deliverable #2: Dependencies on Other Subgroups 2 Big Data Reference Architecture: Use Cases Requirements This section will be prepared by the NIST BDWG Requirements SG and will contain high level requirements relevant to the design of the Reference Architecture. 6 Big Data Reference Architecture: Security and Privacy This section will be prepared by the NIST BDWG Security and Privacy SG and will contain high level security and privacy considerations relevant to the design of the Reference Architecture. 7 Big Data Taxonomy This section will be prepared by the NIST BDWG Def&Tax SG and will contain high level taxonomy relevant to the design of the Reference Architecture. 8/15/2013NIST Big Data WG / Ref Arch Subgroup6

Backup Slides 8/15/2013NIST Big Data WG / Ref Arch Subgroup7

Reference Architecture Objectives (No Change) Addresses a broad range of stakeholders (e.g., data owners, industries, academia, policy makers) Wide scope: Encompasses the whole data life cycle or in the ecosystem Can be applied to different use cases (including various verticals) Represents different system architectures (e.g., an enterprise data warehouse, distributed cloud-based system using multiple service providers) Focus Potentially with initial focus on the Big Data analytics and tools Assists in identifying security and privacy issues Agnostic to any specific technologies 8/15/2013NIST Big Data WG / Ref Arch Subgroup8

Input for Discussion on the BD RA Next Level of Details The next slides are based on various input documents to NIST BD subgroups They are included here to provide insight on the evolution of the NIST BD RA work 8/15/2013NIST Big Data WG / Ref Arch Subgroup9

Proposal I: Highlighting the Different DB Approaches Security Management Cloud Computing Network Sources Transformation Usage Data Infrastructure Collection Export Curation Pre-analytics Visualization Data Manager Storage (disk, memory, etc.)File SystemSpecialized Abstractions Data Mining Analytics Buffer Manager Map Reduce Relational DB NoSQL DB Aggregation Integration Transfer Search Statistics RT Analytics VOLUME VARIETY VELOCITY Streaming Interactive An. Batch Analytics 8/15/2013NIST Big Data WG / Ref Arch Subgroup10

Proposal II: Based on Submitted Requirements 8/15/2013NIST Big Data WG / Ref Arch Subgroup11