Seminar: Deep Dive into Oracle NoSQL Technologies and Solutions Presenter: Zohar Elkayam, CTO, Brillix

2 תוכנית יומית לשבוע אורקל 10:30-09:00 – הרצאה 10:45-10:30 – הפסקת קפה 12:30-10:45 – הרצאה 13:30-12:30 – ארוחת צהריים 14:50-13:30 – הרצאה 15:00-14:50 – הפסקה 16:30-15:00 – הרצאה

3 Agenda Introduction to Big Data and Oracle NoSQL Database – Oracle NoSQL Database Features – Oracle NoSQL Database Architecture Oracle NoSQL Database Planning and Installation Workflow Create and Configure KV Store Manage Memory Operations Optimizing KVStore Performance and Memory Sizing Setting Replication Node Policy Working with Storage Node Parameters Perform Store Backup and Recovery Diagnosing and Troubleshooting the KVStore

Big Data Technology

5 What is Big Data?

6 Volume

7 Velocity

8 Variety

9 And let’s remember our goal: Value!

How do we “do” Big Data? 10

Deep Analytics Agile Development Massive Scalability Real Time Results High Throughput In-Place Preparation All Data Sources/Structures Low, predictable Latency High Transaction Volume Flexible Data Structures Big Data: Infrastructure Requirements AcquireOrganizeAnalyze

Divided Solution Spectrum MapReduce Solutions DBMS (DW) DBMS (OLTP) Advanced Analytics Distributed File Systems Transaction (Key-Value) Stores ETL NoSQL Flexible Specialized Developer Centric SQL Trusted Secure Administered Dynamic Schema Data Variety Schema AcquireAnalyze Organize

Dynamic Schema Data Variety Schema Oracle Integrated Software Solution Stack AcquireAnalyze Organize Oracle Database (DW) Oracle Database (DW) Oracle Database (OLTP) Oracle Database (OLTP) In-DB Analytics “R” Mining Text Graph Spatial In-DB Analytics “R” Mining Text Graph Spatial Oracle BI EE Oracle BI EE Oracle NoSQL DB HDFS Hadoop Oracle Data Integrator Oracle Loader for Hadoop

Short Intro to NoSQL (and basic database theories…) 14

15 The Challenge RDBMS is too generic and doesn’t cut it any more We want scalable, durable, high volume, non-structured, distributed data storage that will fit our specific need.

16 The solution: NoSQL Databases Let’s take some parts of the standatd RDBMS out and replace it with things we actually need. NoSQL databases are designed for a specific uses NoSQL database has been around for ages under different names/solutions

17 Kinds of NoSQL Key-Value stores – Simple K/V lookups (DHT) Column stores – Each key is associated with many attributes (columns) – NoSQL column stores are actually hybrid row/column stores Document stores – Store semi-structured documents (JSON) – Map/Reduce based materialization, sorting, aggregation, etc. Graph databases – Not exactly NoSQL: can’t satisfy the requirements for High Availability and Scalability/Elasticity very well

18 What Is NoSQL Database? What does NoSQL stands for? Is it No SQL or “Not Only” SQL? What does ACID transaction mean? What is the CAP theorem?

19 ACID Transactions RDBMS are built with ACID transactions in mind: Atomicity: All or nothing Consistency: Any transaction will take the DB from one consistent state to another with no broken constraints Isolation: Other operations cannot access data that has been modified during a transaction that has not been completed yet Durability: Ability to recover the commited transaction updates against any kind of system failure (transaction log).

20 ACID Transactions (cont.) ACID is usually implemented by a locking mechanism/manager. Distributed systems central locking would be a bottleneck. Most NoSQL does not use the ACID transactions and replaces it with something else…

21 The CAP Theorem The CAP theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:

22 The CAP Theorem properties Consistency – does all nodes see the same data at the same time? Availability – does the system guarantee that every request receives a response about whether it was successful or failed? Partition tolerance – does the system continues to operate despite arbitrary message loss or failure of part of the system?

23 The CAP Theorem (cont.) According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all three NoSQL are often designed to “give up” one of the CAP properties in order to get the ability to be distributed and therefor very scalable

24 C A P Consistency AvailabilityPartition-resilience CA: available, and consistent, unless there is a partition. AP: a reachable replica provides service even in a partition, but may be inconsistent if there is a failure. CP: always consistent, even in a partition, but a reachable replica may deny service without agreement of the others (e.g., quorum). Single site DB Cluster DB (RAC) Distributed DB DNS

What is Oracle NoSQL? 25

26 Oracle NoSQL Oracle NoSQL Database is: A key-value database Written in Java Accessible using Java APIs Built on Oracle Berkeley DB Java Edition The Oracle solution to acquiring big data

27 Benefits of Using Oracle NoSQL Database Oracle NoSQL Database offers the following benefits: It is easy to install and configure. It is highly reliable. It is a general-purpose database system. It has scalable throughput and predictable latency. It has configurable consistency and durability. It has a web console for administration.

28 Supported Data Types

Common uses for the Key-Value Store Large dynamic schema based data repositories Data capture Web applications Online retail Sensor/statistics/network capture/Mobile Devices Data services Scalable authentication Real-time communication (MMS, SMS, routing) Personalization / Localization Social Networks

Oracle NoSQL DB A distributed, scalable key-value database Simple Data Model Key-value pair with major+sub-key paradigm Read/insert/update/delete operations Scalability Dynamic data partitioning and distribution Optimized data access via intelligent driver High availability One or more replicas Disaster recovery through location of replicas Resilient to partition master failures No single point of failure Transparent load balancing Reads from master or replicas Driver is network topology & latency aware Storage Nodes Data Center A Storage Nodes Data Center B NoSQLDB Driver Application NoSQLDB Driver Application

Operation result New Partition Map RepNodeStorageTable information Operation result New Partition Map RepNodeStorageTable information Hash Major Key to determine Partition id Use Partition Map to map Partition id to a Rep Group Use State Table to determine eligible Storage Node(s) within Rep Group Use Load Balancer to select best eligible Rep Node Contact Rep Node directly Client Operation + Key[M,m] + Value + Transaction Policy

Oracle NoSQL DB Differentiation Commercial Grade Software and Support General-purpose Reliable – Based on proven Berkeley DB JE HA Easy to install and configure Scalable throughput, bounded latency Simple Programming and Operational Model Simple Major + Sub key and Value data structure ACID transactions Configurable consistency & durability Easy Management Web-based console, API accessible Manages and Monitors: Topology; Load; Performance; Events; Alerts Completes Oracle large scale data storage offerings

33 More stuff EE or CE? What to choose?

Database components 34

Oracle KVLite 35

Schema Considerations 36

Accessing and manipulating the data using JAVA API 37

Understanding consistency, transactions and versioning 38

Understanding Durability 39

Using the Admin console to configure the KVStore 40

Summary More info will be available on my blog: ZoharElkayam.wordpress.com ZoharElkayam.wordpress.com 41

Questions and Answers 42

Thank You! Zohar Elkayam