Software Engineering for Business Information Systems (sebis) Department of Informatics Technische Universität München, Germany wwwmatthes.in.tum.de Factors influencing the database selection for B2C web applications. Master Thesis Kick-Off Presentation Al-Saeedi, Bilal |

Agenda Introduction Motivation ? Research questions ? 1 © sebis Matthes Slides sebis Road Plan Progress ? Future plan ? 3 Research Approach Research plan ? Present what has been done ? 2

Introduction – Motivation © sebis Matthes Slides sebis NOSQL DatabasesRelational DBMS

Introduction – Research Questions © sebis Matthes Slides sebis What are the influencing factors for the selection of a database for a B2C application ? What are the challenges in terms of query expressiveness and complexity of the database ? What are the benefits and shortcomings of the database in terms of B2C application scenarios ?

Research Approach © sebis Matthes Slides sebis DB selectionConstructs identificationUse cases identificationLiterature reviewUse cases implementationResults documentation

Databases Selection © sebis Matthes Slides sebis Community Strength MaturityPopularity Availability Scalability Open Source Support Reliability

Databases Selection © sebis Matthes Slides sebis Database Categories Key-Value Document Based Column Based Graph Full Text Search Engines RDBMS MySQL Postgres Oracle …..... Redis Riak Memcached …..... MongoDB Couchbase Cloudant …..... Cassandra HBase Accumulo …..... Neo4j Titan OrientDB …..... Elasticsearch Apache Solr BaseX ….....

Constructs © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Any Database Specific Features Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

B2C Use Cases © sebis Matthes Slides sebis Redis Session Management. Cart Management. Cache Service. Login, Cart, and Products Analytics Service. MongoDB Users Management. Shipping Management Product Review & Wish list Management History Data. Product & Category Management. Cassandra Analytics Service Logging Service Neo4j Recommendations System. ElasticSearch Product Search. MySQL Finance (orders & payments) Inventory Management.

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Redis – Basic Concepts © sebis Matthes Slides sebis

Redis – Transaction Support © sebis Matthes Slides sebis Watch/UnwatchMulti/Exec Discard

Redis – Relational Data Support © sebis Matthes Slides sebis (Hash) Customer (Hash) Order (Hash) Shipping (Hash) Payment 1* * * 1* Set (orderIDs) Orders:customerID Set (orderIDs) Orders:paymentID Set (orderIDs) Orders:shippingID Set (shippingIDs) Shippings:orderID

Redis – Relational Data Support © sebis Matthes Slides sebis Submit an order Get all customer Shippings

Redis – Indexing © sebis Matthes Slides sebis Primary IndexKey Hash { “SKU”: “sku”, “Name”:”ProductName”, “Color”: “ProductColor”, “Price”: “ProductPrice” } Products:ProductID  Primary Index Set {ProductIDs} Product.Color.Index: ProductColor Sorted Set {ProductIDs, Price} Product.Price.Index Secondary Index Composite Index Sorted Set {CategoryID:Price:ProductID, 0} Product.Category.Price.Index Retrieval

Redis – Filtering, Grouping and Sorting © sebis Matthes Slides sebis Group Data Set {ProductIDs} Product. Manufacture.Index: CompanyName Set {ProductIDs} Product.Color.Index: ProductColor Filter Data Sorting Data SORT command  sorts lists, sets, and sorted sets

Redis – Use Cases Implementations © sebis Matthes Slides sebis Cart Management Service Session Management Service Cache Service Analytics Service Other Redis Examples: Inspect Data Structures Job Queue. Handling Data Relation Pipeline Support Transaction Support

Road Plan © sebis Matthes Slides sebis Today CompleteCompleteOngoingOngoing Not Started EndStart RevisingRevising

Technische Universität München Department of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße Garching bei München wwwmatthes.in.tum.de Bilal Al-Saeedi Thank you for your attention! MSc. Informatics Student

Backup slides © sebis Matthes Slides sebis

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Databases Selection – Key-Value © sebis Matthes Slides sebis Built-in persistence support. Vey fast in-memory database Variety of storage data types. Open source

Databases Selection – Document-Based © sebis Matthes Slides sebis JSON-like storage format named BSON Great features such as ad hoc queries, Indexing, sharding, etc. Easy to use and has great community Open source

Databases Selection – Column-Based © sebis Matthes Slides sebis Open source SQL-Like Query language called CQL Decentralized, no single point of failure Great community

Databases Selection - Graph © sebis Matthes Slides sebis Supports a powerful and easy to use graph query language called Cypher. Support complex data modeling. Open source Great community

Databases Selection – Full Search Engine © sebis Matthes Slides sebis Scalable Map/Reduce model Memory efficient and fast. Open source Great community

Databases Selection - RDBMS © sebis Matthes Slides sebis Relational Database System Client/Server Architecture SQL compatibility Open source RDBMS Great community

Introduction – Research Questions © sebis Matthes Slides sebis System product quality model—ISO/IEC9126.

Databases Selection © sebis Matthes Slides sebis Quadrant for Operational DBMS

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query Language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Constructs – Redis © sebis Matthes Slides sebis Introduction Installability Basic Concepts Possible Use cases Basic Features Query language Transaction Support Pipeline Support Data Modeling Data Layout Relational Data Support Referential Integrity NormalizationData Evolution Nested Data Support Searching Data Full Text Search Support Regular Expressions Support Query OptionsIndexingQueries Filtering and Grouping data Sorting Special Features Pub/Sub Support Expire Configuring as a cache Lua Scripting Administration and Maintenance ConfigurationScalability Persistence, and ACID Support Handling failure Backup and Upgrade SecurityAvailabilityData migration

Redis – Relational Data Support © sebis Matthes Slides sebis Difference Union Join

Redis – Configuration & Persistence © sebis Matthes Slides sebis Can be started without configuration file. Redis configuration is in redis.confi, format confugirationKeyWord argument1 argument2... argumentN Configured also using command line./redis-server --port slaveof Configuration on the fly without restarts CONFIG SET SAVE " " Configuration Persistence Snapshotting or RDB Persistence Periodically and with condition save / BGSAVE Append-Only File or AOF Persistence appendonly yes / BGREWRITEAOF fsync every, fsync every second, Never fsync  fsync every for full durability

Redis – Expire © sebis Matthes Slides sebis Expire SetEx

Redis – Configuration © sebis Matthes Slides sebis Can be started without configuration file. Redis configuration is in redis.confi, format confugirationKeyWord argument1 argument2... argumentN Configured also using command line./redis-server --port slaveof Configuration on the fly without restarts CONFIG SET SAVE " "

Redis – Indexing © sebis Matthes Slides sebis Primary IndexKey Hash { “SKU”: “sku”, “Name”:”ProductName”, “Color”: “ProductColor”, “Price”: “ProductPrice” } Products:ProductID  Primary Index Set {ProductIDs} Product.Color.Index: ProductColor Sorted Set {ProductIDs, Price} Product.Price.Index Secondary Index Composite Index Sorted Set {CategoryID:Price:ProductID, 0} Product.Category.Price.Index Retrieval