1. Non-traditional Databases
DBMAN 10
Non-traditional Databases
SzaboZs

2. Non-Traditional = schema-less
Many implementations with each working very differently and serving a specific need
Common aim: less complex storage, more flexibility, faster access, server clustering
Usually also: arbitrary type of relations, or even relation-less structures
NoSql = Not Sql = Not Only Sql
SzaboZs

3. Comparison Structure and type of data being kept:
SQL/Relational databases require a structure with defined attributes to hold the data, unlike NoSQL databases which usually allow free-flow operations.
Querying:
Regardless of their licences, relational databases all implement the SQL standard to a certain degree and thus, they can be queried using the Structured Query Language (SQL). NoSQL databases, on the other hand, each implement a unique way to work with the data they manage.
SzaboZs

4. Comparison
Scaling:
Both solutions are easy to scale vertically (i.e. by increasing system resources). However, being more modern (and simpler) applications, NoSQL solutions usually offer much easier means to scale horizontally (i.e. by creating a cluster of multiple machines).
Reliability:
When it comes to data reliability and safe guarantee of performed transactions, SQL databases are still the better bet.
SzaboZs

5. Comparison
Support:
Relational database management systems have decades long history. They are extremely popular and it is very easy to find both free and paid support. If an issue arises, it is therefore much easier to solve than recently-popular NoSQL databases -- especially if said solution is complex in nature (e.g. MongoDB).
Complex data keeping and querying needs:
By nature, relational databases are the go-to solution for complex querying and data keeping needs. They are much more efficient and excel in this domain.
SzaboZs

6. NoSQL advantages – “BIG DATA” – VVVC
Velocity, Variety, Volume, Complexity
High data velocity – lots of data coming in very quickly, possibly from different locations.
Data variety – storage of data that is structured, semi-structured and unstructured.
Data volume – data that involves many terabytes or petabytes in size.
Data complexity – data that is stored and managed in different locations or data centers.
SzaboZs

7. Typical “BIG DATA” use cases
Flexible Data Models
A NoSQL database is able to accept all types of data – structured, semi-structured, and unstructured – much more easily than a relational database which rely on a predefined schema
Faster operations on semi-structured / unstructured data
Analytics and Business Intelligence
Ability to mine the data that is being collected
“real-time data warehousing”: no need for middle-layer: no group by and join queries
SzaboZs

8. Typical “BIG DATA” requirements – CAP
Modern Transactional Capabilities (vs ACID!)
Consistency (all nodes see the same data at the same time – not the same as the “C” in ACID )
Availability (a guarantee that every request receives a response, even if not up-to date)
Partition tolerance (the system continues to operate despite arbitrary partitioning due to network failures)
P is a minimum  in case of a communication error (partition) we usually must choose between A and C
Some people say that CA without P = RDBMS, C+A+P = NoSQL
SzaboZs

9. Typical “BIG DATA” requirements - PACELC
In case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C)
Else (when no partitioning occurs: E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C)
SzaboZs

10. NoSQL types Key-value Graph database Wide column Document storage
(Search engines)
LOTS of various implementation, extremely rapidly changing features and applications ...
Especially with document databases and the chaotic world of JS in the past 2-3 years: “How it feels to learn JavaScript in 2016”
SzaboZs

11. NoSQL types Key-Value store
least complex, like Dictionary<string, string> in C#
Memcached vs MemcacheDB (cache vs storage), REDIS
Oracle NoSQL Database (Eventually-Consistent)
When to use
Caching
Queueing:
Distributing information / tasks
Keeping live information
SzaboZs

12. NoSQL types Graph database Data is stored in graphs
OrientDB, Neo4J, etc.
Google: Linked data, Linked open data
RDF = Resource Description Framework
When to use
Handling complex relational information
Modelling and handling classifications
SzaboZs

13. NoSQL types Column store / wide-column stores
sounds like the inverse of a standard database
very high performance and a highly scalable architecture
Apache Cassandra  fastest
When to use
Keeping unstructured, non-volatile information:
Scaling
SzaboZs

14. Document Storage formats
HTML = document descriptor, bad for storage, not strict
XML, YAML = data descriptor, strict format
JSON = object descriptor, strict format
SzaboZs

15. Storage formats
XML  many tools, fast and memory efficient, hard to read, hard to parse, XPATH is official
YAML  few tools, 2x memory, easy to read, easy to parse
JSON  more and more tools, 2x memory, hard to read, easy to parse, JSONPATH is unofficial
SzaboZs

16. NoSQL types Document database
expands on the basic idea of key-value stores
“documents” contain data (using a storage format!) and each document is assigned a unique key
Apache CouchDB, Lotus Notes
JSON: MongoDB
XML and JSON field formats exist in most modern relational databases
When to use
Nested information
JSON: JavaScript/programming language friendly (MEAN = MongoDB, Express.js, Angular, Node.js)
SzaboZs

17. Document databases: XML databases
“XML Enabled” vs “native XML” databases
The “big four” supports the XML/JSON type in CLOB fields
Typically an XML enabled database is best suited where the majority of data are non-XML
For datasets where the majority of data are XML/JSON, a native database is better suited.
Native XML database: BDB – not really used any more...
Defines a (logical) model for an XML document
Has an XML document as its fundamental unit of (logical) storage
not required to have any particular underlying physical storage model
SzaboZs

18. Search engines RDBMS systems have been weak in varchar indexing
Especially in fulltext indexing (list of words)
In the past 5 years, things are getting better, but still…
Elasticsearch, Splunk, Solr
Elasticsearch in C#: NEST
LINQ-similar methods & syntax
SzaboZs

19. Document databases: JSON / MongoDB
JSON-like documents (called BSON: Binary JSON) “Libbson expects that you are always working with UTF-8 encoded text”
Ad hoc queries that can include user-defined JavaScript functions
Indexing is similar to RDBMSes
Replication/Load balancing: high availability with replica sets; one replica set = two or more copies of the data
VERY efficient storage and operations for HIGHLY flexible data
SzaboZs

20. document or BSON document
SQL
MongoDB
database
table
collection
row
document or BSON document
column
field
index
table joins
embedded documents and linking
primary key
(automatically set to the _id field)
aggregation (e.g. group by)
aggregation pipeline
SzaboZs

21. MongoDB vs SQL SQL: CREATE TABLE
MongoDB: Imlicitly created when inserting the first record db.users.insert( { user_id: "abc123", age: 55, status: "A" } ) (later, we can insert users with more attributes) or: db.createCollection("users")
SzaboZs

22. MongoDB vs SQL SQL: ALTER TABLE ... ADD ...
MongoDB: db.users.update( { }, { $set: { join_date: new Date() } }, { multi: true } )
SzaboZs

23. MongoDB vs SQL SQL: ALTER TABLE ... DROP COLUMN ...
MongoDB: db.users.update( { }, { $unset: { join_date: "" } }, { multi: true } )
SzaboZs

24. MongoDB vs SQL SQL: INSERT INTO / UPDATE / DELETE
MongoDB: db.users.insert( { user_id: "abc123", age: 55, status: "A" } ) db.users.remove( { status: "D" } ) db.users.update( { age: { $gt: 25 } }, { $set: { status: "C" } }, { multi: true } ) db.users.update( { status: "A" } , { $inc: { age: 3 } }, { multi: true } )
SzaboZs

25. MongoDB vs SQL SQL: SELECT ... FROM ... WHERE
MongoDB: db.users.find() -- where + fields db.users.find().limit(5).skip(10) db.users.find( { }, { user_id: 1, status: 1 } )
db.users.find( { status: "A" }, { user_id: 1, status: 1, _id: 0 } )
db.users.find( { age: { $gt: 25, $lte: 50 } } )
db.users.find( { user_id: /^bc/ } )
SzaboZs

26. SQL MongoDB WHERE $match GROUP BY $group HAVING SELECT $project
ORDER BY
$sort
LIMIT
$limit
SUM()
$sum
COUNT()
join
No direct corresponding operator; use $unwind for somewhat similar functionality
SzaboZs

27. MongoDB vs SQL SQL: SELECT COUNT(*) ... MongoDB:
db.users.count() or db.users.find().count()
db.orders.aggregate( [ { $group: { _id: null, total: { $sum: "$price" } } } ] )
db.orders.aggregate( [ { $group: { _id: "$cust_id", total: { $sum: "$price" } } }, { $sort: { total: 1 } } ] )
SzaboZs

28. MongoDB
Usually works well if bi-directional speed / reports are not that important
Practice showed, that a RamDisk + LOAD DATA INFILE + MyISAM can still be faster than the thousands of insert into commands on a MongoDB (everything depends on the design: a badly designed db with a fast db engine is always SLOWER AND WORSE than a well-designed db with a slow db engine)
When using NoSql, “jury rigging” is harder
Must be very careful about the configuration of the individual components
SzaboZs

29. NoSQL in C# Redis MongoDB: ServiceStack.Redis.Complete
RedisClient.Set(key, value);
RedisClient.Get<type>(key)
MongoDB:
MongoDB.Bson + MongoDB.Driver.Core + MongoDB.Driver
Entity objects with [BsonElement] (code-first, with _id)
MongoClient.GetDatabase(dbName)
IMongoDatabase.CreateCollection(collName)
IMongoDatabase.GetCollection<BsonDocument>(collName)
IMongoCollection<Type>.InsertOne() / InsertMany() + …Async()
- LINQ SUPPORT
SzaboZs

30. SzaboZs

31. SzaboZs