New Mexico State University Capstone Data Analysis Services: Microservices Early Implementation (RESTful microservices and Document Database) July 27th, 2016 Andre Guerrero New Mexico State University RESTful microservices - how we can use this approach to deploy distributed and parallel task execution, that is also resilient, fault tolerant Document DB – Not a regular RDB with tables, and primary keys, and joins. It stores whole JSON documents. Different Schema. Advantages of using it instead of RDB The technologies – Here we used Flask and Flask-RESTful, and mongoDB. And python as a primary programming language.

This presentation will focus on: Introduction This presentation will focus on: concepts of RESTful microservices; Document databases; The technologies used to implement these concepts. RESTful microservices - how we can use this approach to deploy distributed and parallel task execution, that is also resilient, fault tolerant Document DB – Not a regular RDB with tables, and primary keys, and joins. It stores whole JSON documents. Different Schema. Advantages of using it instead of RDB The technologies – Here we used Flask and Flask-RESTful, and mongoDB. And python as a primary programming language. 1

The Architecture Here is the whole application architecture. The same that Ramesh previously presented, but in a different layout. We focus on instances 2 and 3, which is where the Microservices and Document DB have been deployed. 2

What is a Microservice? Independent modules; -Services are processes that communicate with each other over the network. -These services use technology agnostic protocols. -Examples with the calculator functions. Netflix started moving to microservices about 6 years ago. Today it features about two billion API edge requests every day which are handled by approximately 500+ microservices. Netflix also run on top of AWS, just like us! Independent modules; Communicate over the network; Agnostic languages / protocols. 3

What is a RESTful API? REST = Representational State Transfer; HTTP methods: GET, POST, PUT, DELETE REST = Representational State Transfer; Architecture for designing networked applications; Simple HTTP(S) to make calls between machines. 4

Our Microservices + REST design Show that services respond as JSON or HTML Show that services could talk to the DB to store medatada 5

Flask Overview Flask is a web framework that provides tools, libraries and technologies to build a web application. Flask is lightweight and makes it easy to build and deploy RESTful microservices. Vendors (Amazon /Rackspace) are creating architectures that support Flask Compare with Jango, RoR 6

DocumentDB It naturally fits our JSON approach to data transferring JSON = JavaScript Object Notation RDL: I added a comment about the advantage of using DocumentDB – i.e. its flexibility. If not accurate feel free to remove. I thought it might Set the stage for your advantages slide up next. A flexible database work working with heterogeneous data objects… 7

Some DocumentDB advantages Schema Free; Easy to deploy and modify; Great for rapid prototyping; Queryable documents; Ease of scale-out. -Schema Freee means no Alter Table. The ALTER TABLE command required to add a single, new field to Craiglist’s MySQL database would take months to execute. -Even if you store a document in RDB, you can’t query it. In DocumentDB you can query by any term inside the document. - MongoDB has auto-sharding, which splits and manage data in multiple server for scale-out (cheaper than scale-up). Two main features: easy to deploy and good performance right out-of-the-box 8

Our DocumentDB architecture And besides slicer, there can be as many microservice tasks as needed. (not show for space). It also allow us to persist, reproduce a whole workflow easily. Each document has a li9nk to it’s parent, so we can reproduce the whole workflow. Track documents, reuse a few tasks. This can save execution time. 9

Uses of Flask and MongoDB Trusted by: Does not mean they use for they whole system!!!!! -Pinterest uses Flask for since late 2011. 12 billion request per day! -Linked-in uses Flask as internaly https://www.quora.com/What-is-the-largest-site-created-using-Flask 10

Conclusions and Future Work Good architectural approach! How we would like it to be: Implement automated deployment for microservices; More microservices for different use cases; UI for workflow creation (visually connect microservices); User login, authentication; Use the DB to discover, share, and re-run workflows; Deploy for users! This architecture/technologies has ben used by many big companies, Netflix, Ebay, EA. IT worked well for us in just less than 3 months. 11

Questions? Brace yourselves Questions are coming! 12

RDBMS vs MongoDB naming

RDBMS vs MongoDB Query