Modern data architecture at scale in the cloud : Best practices of Serverless, lambda and microservices architecture Prakriteswar Santikary, PhD Vice President.

Slides:



Advertisements
Similar presentations
Cisco and NetApp Confidential. Distributed under non-disclosure only. Name Date FlexPod Entry-level Solution FlexPod Value, Sized Right for Smaller Workloads.
Advertisements

SPRING 2011 CLOUD COMPUTING Cloud Computing San José State University Computer Architecture (CS 147) Professor Sin-Min Lee Presentation by Vladimir Serdyukov.
© 2006, Cognizant Technology Solutions. All Rights Reserved. The information contained herein is subject to change without notice. Automation – How to.
“ Does Cloud Computing Offer a Viable Option for the Control of Statistical Data: How Safe Are Clouds” Federal Committee for Statistical Methodology (FCSM)
CLOUD COMPUTING cloud computing and its importance to large scale firms such as Apple and Microsoft.
Joe Caserta President Elliott Cordo Chief Architect September 30, 2015, Javits Center, New York City Building a Data Lake for Digital Music Dominance.
1 TCS Confidential. 2 Objective : In this session we will be able to learn:  What is Cloud Computing?  Characteristics  Cloud Flavors  Cloud Deployment.
Breaking Up Is Hard To Do From Monolith to Microservices.
Designing Cisco Data Center Unified Fabric
Digital Solutions Extension Framework Mark Myers Director, Global Solutions Go-To-Market July 12, 2016 Solution Partner Forum.
KRISHNACHANDER KALIYAPERUMAL PROJECT MANAGER
Discover your career at Accenture
SAM Baseline Review Engagement
Self-Contained Systems
AuraPortal Cloud Helps Empower Organizations to Organize and Control Their Business Processes via Applications on the Microsoft Azure Cloud Platform MICROSOFT.
Going Serverless with AWS Lambda
DevOps Cloud Native Microservices
Avenues International Inc.
Organizations Are Embracing New Opportunities
Data Platform and Analytics Foundational Training
EI Architecture Overview/Current Assessment/Technical Architecture
By: Raza Usmani SaaS, PaaS & TaaS By: Raza Usmani
Continuous Delivery- Complete Guide
What is Wrong with Models?
CIM Modeling for E&U - (Short Version)
Microsoft Operations Management Suite Insight and Analytics
A UNIFIED ECOSYSTEM FOR MARKET DATA VISUALIZATION
Cloud Computing: Delivering Your Right Mix
Presented by Munezero Immaculee Joselyne PhD in Software Engineering
The Azure Cloud Platform Delivers Data-Driven Digital Transformation for Credit Union Industry Partner Logo “When Helios and Matheson Analytics embarked.
Discover the New ADM and App Testing
What is all the fuss over Containers?
Cloud Computing.
AWS. Introduction AWS launched in 2006 from the internal infrastructure that Amazon.com built to handle its online retail operations. AWS was one of the.
Open Technology for an Open Lifecycle Sean Kennedy, OSLC Community Development Leader @seanpk8 A look at the challenges facing today’s organizations.
Built on the Powerful Microsoft Azure Platform, Lievestro Delivers Care Information, Capacity Management Solutions to Hospitals, Medical Field MICROSOFT.
Operationalize your data lake Accelerate business insight
Why DevOps Success Depends on the Right Infrastructure.
Welcome! Power BI User Group (PUG)
Hybrid Cloud Foundation
Future Data Architectures Big Data Workshop – April 2018
Case Study Modernizing an Operational Data Architecture
Winter 2016 (c) Ian Davis.
Accelerate application delivery with a Cloud-native mindset
Digital Transformation Asia 2018 – CALL FOR SPEAKERS
Extending Your Integration Strategy
Welcome! Power BI User Group (PUG)
Crypteron is a Developer-Friendly Data Breach Solution that Allows Organizations to Secure Applications on Microsoft Azure in Just Minutes MICROSOFT AZURE.
Dell Data Protection | Rapid Recovery: Simple, Quick, Configurable, and Affordable Cloud-Based Backup, Retention, and Archiving Powered by Microsoft Azure.
3 Cloud Computing.
XtremeData on the Microsoft Azure Cloud Platform:
ABHISHEK SHARMA ARVIND SRINIVASA BABU HEMANT PRASAD 08-OCT-2018
DevOps in Localization Continuous Delivery
TEMPLATE NOTES Our datasheet and mini-case study templates are formatted specifically for consistency of branding at Microsoft. Please do not alter font.
JOINED AT THE HIP: DEVSECOPS AND CLOUD-BASED ASSETS
Cloud Consulting Services and Solutions
Introduction to Systems Analysis and Design Stefano Moshi Memorial University College System Analysis & Design BIT
Committed to delivering winning solutions
Last.Backend is a Continuous Delivery Platform for Developers and Dev Teams, Allowing Them to Manage and Deploy Applications Easier and Faster MICROSOFT.
Why Innovate with Lagom & SAP?
Migrate2Micro-services
2/25/2019.
E-BUSINESS E-Business is the powerful business environment that is
Node.js Test Automation using Oracle Developer Cloud- Simplified
Agenda Start with Why What Are Best Practice Frameworks, and Why Do We Need Them? Best Practices Defined Lean, Agile, DevOps and ITSM/ITIL 4 The Increasing.
Sachiko A. Kuwabara, PhD, MA
DBOS DecisionBrain Optimization Server
Prakriteswar Santikary, PhD
ONAP Architecture Principle Review
Michael Stephenson Microsoft MVP - Azure
Presentation transcript:

Modern data architecture at scale in the cloud : Best practices of Serverless, lambda and microservices architecture Prakriteswar Santikary, PhD Vice President and Global Chief Data Officer

ABOUT ERT Founded in 1977; privately held Supporting Pharmacos, Biotechs & CROs around the world Operations in 12 countries 2500+ employees MINIMIZING RISK & UNCERTAINTY, SO YOU CAN MOVE AHEAD QUICKLY WITH CONFIDENCE │ © Copyright ERT 2018

Agenda 01 Definition Characteristics Advantages Disadvantages Monolithic Architecture Microservices Architecture Serverless Architecture Definition Characteristics Advantages Disadvantages Definition Characteristics Advantages Disadvantages Definition Characteristics Advantages Disadvantages Here are a few key learnings we hope to provide with this presentation that may inspire these organizations to embrace this change as an opportunity to innovate. Paradigm Shift: Companies embarking on new innovation initiatives to determine feasibility of mHealth solutions for clinical research Audience Questions: How many of you currently employ mHealth technologies in your clinical trials? How may of you currently utilize Actigraphy and/or Continous Monitoring devices in your clinical trials? How many of you work in organizations that are now tasked with POCs or pilots using mHealth devices? Data Optimization: Enhancing clinical research with mHealth data requires new techniques against non-conventional sources. mHealth data alone does not enhance clinical research. Practical Applications: Applied use of mHealth in select therapeutic areas in combination with other clinical source data and/or ways where mHealth data may be combined for more visual insights | Copyright ERT 2018

Monolithic Architecture – a refresher TIPS ; Fresh business idea, needs quicker turn around, small engineering team – start with monolith – build simple and lightweight application

Monolithic Architecture – Weaknesses and challenges

Software Complexity has been growing steadily 01 World of Big Data 02 DevOps and Automation Small incremental release Continuous deployment Continuous delivery Quick time to market Competitive pressure Multiple data sources Multiple channels Multiple data types Scale requirements Real time insight Big Data (unstructured) Forces behind Microservices Architecture 04 Cloud/Serverless computing 03 Emerging Technology New Programming language New Programming framework NoSQL databases API-first design Cloud-first design Focus on code, not on infrastructure – less IT Serverless deployment Pay per use Cost reduction │ © Copyright ERT 2018

Monolith to Microservices Transition – loosely coupled, independent services communicating via lightweight protocol TIPS : Microservices architecture is not a silver bullet for a bad software design. Design discipline must be exercised as to when to use monolith Vs Microservices.

Monolith is hard to scale; Microservices enable y-axis scaling by breaking the application into smaller independent services. Each service is indepedently scaled based on need, enabling horizonal scaling.

Microservices Logical Architecture

Let’s take a n-tier e-commerce application to explain monolith and microservices

Break it down into microservices │ © Copyright ERT 2018

Microservices architecture – a process view │ © Copyright ERT 2018

Microservices Architecture- Advantages │ © Copyright ERT 2018

Microservices Architecture- DisAdvantages │ © Copyright ERT 2018

Microservices architecture Design patterns 01 Decomposition Pattern 02 Deployment Pattern Service per host/VM Multiple services per host/VM Serverless Deployment On-premise Hybrid (with Cloud) By Business Capability By business domain Single responsibility principle Independently deployable Small service/small team You build it, you own it Domain-driven design Microservices Best Practice 04 Monitoring/Tracing Pattern 03 Data Management Pattern Polyglot persistence Database per service Data consistency Event sourcing Eventual consistency CAP theorem Log Aggregation Audit logging Distributed Tracing Exception Tracking Health-check API │ © Copyright ERT 2018

Microservices architecture – best practices │ © Copyright ERT 2018

Microservices Architecture – a summary view Characteristics Benefits Challenges Decentralized Independent Do one thing well Polyglot API-first Design You build it; You own it Agility Innovation Quality Scalability Availability Distributed systems Monolith->Microservices transition not easy Organizational issues (DevOps) Skillsets TIPS : Microservices is not a silver bullet. You still need to follow sound software design principles. Here are a few key learnings we hope to provide with this presentation that may inspire these organizations to embrace this change as an opportunity to innovate. Paradigm Shift: Companies embarking on new innovation initiatives to determine feasibility of mHealth solutions for clinical research Audience Questions: How many of you currently employ mHealth technologies in your clinical trials? How may of you currently utilize Actigraphy and/or Continous Monitoring devices in your clinical trials? How many of you work in organizations that are now tasked with POCs or pilots using mHealth devices? Data Optimization: Enhancing clinical research with mHealth data requires new techniques against non-conventional sources. mHealth data alone does not enhance clinical research. Practical Applications: Applied use of mHealth in select therapeutic areas in combination with other clinical source data and/or ways where mHealth data may be combined for more visual insights | Copyright ERT 2018

Serverless Architecture Focus on your application, not on infrastructure Characteristics Benefits Challenges No server to maintain No server to provision Focus on code Ephemeral; Stateless Event-driven Horizontal scaling (auto) Pay per use; no idle time Faster innovation Lower operational cost Reduce operational management Speed to market Fosters innovation Skillsets Vendor lock-in concerns Lack of operational tools Architectural complexity Restriction on execution time Restriction on memory TIPS : Build scalable platform and applications using a combination of serverless and microservices in the cloud Here are a few key learnings we hope to provide with this presentation that may inspire these organizations to embrace this change as an opportunity to innovate. Paradigm Shift: Companies embarking on new innovation initiatives to determine feasibility of mHealth solutions for clinical research Audience Questions: How many of you currently employ mHealth technologies in your clinical trials? How may of you currently utilize Actigraphy and/or Continous Monitoring devices in your clinical trials? How many of you work in organizations that are now tasked with POCs or pilots using mHealth devices? Data Optimization: Enhancing clinical research with mHealth data requires new techniques against non-conventional sources. mHealth data alone does not enhance clinical research. Practical Applications: Applied use of mHealth in select therapeutic areas in combination with other clinical source data and/or ways where mHealth data may be combined for more visual insights | Copyright ERT 2018

SERVERLESS ARCHITECTURE – FEW EXAMPLES | Confidential – Internal Use Only

SERVERLESS ARCHITECTURE USING MICROSERVICES | Confidential – Internal Use Only

Lambda Architecture – execution of real time and batch together Characteristics Benefits Challenges Real time data processing Batch data processing Merging of real time and batch in single architecture Architectural complexity skillsets Serve both real and batch use cases simultaneously TIPS : One architecture supports both real time and batch Here are a few key learnings we hope to provide with this presentation that may inspire these organizations to embrace this change as an opportunity to innovate. Paradigm Shift: Companies embarking on new innovation initiatives to determine feasibility of mHealth solutions for clinical research Audience Questions: How many of you currently employ mHealth technologies in your clinical trials? How may of you currently utilize Actigraphy and/or Continous Monitoring devices in your clinical trials? How many of you work in organizations that are now tasked with POCs or pilots using mHealth devices? Data Optimization: Enhancing clinical research with mHealth data requires new techniques against non-conventional sources. mHealth data alone does not enhance clinical research. Practical Applications: Applied use of mHealth in select therapeutic areas in combination with other clinical source data and/or ways where mHealth data may be combined for more visual insights | Copyright ERT 2018

Lambda architecture- Physical view on aws (Serverless) │ © Copyright ERT 2018

KEY TAKEAWAYS FROM TODAY’S DISCUSSION Monolithic architecture Simple application; quick turn around – good for simple lightweight application – validate ideas quickly. Not ideal for complex and growing applications – scalability is the biggest issue. Microservices Complex application; scalability need is high; strong engineering skillsets available; distributed computing is complex; Microservices is no silver bullet – still depends on sound software design. Serverless Architecture Cloud-native; no HW to maintain; focus on code, not on infrastructure; works well with microservices; │ © Copyright ERT 2018

Questions? Prakriteswar Santikary, PhD prakriteswar.santi@ert.com ? …