1. Distributed Systems
Bina Ramamurthy
2018 B. Ramamurthy
2/17/2019

2. Introduction
A distributed system is a network of autonomous computers cooperating to accomplish a task.
Hardware and software components of a distributed system coordinate their activity by sharing resources such as data, computation, compute cycles, bandwidth and storage.
Examples: Internet, intranet, mobile computing systems, the cloud, Hadoop, Kafka, Cassandra, …
2018 B. Ramamurthy
2/17/2019

3. Learning Outcomes Discuss the evolution of the internet technology
Explore the fundamental concepts that define a distributed systems.
Explain the basics of a client/server system.
List some of the challenges facing the design and development of distributed systems.
In general, learn about the distributed ecosystem in computing.
2018 B. Ramamurthy
2/17/2019

4. Topics Some fundamental terms Advances in Client-side technologies
Communication network
Middleware concept
Server-side technology
Application models (web, web2.0, web3.0)
2018 B. Ramamurthy
2/17/2019

5. Evolution of Internet Computing
2018 B. Ramamurthy
deep web
scale
web
Parallel
HPC
Semantic
discovery
??????
Automate (discovery)
Discover (intelligence)
Transact
Integrate
Interact
Inform
Publish
time
2/17/2019

6. Evolution of Internet Computing (2019)
Deep web : Take 2
2018 B. Ramamurthy
scale
web
Parallel
HPC
Semantic
web
Data Analytics; AI; ML
Peer to Peer Blockchain Tx
Automate (discovery)
Discover (intelligence)
Transact
Integrate
Interact
Inform
Publish
time
2/17/2019

7. Beyond Search Engines: Enabling Information Technology and Scientific Applications: we need security and trust layers
Simple Search (stateless)
TV/Remote
Financial: Build Portfolio
Medicine: plan treatment
Environment: Plan Forestation
Wireless device
Biotech: drug discovery
Complex multi-organizational applications
2018 B. Ramamurthy
2/17/2019

8. Terminologies
2018 B. Ramamurthy
2/17/2019

9. Protocol
Protocol is a set of rules that end points in a telecommunication system use when exchanging information.
IP: Internet protocol defines an unreliable packet transfer protocol.
TCP: Transmission Control Protocol builds on IP to define a reliable data delivery protocol.
LDAP: Lightweight Directory Access Protocol builds on TCP to define a query-response protocol for querying the state of a remote database.
HTTP: Hyper Text Transfer Protocol builds on TCP to facilitate hyper-text document exchange.
HTTPS: Security layer over HTTP
Bitcoin runs on its protocol, Ethreum too!
2018 B. Ramamurthy
2/17/2019

10. Service
Service is a network-enabled entity that provides a specific capability.
Service = Protocol + Behavior
A service definition permits many implementations.
Examples: ability to move files, create processes, verify access rights
An SFTP server speaks (secure) File Transfer Protocol and supports remote read and write access to a collection of files.
2018 B. Ramamurthy
2/17/2019

11. API
Application Program Interface (API) defines a standard interface for invoking a specified set of functionality.
Twitter API, Facebook API that allow third party developers to create applications based on the services (functions) exposed on certain endpoints (ports)
2018 B. Ramamurthy
2/17/2019

12. SDK
Software Development Kit (SDK) denotes a set of code designed to be linked with, and invoked from within, an application program to provide specified functionality.
An SDK typically implements an API.
An SDK is a set of tools that facilitates application development
Example: Android SDK, for example, provides tools such as its emulator, adb (Android Debug Bridge)
2018 B. Ramamurthy
2/17/2019

13. Internet Internet is a very large distributed system.
Interconnection of a collection of heterogeneous networks of computers.
Protocols: IP, TCP, HTTP
Services: world wide web (www), file transfers (ftp), , etc.
Web : application on top of the internet: Linux Apache MySQL PHP (LAMP) stack; being replaced by MEAN stack: MongoDB, Express, Angular, Node(.js)
Hmm.. What is a stack? Layers of technology to accomplish a general application.
2018 B. Ramamurthy
2/17/2019

14. Client-server DS: Web Application
webpage
Server
Web browser
Firefox, Safari
.html file
Style files
displays
interprets
image and audio files
Prepare/edit files
2018 B. Ramamurthy
2/17/2019

15. Advances in Client-side programming
2018 B. Ramamurthy
2/17/2019

16. Client programming Simple programs written as a single module
Single entry point typically in a “main” function
Procedural, functional and object-oriented
Applications, (web-based)
More recently the focus is on rich content: Rich Internet Application (RIA)
JS, JS frameworks (React, Angular)
Model-View-Controller (MVC) model for design and deployment flexibility. Ruby on Rails, HTML-CSS-Browser/js
2018 B. Ramamurthy
2/17/2019

17. Client/Server
Server: refers to a process on a networked computer that accepts requests from other (local or remote) processes to perform a service and responds appropriately.
Client: requesting process in the above is referred to as the client.
Request and response are in the form of messages.
Request/response model
Client is said to invoke an operation on the server.
Many distributed systems today are constructed out of interacting clients/servers.
Issues: connectivity (speed and accessibility), addressing, naming
Standards: WS, SOAP, REST
2018 B. Ramamurthy
2/17/2019

18. Advances in Networking
2018 B. Ramamurthy
2/17/2019

19. Internetworking Internet stack Standardization
IPV4, IPV6: Internet protocol version 4, 6
Tremendous increase in network bandwidth: measured in bits per second
From few kilobits per second (56kb/s dial up lines to
1.5Mb/s T1 to 100Gb/s ethernet)
What can you do with such fast delivery speed? Connect them up.. Networked application models
2018 B. Ramamurthy
2/17/2019

20. Communication Network
Application
Application
A communication middleware framework isolates the application developers from the details of the network protocol.
Network Protocol Stack
Network Protocol Stack
A communication middleware framework isolates the application developers from the details of the network protocol.
2018 B. Ramamurthy
2/17/2019

21. Client/server Issues
Basic object-technology could not fulfill the promises such as reusability and interoperability fully in the context internet and enterprise level applications. Deployment was still a major problem and as a result portability and mobility were impaired.
2018 B. Ramamurthy
2/17/2019

22. Issues in networked systems
Heterogeneity in various aspects of a distributed systems
Communication modes different
Synchronous: RPC
Asynchronous: P2P, Publish and subscribe
Variations in products
Many vendors: IBM, CISCO, Apache, Adobe
Additional runtime features:
fault tolerance, load balancing, transaction handling, usage metering, auditing, ..
2018 B. Ramamurthy
2/17/2019

23. Advances in Middleware
2018 B. Ramamurthy
2/17/2019

24. Communication Middleware
Application
Application
Middleware
Middleware
Network Protocol Stack
Network Protocol Stack
2018 B. Ramamurthy
2/17/2019

25. Remote procedure call (RPC)
Client
Application
Server
Application
Procedure
call
Execute
call
RPC Stub code
RPC stub code
RPC library/runtime
RPC library/runtime
RPC stubs and runtime enable location transparency, encapsulate RPC comm infrastructure and provide a proceudre call interface.
Network Protocol Stack
Network Protocol Stack
RPC stubs and runtime enable location transparency, encapsulate RPC communication
infrastructure and provide a procedure call interface.
2018 B. Ramamurthy
2/17/2019

26. Technical Layers Participant A Participant B Participant C XML
Core
Assets
Participant A
Participant B
Participant C
Business
Logic
Technology
Independent
Interface
Description
Technology
Independent
Interface
Description
Technology
Independent
Interface
Description
Middleware
Mapping
XML
Technology
Adapters
Middleware
Buses
Communication
facilities
XML Web services
2018 B. Ramamurthy
2/17/2019

27. Message-oriented Middleware (MOM)
Made famous by IBM’s MQseries and TIBCO’s Rendezvous products.
Based on messages and queues.
A message contains a header and a payload.
A queue can store and distribute messages.
Publish/subscribe model of communication:
A topic offers another model of communication between subscribers and publishers.
MOM allows for loose coupling between message consumers and message producers enabling dynamic, reliable, flexible, high-performance systems to be built.
2018 B. Ramamurthy
2/17/2019

28. Server-side Advances
2018 B. Ramamurthy
2/17/2019

29. Server-side advances: Two-tier applications
Presentation
Logic
Business
Database
Server
2018 B. Ramamurthy
2/17/2019

30. Server-side: Three-tier Applications
Presentation
Logic
Business
Database
Server
Tremendous advances in DBMS: relational, query languages,
Object-relational…ACID property transactional systems
2018 B. Ramamurthy
2/17/2019

31. Programming Model for Web-based applications
Web Service
Business Logic
Web
Application
Database
Server
Enterprise
components
Logic container
Web Container
Web
client
2018 B. Ramamurthy
2/17/2019

32. Application Programming Model for Three-tier Applications
Presentation
Components
Database
Server
Enterprise
Container
Application
Business Logic
2018 B. Ramamurthy
2/17/2019

33. Expectations of a Distributed System
Access transparency: enables local and remote resources to be accessed using identical operations.
Location transparency: enables resources to be accessed without knowledge of their location.
Concurrency transparency: enables several processes to operate concurrently using shared resources without interference between them.
Replication transparency: enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers.
Failure transparency: enables the concealment of faults, allowing users and application programs to complete their tasks despite the failure of hardware or software components.
Mobility transparency: allows the movement of resources and clients within a system without affecting the operation of users or programs.
Performance transparency: allows the system to be reconfigured to improve performance as loads vary. “Scalability”
Expansion transparency: allows the system and applications to expand in scale without change to the system structure or the application algorithms.
2018 B. Ramamurthy
2/17/2019

34. Issues contd. Heterogeneity of components
Scalability : ability to perform well under increased loads and data sizes
Failure handling
Concurrency
Transparency
Reliability
Interoperability
Performance
Openness
Security and protection
2018 B. Ramamurthy
2/17/2019

35. Emerging Application Models
2018 B. Ramamurthy
2/17/2019

36. Large scale systems
Most emerging distributed applications are very large demanding large amounts of storage and data resources
E-commerce systems and online businesses
Applications connecting communities: from search to social networking
Amount of data collected by various sources from terrorism monitoring to environmental monitoring
Data deluge: most of the data is write once read many (WORM)  Analytics resulting in data-intensive computing models and big-data computing (CSE487 material)
We will look just one success story of distributed system: amazon.com
2018 B. Ramamurthy
2/17/2019

37. Amazon.com
Werner Vogels’ talk “Order in the Chaos: Building the Amazon.com Platform."
1995: Started out with a single web service on a single server. Today amazon has about 150 web services on its homepage alone.
1 million merchant partners; 60 million customers
One server of customers and inventory grew into two servers; more database servers were added as the business expanded
1999: A mistep during this exponential growth period was moving to mainframe from distributed server. Failed to meet scalability, reliability and performance; it was scratched in 2000.
2018 B. Ramamurthy
2/17/2019

38. Amazon (contd.)
Robustness: Shopping cart is tested for items by a single customer, for example!
Amazon’s secret sauce is “operating reliably at scale”.
After “the denial of service” debacle in 1999, they decided to use Web services to insulate the databases from being overwhelmed by direct interaction with online applications.
Each web service is the responsibility of a team of developers:
“And they are not just responsible for writing the service and then tossing it over the wall for testing and eventual entry into production where some poor maintenance geek has to look after it.
The Amazon CTO tells his Web services team members: "You build it. You own it."
That means the team is responsible for its Web service's on-going operation. If a Web service stops working in the middle of the night, team members are called to fix it.”
Web services are kept simple: complexity is the notorious enemy of reliability
No attachment to one technology or standard: what ever customer wants, give it.
2018 B. Ramamurthy
2/17/2019

39. On to more fundamental concepts
2018 B. Ramamurthy
2/17/2019

40. Synchrony Synchronous and asynchronous communications Synchronous:
immediate response of communicating partners
Server process/thread blocks until response is completed
Follows request/response pattern
Used when servers are available all the time
Typically communicating partners are tightly coupled
Examples:
request from web client to a web browser for “search” or for “information”
Traditional remote procedure call (RPC)
2018 B. Ramamurthy
2/17/2019

41. Asynchronous communication
Communicating partners are decoupled
Message driven:
sender creates a message and delivers it to a mediator who then sends it to “a” recipient
Server need not be available all the time
Sender and receiver loosely coupled
Can facilitate high-performance message-based system
Example:
Any event-driven system
Any messaging system (instant messenger)
Publish-subscribe mode communications
2018 B. Ramamurthy
2/17/2019

42. Interface vs Payload Semantics
Typically interaction between a client and a server results in the execution of an activity (or transaction)
Request needs to be specified by the request.
Interface semantics: Requested activity can be encoded in the operation signature in the server’s “interface” or
Payload semantics: It can be embedded in the message itself
2018 B. Ramamurthy
2/17/2019

43. Interface Semantics
Process1
Process2
getCustomer()
retrieveCustomerData()
returnResult()
Semantics of the activity is explicitly stated in the
message/method call: RPC
2018 B. Ramamurthy
2/17/2019

44. Payload Semantics Process 1 Process 2
Envelop
With message
Process 1
Process 2
Requested transaction/activity is embedded in the message
Details of the activity not explicit; the semantics are
embedded in the message
2018 B. Ramamurthy
2/17/2019

45. Payload Semantics
onMessage()
2018 B. Ramamurthy
2/17/2019

46. Payload semantics is generic
String transferMoney (amt: decimal, accTo: String) { …} String executeService (message: String)
2018 B. Ramamurthy
2/17/2019

47. Tight vs. Loose Coupling
On the business front loose coupling addresses the growing need for companies to be flexible and agile with respect changes in their own processes and those of their partners
How does loose coupling help in improving agility, flexibility and performance?
2018 B. Ramamurthy
2/17/2019

48. Tight vs. Loose coupling
Level
Tight coupling
Loose coupling
Physical coupling
Direct physical link required
Physical intermediary
Communication style
synchronous
asynchronous
Type system
Strongly typed (interface semantics)
Weak type system (payload semantics)
Interaction pattern
OO-style navigation of complex object trees
Data-centric, self-contained messages
Control of process logic
Central control of process logic
Distributed logic components
Service discovery and binding
Statically bound services
Dynamically bound services
Platform dependencies
Strong OS and programming language dependencies
OS- and programming language dependent
2018 B. Ramamurthy
2/17/2019

49. Challenges
2018 B. Ramamurthy
Need transformative solutions such as the Internet and the Search
Alignment with the needs of the business / user / non-computer specialists / community and society
Need to address the scalability issue: large scale data, high performance computing, automation, response time, rapid prototyping, and rapid time to production
Need to effectively address (i) ever shortening cycle of obsolescence, (ii) heterogeneity and (iii) rapid changes in requirements
Transform data from diverse sources into intelligence and deliver intelligence to right people/user/systems
2/17/2019

50. Tools to explore
Design tool: Design representation: Any design tool for class diagrams, sequence diagrams; starUML
Block diagrams: MS Visio, draw.io
Review and brush up your coding skills
Login into CSE Linux system (ssh timberlake.cse.buffalo.edu) and make sure you are able to transfer files to and from it (Filezilla, winscp)
2018 B. Ramamurthy
2/17/2019

51. Summary
We discussed the fundamental choices available to a designer in assembling a distributed system
A designer must choose appropriate communication infrastructure, synchrony, call semantics, use of intermediary, tightvs loose coupling, and challenges.
In this course, we will study different types of distributed systems and learn to design, develop and implement distributed systems.
2018 B. Ramamurthy
2/17/2019