Building reliable, high- performance communication systems from components Xiaoming Liu, Christoph Kreitz, Robbert van Renesse, Jason Hickey, Mark Hayden,

Slides:

Advertisements

Similar presentations

Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:

Advertisements

Berkeley dsn declarative sensor networks problem David Chu, Lucian Popa, Arsalan Tavakoli, Joe Hellerstein approach related dsn architecture status  B.

© Chinese University, CSE Dept. Software Engineering / Software Engineering Topic 1: Software Engineering: A Preview Your Name: ____________________.

Background information Formal verification methods based on theorem proving techniques and modelchecking –to prove the absence of errors (in the formal.

Distributed Data Flow Language for Multi-Party Protocols Krzysztof Ostrowski †, Ken Birman †, Danny Dolev § † Cornell University, § Hebrew University

Online Performance Auditing Using Hot Optimizations Without Getting Burned Jeremy Lau (UCSD, IBM) Matthew Arnold (IBM) Michael Hind (IBM) Brad Calder (UCSD)

Stuart AllenMark Bickford Robert Constable (PI) Christoph KreitzLori LorigoRobbert Van Renesse Secure software infrastructure Logic Programming Communications.

Algorithm for Virtually Synchronous Group Communication Idit Keidar, Roger Khazan MIT Lab for Computer Science Theory of Distributed Systems Group.

Formal Methods. Importance of high quality software ● Software has increasingly significant in our everyday activities - manages our bank accounts - pays.

Behavioral Design Outline –Design Specification –Behavioral Design –Behavioral Specification –Hardware Description Languages –Behavioral Simulation –Behavioral.

CS 501: Software Engineering Fall 2000 Lecture 16 System Architecture III Distributed Objects.

Programmability with Proof-Carrying Code George C. Necula University of California Berkeley Peter Lee Carnegie Mellon University.

Group Communication using Ensemble Part II. 2 Introduction From previous tutorial: Ensemble’s application interface: Concepts of Group Membership, View,

Illinois Institute of Technology

Microkernels: Mach and L4

A Type System for Expressive Security Policies David Walker Cornell University.

State Machines Timing Computer Bus Computer Performance Instruction Set Architectures RISC / CISC Machines.

CS294-9 :: Fall 2003 vic and NAÏVE K. Mayer-Patel.

Figure 1.1 Interaction between applications and the operating system.

Stuart AllenMark Bickford Robert Constable Richard Eaton Christoph KreitzLori Lorigo Secure software infrastructure Logic Programming Communications Advances.

Group Communication Robbert van Renesse CS614 – Tuesday Feb 20, 2001.

SNAL Sensor Networks Application Language Alvise Bonivento Mentor: Prof. Sangiovanni-Vincentelli 290N project, Fall 04.

Lecture 12 Synchronization. EECE 411: Design of Distributed Software Applications Summary so far … A distributed system is: a collection of independent.

The Horus and Ensemble Projects: Accomplishments and Limitations Ken Birman, Robert Constable, Mark Hayden, Jason Hickey, Christoph Kreitz, Robbert van.

.NET Mobile Application Development Remote Procedure Call.

Masking the Overhead of Protocol Layering CS514: Intermediate Course in Operating Systems Robbert van Renesse Cornell University Lecture 14 Oct. 12.

Data Structures and Programming.  John Edgar2.

EECS 262a Advanced Topics in Computer Systems Lecture 26 seL4 Kernel verification December 3rd, 2014 John Kubiatowicz Electrical Engineering and Computer.

CS 355 – Programming Languages

A Portable Virtual Machine for Program Debugging and Directing Camil Demetrescu University of Rome “La Sapienza” Irene Finocchi University of Rome “Tor.

Review: – computer networks – topology: pair-wise connection, point-to-point networks and broadcast networks – switching techniques packet switching and.

OHTO -99 SOFTWARE ENGINEERING “SOFTWARE PRODUCT QUALITY” Today: - Software quality - Quality Components - ”Good” software properties.

Introduction to Networks CS587x Lecture 1 Department of Computer Science Iowa State University.

Scientific Computing By: Fatima Hallak To: Dr. Guy Tel-Zur.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Proof Carrying Code Zhiwei Lin. Outline Proof-Carrying Code The Design and Implementation of a Certifying Compiler A Proof – Carrying Code Architecture.

High Performance Embedded Computing © 2007 Elsevier Lecture 3: Design Methodologies Embedded Computing Systems Mikko Lipasti, adapted from M. Schulte Based.

High Performance Embedded Computing © 2007 Elsevier Chapter 1, part 2: Embedded Computing High Performance Embedded Computing Wayne Wolf.

1 IOA: Mathematical Models  Distributed Programs Nancy Lynch November 15, 2000 Collaborators: Steve Garland, Josh Tauber, Anna Chefter, Antonio Ramirez,

Implementation support z programming tools y levels of services for programmers z windowing systems y core support for separate and simultaneous user-system.

Consistent and Efficient Database Replication based on Group Communication Bettina Kemme School of Computer Science McGill University, Montreal.

Ensemble and Beyond Presentation to David Tennenhouse, DARPA ITO Ken Birman Dept. of Computer Science Cornell University.

Improving the Efficiency of Fault-Tolerant Distributed Shared-Memory Algorithms Eli Sadovnik and Steven Homberg Second Annual MIT PRIMES Conference, May.

Farnaz Moradi Based on slides by Andreas Larsson 2013.

Model construction and verification for dynamic programming languages Radu Iosif

COP 5611 Operating Systems Spring 2010 Dan C. Marinescu Office: HEC 439 B Office hours: M-Wd 2:00-3:00 PM.

1 CSEP590 – Model Checking and Automated Verification Lecture outline for August 6, 2003.

Ensemble Fault-Tolerance Security Adaptation. The Horus and Ensemble Projects Accomplishments and Limitations Kent Birman, Bob Constable, Mayk Hayden,

Network Protocols Network Systems Security Mort Anvari.

Axel Jantsch 1 Networks on Chip A Paradigm Change ? Axel Jantsch Laboratory of Electronics and Computer Systems, Royal Institute of Technology, Stockholm.

MK++ A High Assurance Operating System Kernel Shai Guday David Black.

Programmability Hiroshi Nakashima Thomas Sterling.

Chapter 1 Basic Concepts of Operating Systems Introduction Software A program is a sequence of instructions that enables the computer to carry.

Enterprise Network Systems TCP Mark Clements. 3 March 2008ENS 2 Last Week – Client/ Server Cost effective way of providing more computing power High specs.

Formal Verification. Background Information Formal verification methods based on theorem proving techniques and modelchecking –To prove the absence of.

Motivation FACE architecture encourages modularity of components on data boundaries Transport Services Segment interface is centered on sending and receiving.

General requirements for BES III offline & EF selection software Weidong Li.

September 1999Compaq Computer CorporationSlide 1 of 16 Verification of cache-coherence protocols with TLA+ Homayoon Akhiani, Damien Doligez, Paul Harter,

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Copyright 1999 G.v. Bochmann ELG 7186C ch.1 1 Course Notes ELG 7186C Formal Methods for the Development of Real-Time System Applications Gregor v. Bochmann.

CHAPTER 1 INTRODUCTION TO COMPILER SUNG-DONG KIM, DEPT. OF COMPUTER ENGINEERING, HANSUNG UNIVERSITY.

CSE 219 Final exam review.

Chapter 3: Windows7 Part 1.

Logical architecture refinement

IOA Code Generator (Making IOA Run)

Lecture Topics: 11/1 General Operating System Concepts Processes

A Trusted Safety Verifier for Process Controller Code

Implementation support

CSE 1020:Software Development

Implementation support

Presentation transcript:

Building reliable, high- performance communication systems from components Xiaoming Liu, Christoph Kreitz, Robbert van Renesse, Jason Hickey, Mark Hayden, Ken Birman, Bob Constable (shepherd: Peter Lee) Cornell, Caltech, Compaq (CMU, Cedilla)

Why software components? 1.Ease design and development 2.Tuning to environment 3.Customization to user app 4.Extensibility 5.Verification and robustness

Why not? Configuration is hard Performance is bad –Abstraction barriers –Poor locality –Redundant code Not reliable Not faster

Group Communication Network

History V Ensemble Isis Horus Multicast Membership CATOCS Layers  Protocols ML Formal methods

Specification Concrete (ML) Implementation Properties (English) code of a layer logical predicates

Specification Abstract Spec Concrete (ML) Implementation Properties (English) refinement proof (I/O Automata)

Abstract IOA specification of totally ordered multicast S: array[integer] of message next: integer deliv: array[process] of integer action Multicast(m) { S[next++] := m; } action Deliver(p, m) precond: deliv[p] < next && m == S[deliv[p]] { deliv[p]++; } global state

Layer correctness abstract FIFO network spec abstract Total Order network spec Hickey, Lynch, Van Renesse, TACAS’99 network + layer == network++ Token layer on each CPU

Stack correctness Seqno layer FIFO total unreliable Token layer For example: Seqno Token

Efficiency? Ensemble stacks have many layers, improving clarity, but inefficient. 5 optimization techniques: 1.Avoiding (in-line) garbage collection 2.Avoiding marshaling 3.Delaying non-critical message processing 4.Identifying common paths 5.Header compression

A protocol layer is a function! old state new state input event output events layer

(off-line) partial evaluation Common Case Predicate original code Specialized code Hacker + Formal person NuPrl layer

Two-phase optimization ProgrammerFormalUser codeCCP bypass function Off-line On-line TT

Header compaction T_Data seqno T_Last Hash seqno Less space Faster processing

Architecture (deliver only) Network Application ? generated bypass original stack CCP (e.g., hdr.seq = win.lo) Transport driver (marshaling, device independence)

Architecture Network Application ? ? generated bypass original stack CCP Transport driver (marshaling, device independence)

Performance Three different versions: 1.Original (ORIG) 2.Hand-optimized (HAND) 3.Machine-optimized (MACH) 300 MHz UltraSparc/Solaris 2.6 OCaml 2.0 native code compiler

Code latency (  sec) 10 layer4 layer stack ORIGMACHORIGMACHHAND Down Stack Down Transport Up Transport Up Stack Total (See paper for CPU cycles and TLB misses)

Lessons learned 1.Design with formalization in mind 2.Use small, but not too small components 3.Use a language with formal semantics 4.Use IOA as a specification language 5.Use formal tool with in-house expertise

Final remarks See CD or Web for code samples, links to all code, as well as how to reproduce our results Still working on a machine-generated proof of correctness