Software System Performance

Slides:



Advertisements
Similar presentations
Web Performance Tuning Lin Wang, Ph.D. US Department of Education Copyright [Lin Wang] [2004]. This work is the intellectual property of the author. Permission.
Advertisements

Categories of I/O Devices
Operating System.
  Copyright 2003 by SPAN Technologies. Performance Assessments of Internet Systems By Kishore G. Kamath SPAN Technologies Testing solutions for the enterprise.
CS CS 5150 Software Engineering Lecture 19 Performance.
CS CS 5150 Software Engineering Lecture 25 Performance.
RDMA ENABLED WEB SERVER Rajat Sharma. Objective  To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP.
Measuring Performance Chapter 12 CSE807. Performance Measurement To assist in guaranteeing Service Level Agreements For capacity planning For troubleshooting.
1 Software Testing and Quality Assurance Lecture 40 – Software Quality Assurance.
1 CS 501 Spring 2005 CS 501: Software Engineering Lecture 22 Performance of Computer Systems.
Basics of Operating Systems March 4, 2001 Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard.
Virtual Memory Tuning   You can improve a server’s performance by optimizing the way the paging file is used   You may want to size the paging file.
Measuring zSeries System Performance Dr. Chu J. Jong School of Information Technology Illinois State University 06/11/2012 Sponsored in part by Deer &
Bottlenecks: Automated Design Configuration Evaluation and Tune.
Software Writer:-Rashedul Hasan Editor:- Jasim Uddin.
Section 2 Section 2.1 Identify hardware Describe processing components Compare and contrast input and output devices Compare and contrast storage devices.
1 CS 501 Spring 2006 CS 501: Software Engineering Lecture 22 Performance of Computer Systems.
1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management.
Chapter 7 Operating Systems. Define the purpose and functions of an operating system. Understand the components of an operating system. Understand the.
Operating Systems David Goldschmidt, Ph.D. Computer Science The College of Saint Rose CIS 432.
CS 360 Lecture 11.  In most computer systems:  The cost of people (development) is much greater than the cost of hardware  Yet, performance is important.
CS 127 Introduction to Computer Science. What is a computer?  “A machine that stores and manipulates information under the control of a changeable program”
We will focus on operating system concepts What does it do? How is it implemented? Apply to Windows, Linux, Unix, Solaris, Mac OS X. Will discuss differences.
Assoc. Prof. Dr. Ahmet Turan ÖZCERİT.  What Operating Systems Do  Computer-System Organization  Computer-System Architecture  Operating-System Structure.
FroNtier Stress Tests at Tier-0 Status report Luis Ramos LCG3D Workshop – September 13, 2006.
Software System Performance CS 560. Performance of computer systems In most computer systems:  The cost of people (development) is much greater than.
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Measuring Performance Based on slides by Henri Casanova.
Databases and DBMSs Todd S. Bacastow January 2005.
Section 2.1 Section 2.2 Identify hardware
OPERATING SYSTEMS CS 3502 Fall 2017
Lecture 2: Performance Evaluation
Memory Management.
Module 12: I/O Systems I/O hardware Application I/O Interface
Chapter 13: I/O Systems Modified by Dr. Neerja Mhaskar for CS 3SH3.
Resource Management IB Computer Science.
Processes and threads.
Operating System.
CS 6560: Operating Systems Design
Software Architecture in Practice
Diskpool and cloud storage benchmarks used in IT-DSS
Lecture 16: Data Storage Wednesday, November 6, 2006.
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Operating Systems (CS 340 D)
Chapter 2: System Structures
Introduction to Operating System (OS)
Computer Basics Section 2.1 YOU WILL LEARN TO… Identify hardware
Systems Software Keywords Operating Systems
CSCI 315 Operating Systems Design
OPERATING SYSTEMS.
Computer Science I CSC 135.
Software System Performance 2
Software System Performance
Lecture 1: Multi-tier Architecture Overview
I/O Systems I/O Hardware Application I/O Interface
Operating System Concepts
13: I/O Systems I/O hardwared Application I/O Interface
CS703 - Advanced Operating Systems
CSE8380 Parallel and Distributed Processing Presentation
CS 501: Software Engineering Fall 1999
Software models - Software Architecture Design Patterns
Lecture Topics: 11/1 General Operating System Concepts Processes
Software System Testing
Software Verification, Validation, and Acceptance Testing
Chapter 13: I/O Systems I/O Hardware Application I/O Interface
Performance And Scalability In Oracle9i And SQL Server 2000
Chapter 13: I/O Systems.
An Introduction to Operating Systems
Module 12: I/O Systems I/O hardwared Application I/O Interface
Chapter 13: I/O Systems “The two main jobs of a computer are I/O and [CPU] processing. In many cases, the main job is I/O, and the [CPU] processing is.
Presentation transcript:

Software System Performance IMPORTANT NOTICE TO STUDENTS: These slides are NOT to be used as a replacement for student notes. These slides are sometimes vague and incomplete on purpose to spark class discussions. Software System Performance CS 360 Lecture 13

Software System Performance Outline Software system performance definition When performance matters Software Performance Requirements Performance challenges Performance metrics and testing Sysbench (synthetic benchmarking) Software timers (application-specific benchmarking) Performance monitor

Software System Performance The amount/type of work accomplished by a computing system involving one or more of the following: Data processing efficiency Response time Throughput Resource utilization Resource availability Scalability Portability Reliability

When Performance Matters Real time systems Computation must be fast enough to support the service provided. EX: Internet routers examining packet headers Drive-by-wire, vehicle throttle controlled by software Real time performance measurements Context switching Time required to save the current process/thread and find the highest priority ready process/thread and restore its context Interrupt Latency Time Time between internal/external interrupt event until the interrupt processing begins. Immediate Response Time Time required to process a request immediately without context switching

When Performance Matters User interfaces Where humans have high expectations. Mouse tracking must appear instantaneous. Performance considerations Start-up time factors for applications GPU initialization, Storage initialization, GUI process load time, GUI rendering time Web interfaces must process requests in near real time Use client-side and server-side programs to increase performance

Software Performance Requirements (documentation) Tasks needed for performance documentation: Gathering performance requirements Developing a plan to test for performance Decide whether to use internal or external resources to perform tests. Ex: Sysbench (internal) requires resources to run Ex: Web browser (external) requires no resources to run Specify testing tools Define the test data Configure the testing environment

Software Performance Requirements (documentation) Tasks needed for performance documentation: Develop proof of concept test scripts for each component/interface Initial test run – to check correctness of testing script Execute tests (repeatedly, to get average) Record and analyze the results (Pass/Fail) Investigate corrective action if test fails Re-tune components/interfaces/system and repeat process

Performance challenges for all software systems (Documentation) Predict performance problems before a system is created. Hardware bottlenecks? Research/Document Raspberry Pi hardware component performance. (CPU, memory type/interface, SD card type/interface, network interface) Software bottlenecks? Research/Document performance for software languages/frameworks used, component interfaces, process algorithms.

Look for bottlenecks Usually, CPU performance is not the limiting factor. Hardware bottlenecks Moving data Disk to main memory Main memory to CPU Shortage of memory Paging Network capacity Bandwidth Inefficient software Poorly written algorithms that do not scale well Sequential processing where a parallel approach should be used.

Performance as Time Metrics Time between the start and the end of an operation Also called running time, elapsed time, wall-clock time, response time, latency, execution time, ... Most straightforward measure: “Component X takes 2.5s (on average) using a Cortex-A53 CPU running at 1.2GHz.” Must be measured on a “dedicated” machine Raspberry Pi3 model B Time performance metrics will be different on other machines. Component X time metrics on Pi3 vs. Pi2 vs. Pi1 … Research the Linux “time” command.

Performance as Rate Metrics Measurement of the throughput of a software/hardware component. Performance rate can be independent on the “size” of the application: Compressing a 1MB file takes 1 minute. compressing a 2MB file takes 2 minutes; the performance is the same in both cases. 1MB/sec Mflops: Millions of floating point operations /sec Very popular assessment of hardware (CPU), but often misleading. Ex: 450 MFlops Doesn’t take into account data movement between hardware components. Research “Linpack” Application-specific Number of frames rendered per second. Number of database transactions per second. Number of HTTP requests served per second.

Predicting system performance Direct measurement on subsystem (benchmark) File I/O CPU Network All require detailed understanding of the interaction between software and hardware systems.

Direct measurement on subsystem – SysBench (Documentation) Sysbench is a modular, cross-platform, multi-threaded benchmark tool used to quickly gain information about system performance.

Direct measurement on subsystem – SysBench (Documentation) Test the following (on the base OS, and in a Docker Container): CPU performance Cpu-max-prime test: 10,000 (calculates all prime numbers between 1 and 10,000). Graph (line graph) the results Number of threads (X-axis) vs. Execution time (Y-axis) Threads should increase from 1 to 256, by powers of 2. Generate documentation for the graph, why it curves, etc.

Direct measurement on subsystem – SysBench (Documentation) Test the following (on the base OS, and in a Docker Container): File I/O performance fileio test: perform random reads/writes to a dataset of a specified size. Graph (line graph) the results File size (X-axis) vs. Throughput(Y-axis) File size should increase from 1MB to 2GB, by powers of 2. Generate documentation for the graph, why it curves, etc. Execute the cleanup command after each test run: sysbench --test=fileio --file-total-size=X cleanup

Direct measurement on subsystem – SysBench (Documentation) Test the following (in the DB container): Database Service Performance oltp test: performs random read, write, select, update, delete, and insert queries on a database table of a specified size. Num-threads (simulated users/requests) (X-axis) vs. Throughput(Y- axis) Threads should increase from 1 to 256, by powers of 2. Max_connections will be reached before 256. Generate documentation for the graph, why it curves, etc. Execute the cleanup command after each test run: sysbench --test=oltp --mysql-db=test --mysql-user=root --mysql- password=yourrootsqlpassword cleanup

Measurements on Software modules (documentation) Creating timers to measure function performance: DatabaseCallMethod(String[] d){ for 1 to d.length //format values in String array //connect and update database with String data }//end method

Measurements on Software modules (documentation) Creating timers to measure function performance: DatabaseCallMethod(String[] d){ for 1 to String.length //format values in String[] //connect and update database with String data }//end method DatabaseCallMethod(data);

Measurements on Software modules (documentation) Creating timers to measure function performance: DatabaseCallMethod(String[] d){ for 1 to String.length //format values in String[] //connect and update database with String data }//end method start = getSystemTime(); DatabaseCallMethod(data); end = getSystemTime(); MethodRunTime = end – start; Invoke the calling method multiple times to calculate the average execution time.

Fixing bad performance If a system performs badly, begin by identifying the cause: Instrumentation: Add timers to the code. Reveals delays in specific parts of the system. Test loads: Run the system with varying loads High transaction rates, large input files, many users, etc. Design and code reviews: Team review of system and program design, and suspect sections of code. May reveal algorithms/components that are running very slowly. Find the underlying cause and fix it or the problem will return!

Analyzing system performance Five common issues when a performance problem exists: Product wasn’t developed using performance testing requirements. When a problem develops or is found, no one takes responsibility. Developers don’t use (don’t know about) tools that are available to solve the problem. After developing a list of possible causes, there’s no elimination of the major problems. Developers can’t determine the priority of the problems. Often developers don’t have the patience to examine the large amounts of data generated by performance testing.

Performance Monitor (Documentation) Each group should implement a (web-based) admin interface. This interface should provide real-time results about multiple components developed for the project. Examples: Average number of HTTP requests per hour Number of page visits for each page Number of user logins for a given time span Current number of active users Current CPU/RAM utilization Reliability Metrics Number of faults/transactions or requests Docker Container(s) status Active, unresponsive, etc.