Practical Web/E-Commerce Testing Part 1 – Functional Testing Robert Sabourin President AmiBug.Com, Inc. Montreal, Canada rsabourin@amibug.com September 19, 2018 © Robert Sabourin, 2001

Practical Web/E-Commerce Testing Part 1 – Functional Testing Overview: Quality Attributes & What can break? Web Tier Application Tier Data Tier September 19, 2018 © Robert Sabourin, 2001

Practical Web/E-Commerce Testing Part 1 – Functional Testing Robert Sabourin , Software Evangelist President AmiBug.Com Inc. Montreal, Quebec, Canada rsabourin@amibug.com September 19, 2018 © Robert Sabourin, 2001

AmiBug.Com, Inc. Software Development & SQA Consulting Services Training, Coaching and Professional Development Light Effective Process Team Building and Organization We help people to get things done! September 19, 2018 © Robert Sabourin, 2001

I am a Bug In the style of a children's book. Robert & Catherine Sabourin ISBN: 0-9685774-0-7 www.amazon.com In the style of a children's book. Explains elements of software development process in a fun easy to read format. September 19, 2018 © Robert Sabourin, 2001

Fundamental Question How do you know when you are finished? September 19, 2018 © Robert Sabourin, 2001

Crosby on Quality “Quality is defined as conformance to requirements” “Quality is not a measure of GOODNESS” Phil B. Crosby, Quality is Free September 19, 2018 © Robert Sabourin, 2001

Deming Quality approach (PDCA) Plan, Do, Check, and Act: Plan what you want to implement. Do the pilot implementation. Check the results of the pilot. Act on the results by tweaking the process before the next project. September 19, 2018 © Robert Sabourin, 2001

Edsger W. Dijkstra “Program testing can be used to show the presence of bugs, but never to show their absence” September 19, 2018 © Robert Sabourin, 2001

"No! Try not, Do. Or do not. There is no try." September 19, 2018 © Robert Sabourin, 2001

Definition of a Bug To make our job more fun, whenever we have a concern with software, we call it a “bug”. September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications It’s all about people! (and the occasional bug too) September 19, 2018 © Robert Sabourin, 2001

Purpose What is the purpose of testing? Common definition of the purpose of testing Our purpose is to find bugs before our customers do! Broader definition The role of testing is to provide objective input to facilitate business decisions (wise smart and good decisions) September 19, 2018 © Robert Sabourin, 2001

About Bugs Bugs are not Good or Bad September 19, 2018 © Robert Sabourin, 2001

About Bugs Some bugs are important and have a high priority! September 19, 2018 © Robert Sabourin, 2001

About Bugs Some bugs are dangerous and have a high severity! September 19, 2018 © Robert Sabourin, 2001

About Bugs Setting the priority and severity of a bug is a business decision September 19, 2018 © Robert Sabourin, 2001

About Bugs Changing business conditions impact the priority and severity of a bug! Always review previous decisions in light of changing business context Ensure staff assigning priority and severity are aware of all relevant business drivers September 19, 2018 © Robert Sabourin, 2001

Bug Quadrants September 19, 2018 © Robert Sabourin, 2001

Business Decisions SQA Role: Development Role: Objective input Development Role: Technical implementation/Risk Project/Product Management Role: Customer driven requirements/Impact September 19, 2018 © Robert Sabourin, 2001

Quadrant Changing Same technical bug can be in a different quadrant depending on the business context Monitor business drivers! Focus find and fix quadrant -1- bugs high priority/high severity September 19, 2018 © Robert Sabourin, 2001

Finished? How do you know you are finished? September 19, 2018 © Robert Sabourin, 2001

You know you are finished when … … the only bugs left are the ones that Product Management and Development agree are acceptable (based on objective SQA input) ... September 19, 2018 © Robert Sabourin, 2001

You know you are finished when … … the only bugs left are the ones that Product Management and Development agree are acceptable (based on objective SQA input) … At least for now! September 19, 2018 © Robert Sabourin, 2001

Just for Fun September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications Quality Attributes September 19, 2018 © Robert Sabourin, 2001

Quality Factors Definition Attributes of software If not implemented they pose a risk to the success of the software business risk Factors that need to be considered when deciding your testing strategy September 19, 2018 © Robert Sabourin, 2001

Quality Factors Why? Purpose Focus The purpose of testing can be viewed as being activities done to identify issues of concern to the commercial use or deployment of software Focus Quality factors help us to focus testing on the most important characteristics of the software September 19, 2018 © Robert Sabourin, 2001

Perceived Software Quality Factor Perceived Software Quality Adaptability Can functions be enhanced without major redesign? Accessibility Can all users access your application? Auditibility Can a log be presented showing all access&transactions? Availability Can functions be accessed when needed? Continuity Can system continue operation recovering from a failure? Dependability Can users trust results provided? Expandability Can functions be added without major redesign? Functionality Does software perform as advertised? Integrity Is all data stored and returned unaltered. Interoperability Can be used with other software? September 19, 2018 © Robert Sabourin, 2001

Perceived Software Quality Factor Perceived Software Quality Maintainability Can software be updated to new release levels? Operability Can software be easily put into operation? Portability Can software be run in different environments? Reliability How long does the software operate without failure? Re-usability Can elements of software be reused in future? Scalability Can transaction volume increase with more resources? Security Can unauthorized users access systems and data? Serviceability Can bug fixes be integrated with minimum downtime? Testability Can all functions and features be tested? Usability Can the software be easily used by target user? September 19, 2018 © Robert Sabourin, 2001

Ranking Quality Factors It is important to determine the relative importance of each Quality Factor for your project! Testing resources are always severely limited Focus on things important for our business Use a systematic approach Ranking will be used to ensure testing effort is reasonably spread across Quality Factors September 19, 2018 © Robert Sabourin, 2001

Ranking Quality Factors September 19, 2018 © Robert Sabourin, 2001

Example Ranking September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications What Can Break? September 19, 2018 © Robert Sabourin, 2001

Overview Types of Web Systems Basic Web Architecture Typical Transaction Exercise September 19, 2018 © Robert Sabourin, 2001

Types of Web Systems Any Combinations Informational Delivery Customized access User-provided content Interactive File sharing Transaction oriented Application service provider Database access Document access Workflow oriented Automatic content generator Any Combinations September 19, 2018 © Robert Sabourin, 2001

Quality Factors Importance For Different Web Application Types September 19, 2018 © Robert Sabourin, 2001

Basic Web Architecture Data Services Application Components Process HTTP Requests Application Dependent Code September 19, 2018 © Robert Sabourin, 2001

Workstation Component WORKSTATIONS Browser Operating System Java Virtual Machine User’s Locale Plug-Ins Media Players Viewers Display depth Display resolution Resident applications September 19, 2018 © Robert Sabourin, 2001

Workstation Processing WORKSTATIONS Render all pages based on input from Web Server. Locally run objects such as FLASH animations Execute active X components Execute JAVASCRIPT Execute JAVA Execute VB SCRIPT Execute Viewers Initiate Email Store local user files Cookies, Cached objects September 19, 2018 © Robert Sabourin, 2001

Internet Connections Internet - Network - Servers - Switches - Routers September 19, 2018 © Robert Sabourin, 2001

Internet Connections Web Server - All HTTP Requests are handled by a Web Server - To service the request: - Find page requested and send it to requestor - or Generate another request to another server - or Execute a local script or program requestor - or Execute a local active server page script which generates a page to send to the requestor - or Execute a local CGI script and send result to - or Return an error message indicating that the request could not be filled presently September 19, 2018 © Robert Sabourin, 2001

Internet Connections Application Server - Web Application Business Logic is executed on the application server - Services and objects processed are totally dependent on the Web Application Purpose - Sometimes developed using third party application infrastructures such as IBM WebSphere September 19, 2018 © Robert Sabourin, 2001

Internet Connections Data Server - Data layer - May contain application specific data - May contain security data - Implemented with one or more physical servers - May include several Databases - Communication to Application Server and Web Server via standard interfaces (SQL) - May include stored procedures and other data handling related application code - May be shared with other applications September 19, 2018 © Robert Sabourin, 2001

Firewalls Firewalls - Software or hardware combined - Restricts traffic - Source - Destination - Type of data - Authentication - Authorization September 19, 2018 © Robert Sabourin, 2001

What can break? Time Attention span on Web is limited! If the time to get a response from any Web or E-Commerce application is longer than 8-12 seconds most users will surf to some another site! September 19, 2018 © Robert Sabourin, 2001

What can break? Time Quick fixes and bandage solutions often lead to performance degradation! Operation with full databases, nearly 100% consumed system resources. Stuff outside and within your control impacts response time! September 19, 2018 © Robert Sabourin, 2001

Areas of control Outside of Web Application Developers Direct Control Within Applications Developers Direct Control September 19, 2018 © Robert Sabourin, 2001

Workstation Bugs! User Application Displays and interactivity dependent on Browser Type, Version & O/S User Application Missing or wrong version of Third Party Plug-In or Object Incompatible or Missing JAVA Runtime environment and engine Locale not supported including Language, Fonts other resources Screen resolution and color support not match application Incompatible or Missing Viewer (PDF, 3rd party formats) Incompatible or Missing JAVASCRIPT interpreter September 19, 2018 © Robert Sabourin, 2001

Web Server Bugs! Application User Defective HTLM constructs! User interface design errors - unusable! Missing, incorrect or broken links Runtime errors in Active Server Page (ASP) scripts. Application User Missing, incorrect images, incorrectly identified objects Resources restricted - not enough memory, files system, capacity Runtime errors in JAVASCRIPT or VBSCRIPT code Logic, design or programming logic in code implementing user interface Missing or wrong version of web pages! Too many users for capacity of HTTP server. Denial of service attacks! Too much traffic for capacity of Web server. Pirates, Vandals, Hackers, Snoopers, Bad Guys redirections Incorrect exception handling in code implementing user interface. September 19, 2018 © Robert Sabourin, 2001

Web Server Bugs too! Application User HTTP Server Failure Error handling errors in communicating with other servers Java Server Failure Connection time out due to communication delays Application User O/S Failure - O/S Related problem moving between Unix and Windows Logic error in detecting and processing time out condition Third Party Component Failure Scripting error in CGI script Server Hardware failure Incorrect processing of cookie data in determining session status. Memory Leaks - resource management Access failure communicating with Date Server. Access failure communicating with Application Server Logic error in implementation of CGI scripts. September 19, 2018 © Robert Sabourin, 2001

Application Server Bugs! Missing, required functionality Application User Incorrectly implemented business logic Runtime errors in scripted code Incorrect version of code or objects Server, O/S or third party software failure - middle-ware failure Server hardware failure Cannot handle transaction volume September 19, 2018 © Robert Sabourin, 2001

Application Server Bugs too! Communication failure with Data server Application User Communication failure with Web Server Logic error in implementation of error handling code Memory leaks - resource usage Failure of third part components B2B link to other site failed Design errors in implementation of business logic September 19, 2018 © Robert Sabourin, 2001

Data Server Bugs! Application User Performance delays due to amount of data or system load Communication failure with Application server Application User Communication failure with Web Server Logic error in implementation of error handling code Runtime errors executing stored procedures Design errors in implementation of database logic Memory leaks - resource usage Incorrect schema referenced by different servers - configurations September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications Web Tier September 19, 2018 © Robert Sabourin, 2001

Basic Web Architecture Web Tier September 19, 2018 © Robert Sabourin, 2001

Web Tier Content Testing Content Testing Correctness Validation and Verification of content is a critical part of testing any Web or E-Commerce application Validate that all information presented is factual Verify that the correct information is presented September 19, 2018 © Robert Sabourin, 2001

Platform Coverage What is a Platform? In testing we consider a “Platform” to be the “Hardware” and “Software” environment that an application under test is run on Typically required to operate on several different platforms Platform can also include other concurrently running applications September 19, 2018 © Robert Sabourin, 2001

Browser Compatibility Testing Web application should: Operate on all popular platforms (Requirements based) Operate with all popular versions of plug-ins and third party components or viewers required Up to corporate policy how far to take this! www.browserwatch.com Pareto principal September 19, 2018 © Robert Sabourin, 2001

Browser Compatibility Testing Standards? Largely overridden by browsers Differences user input display processing Differences in Java virtual machines Differences in Plug-Ins September 19, 2018 © Robert Sabourin, 2001

Browser Compatibility Testing Which Browser? Microsoft Internet Explorer Netscape Navigator AOL Browser Lotus Notes Browser Web TV Browser Mosaic Opera September 19, 2018 © Robert Sabourin, 2001

Browser Compatibility Testing Which Version of Which Browser Battle of the browsers led to unprecedented number of incompatible versions of supposedly compatible products and technologies Beta releases used as if they were commercial gold releases Non-standard features implemented in different manners on each browser Patches and service packs September 19, 2018 © Robert Sabourin, 2001

Browser Compatibility Testing September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Which operating system (O/S)? Windows 2000, XP Windows 98 Original, 2nd Edition Windows 95 Original, Upgrade A, OSR2 B Window NT 4.0, SP3, SP4, SP5, SP6 Linux Solaris MAC OS September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Which Version of Co-dependent Software? Multiple versions possible for each object used or depended upon! Which Display Resolution? 640 x 480 800 x 600 1024 x 768 1280 x 1024 Custom Which System Font Size? Small Large Which Color Depth? Monochrome 4-bit 8-bit 16-bit 24-bit 32-bit September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Number of possible combinations are tumultuous Cannot possible try each combination even for a short period of time each Even if we restrict out testing to the most commonly used variations over the past two years or so! September 19, 2018 © Robert Sabourin, 2001

Platform Coverage We cannot possibly try all combinations Instead vary platform used under testing in a manner consistent with Technical Risk and Business Risk associated with each platform component define a grid which indicates which platform will be used at which time vary platforms with tests September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern For each relevant platform component a pattern is defined, in our example we care about the following three characteristics of the workstation: Pattern A - Browser Pattern B - Operating System Pattern C - Locale September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Two testing workstations are made available to testers and are prepared well in advance according to a predefined pattern, identified as W001 and W002 Four testing sessions are run per day in different time slots know as TS01, TS02, TS03 and TS04 September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Testing is organized into 17 different testing sessions identified as T001, T002, … , T017 80 sessions over 10 days of testing September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern A - Browser Technical and Market Risk Assessment September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern A - Browser Resulting Pattern - Uniform Random September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern B - Operating System Technical and Market Risk Assessment September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern B - Operating System Resulting Pattern - Uniform Random September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern C - Locale Technical and Market Risk Assessment September 19, 2018 © Robert Sabourin, 2001

Platform Coverage Pattern Example - Multi-Layered Pattern Pattern C - Locale Resulting Pattern - Uniform Random September 19, 2018 © Robert Sabourin, 2001

Web Testing Checklists Your team can develop a series of one page checklists as an aid to testing the user interface of your web and e-commerce applications All team members are trained in their use During assigned testing activities, whenever a checklist violation is observed, it should be reported as a bug September 19, 2018 © Robert Sabourin, 2001

Checklists Start with Checklists for: Usability Testing Checklist GUI Testing Checklist Application Style Consistency Checklist Design Mistakes Checklist Choose check list items from examples, standards, guidelines and relevant past experience September 19, 2018 © Robert Sabourin, 2001

Checklists For each checklist: One page per checklist Related, Rational, Relevant Train testing team Example of conformance Example of violation Same list used in design & development September 19, 2018 © Robert Sabourin, 2001

Link Testing Hyperlinks Broken Missing Wrong Point to nothing Tools can help find broken links Missing Link should be there but it is not Wrong Point to the wrong thing Take user to the wrong page September 19, 2018 © Robert Sabourin, 2001

Link Testing Links Are all links from a page valid? Are all internal links relative? Are all external links absolute? External links should be periodically verified as valid after the site is live. Can key pages be reached with minimal links, for example under three links? September 19, 2018 © Robert Sabourin, 2001

Link Testing Links Use site mapping tools to review the structure of the Web Application Any recursive links? Any bi-directional links? Are image maps correctly assigned links? Are frame-sets managed correctly? Can each page be accessed? September 19, 2018 © Robert Sabourin, 2001

Link Testing Example with eValid September 19, 2018 © Robert Sabourin, 2001

Link Testing Example with eValid September 19, 2018 © Robert Sabourin, 2001

Link Testing Links Other Can a search engine of web application find each page? Can you navigate to each page? Other Is Web Page Description correct and relevant when bookmarked? September 19, 2018 © Robert Sabourin, 2001

Link Testing Re-direction Are HTTP error codes managed by appropriate Web Application pages 404 - Not Found 400 - Bad Request 401 - Unauthorized 403 - Forbidden 500 - Internal error 502 - Temporary overload September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications Application Tier Testing September 19, 2018 © Robert Sabourin, 2001

Basic Web Architecture Application Tier Business Logic Web Application Functionality September 19, 2018 © Robert Sabourin, 2001

Functional Testing Identify Functions Prioritize Identify Test Cases Primary Contributing Prioritize Identify Test Cases Per function Explicit September 19, 2018 © Robert Sabourin, 2001

Functional Testing So what is a function! “Anything the software is supposed to do” Results in a display Changes data Effects the environment Basically two classifications Primary FAILURE RENDERS PRODUCT USELESS FOR TYPICAL USER Contributing PRODUCT COULD STILL PERFORM USEFUL FUNCTIONS FOR TYPICAL USER IF FAILURE September 19, 2018 © Robert Sabourin, 2001

Test Case Design Characteristics of Good Test Case Reasonable probability of catching a bug Can be repeated Not redundant Best of breed Neither too simple nor too complex Based on reality September 19, 2018 © Robert Sabourin, 2001

Test Case Design Test Case Characteristics Consider Input Consider Outcome Consider Initial State Consider End State Input/Outcome Input/Outcome September 19, 2018 © Robert Sabourin, 2001

Modeling Test Case Design All state transitions All paths through the system Rational input Unexpected input Can we get to all states? September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Consider each page of the site as a State Consider the users action as the input Consider the variable output as the outcome September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! Starting Page September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! Valid ID, Password September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! Mail Session Page September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! Invalid ID, Password September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! ID Error Page September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! Acknowledged Valid ID Mail Session Start ID Error Invalid ID Completed Session September 19, 2018 © Robert Sabourin, 2001

State Transition Model of Web Site Example mail.yahoo.ca! September 19, 2018 © Robert Sabourin, 2001

Test Case Design Equivalence Classes Boundary Values Sets of input events which are similar and exercise the application in the same way! Boundary Values Sets of input events on or very close to the boundary of accepted valid input Test cases should include a mix of valid and invalid input for all equivalence classes and boundary conditions September 19, 2018 © Robert Sabourin, 2001

Test Case Design Outcomes Positive Negative have at least one test case to generate each possible outcome Negative have at least one test case which does not generate each possible outcome September 19, 2018 © Robert Sabourin, 2001

Test Case Design Input Invalid inputs Pairs at least one test with each possible type of invalid input for each type of field on a dialogue use gray box methods to determine common types of fields Pairs at least one test for each possible pair of invalid inputs September 19, 2018 © Robert Sabourin, 2001

Test Case Design Error Conditions Error conditions at least one test for each possible error condition September 19, 2018 © Robert Sabourin, 2001

Test Case Design Input Text literal numbers broken HTML, XML one, “1”, “i”, “l”, “I” broken HTML, XML cut and paste control characters navigation to field! September 19, 2018 © Robert Sabourin, 2001

Real Data Real test data Logs from field are excellent source of real data which is a gold mine for testing since it defines the real usage scenarios from the field! Scramble to protect confidential information September 19, 2018 © Robert Sabourin, 2001

Wisdom Repeated testing Make sure test cases can be repeated allows for regression allows for data driven test automation Test descriptions should be explicit September 19, 2018 © Robert Sabourin, 2001

Scenario Based Testing Web Tier Testing Scenario Based Testing September 19, 2018 © Robert Sabourin, 2001

Modeling Story Boards E-Commerce user experience what can be done paths through the system September 19, 2018 © Robert Sabourin, 2001

Modeling User Experience Login Read message Check Out specials Buys some stuff Goes to affiliate September 19, 2018 © Robert Sabourin, 2001

Modeling User Experience Test Case Parameterize experience Walk through scenario from start to end Use pre selected input for each case Always run every test as if it were a user experience September 19, 2018 © Robert Sabourin, 2001

Data Parameterization Separate test procedure from test data Procedure Instructions to follow in order to run the test Navigation instructions Invariant part of a test September 19, 2018 © Robert Sabourin, 2001

Data Parameterization Test Data Any data typed into a form or field by the end user Variable part of a test Potential other data Any other field Any decision September 19, 2018 © Robert Sabourin, 2001

Data Parameterization Purchase a book from Amazon.com Procedure go to URL www.amazon.com wait for home page to load in search field type in ISBN number of book add book to shopping cart proceed to checkout log in with user name and password ship to this address place order September 19, 2018 © Robert Sabourin, 2001

Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization ISBN Number Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization User Name Data Password September 19, 2018 © Robert Sabourin, 2001

Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization September 19, 2018 © Robert Sabourin, 2001

Data Parameterization URL ISBN USER NAME PASSWORD September 19, 2018 © Robert Sabourin, 2001

Data Parameterization Advantages Separates test code from data Leverage well defined test procedure Same procedure can be used for many different purposes depending on data selected Critical first step in design of scenario based test automation scripts Forces decisions to be deliberate - not random or ad-hoc! September 19, 2018 © Robert Sabourin, 2001

Exploratory Testing September 19, 2018 © Robert Sabourin, 2001

Exploratory Testing An approach to “test design” that employs general-systems heuristics and specific-systems expertise to enable rapid development of test cases. Approach formalized by James Bach (www.satisfice.com) Used in General Functionality and Stability Test Procedure for Windows 2000 Application Certification September 19, 2018 © Robert Sabourin, 2001

Exploratory Testing Test cases Not known in advance Defined & executed “on the fly” while you learn about the product Testers need to “hone up” their skills in making maps! Consistent note taking style Practice We need to have a standard template Uniform way to capture details September 19, 2018 © Robert Sabourin, 2001

Exploratory Testing During test we must capture Function, options or sub-functions being explored Test cases attempted Comments, notes, images or attachments Hints, reminders and observations which may be useful to future testers Date, Platform, Build or Configuration under test Name of person running test Oracles, “strategy to assess correctness” Other relevant details September 19, 2018 © Robert Sabourin, 2001

Example Exploration Process Confirm Test Objective Ensure context known Ensure HW and SW OK All tools available Kick Off Chunk of 90 to 120 min Test, Plan, Discover Prepare Run Wrap up Collect all notes data Complete Review results with Test Lead Review Follow Up Reassess goals Piece together map September 19, 2018 © Robert Sabourin, 2001

Testing Web and E-Commerce Applications Data Tier Testing September 19, 2018 © Robert Sabourin, 2001

Basic Web Architecture Data Tier Application Various Repositories September 19, 2018 © Robert Sabourin, 2001

Data Operations Search for a record Insert a new record Delete an existing record Change an existing record September 19, 2018 © Robert Sabourin, 2001

Data Operations Search for a record Different Sizes of Database Small Typical Large September 19, 2018 © Robert Sabourin, 2001

Data Operations Search for a record DBMS Cache impact Random order Inverse order In order Worst case Best case Repeated September 19, 2018 © Robert Sabourin, 2001

Data Operations Search for a record Rules and syntax Data should not be found Data should be found Multiple selections found September 19, 2018 © Robert Sabourin, 2001

Data Operations Search for a record Multilingual database searches Many concurrent queries in different languages Watch out for search being correct Observe performance and timing! September 19, 2018 © Robert Sabourin, 2001

Data Operations Insert a new record Before first record After last record Random points in table Same data repeatedly Invalid data Valid data September 19, 2018 © Robert Sabourin, 2001

Data Operations Delete a record Insert / Delete First record Last record Random points in table Delete all data Delete data which is pointed to! Insert / Delete Sequence repeatedly and randomly September 19, 2018 © Robert Sabourin, 2001

Data Operations Change an existing record Build record progressively by a series of changes Change name, then change address, then change phone number ... Change to records being pointed to September 19, 2018 © Robert Sabourin, 2001

Test Lab DBMS Folk Wisdom Access Ensure that database use is Restricted accessed only by Test Team accessed by DBA when requested by Test Lead Privileges Ensure user privileges are consistent with typical usage environment execute tests as a NORMAL USER not as a ROOT USER! Tuning Ensure DBMS is configured with recommended or typical customer target parameters for all tunable characteristics such as CACHE size State Ensure you can reset DBMS content by a restore or copy function the Database to a known state September 19, 2018 © Robert Sabourin, 2001

Real World Test Data Use test data which is derived from or inspired by real world data Protect real names and identification by carefully scrambling data Ensure security of any commercial medical personal identification September 19, 2018 © Robert Sabourin, 2001

Self Checking Data Approach Test Data Set Each record has at least one field which can be used to confirm the integrity of the entire record Useful to see if a database or a record of a database has been corrupted as a result of insert, delete and change operations. September 19, 2018 © Robert Sabourin, 2001

Self Checking Data Approach Consistency Checking Rules based Valid postal/zip code? Valid ID? Checksums Computed from data Side by Side Reviews Does data match across applications? Visible fields dependent of state of application September 19, 2018 © Robert Sabourin, 2001

References “Software Testing Methods and Tools Course Notes”, AmiBug.com, 2001, Robert Sabourin. “Stress Testing Web Applications Workship Notes”, AmiBug.com, 2001, Robert Sabourin. “Testing Applications on the Web”, Wiley 2000, Hung Nguyen. “Testing Process”, MSDN, 2000, Biraj Rath, Raj Nath, Mukesh Agarwal, Jas Lamba www.satisfice.com “General Functionality and Stability Test Procedure for Windows 2000 Application Certification”, Microsoft, 2000, James Bach. www.riceconsulting.com www.useit.com www.csst-technologies.com September 19, 2018 © Robert Sabourin, 2001

References “Effective Methods for Software Testing”, Second Edition, Wiley 2000, William Perry. “The Engineering of Software”, Addison Wesley, 2001, Hamlet, Maybee. “Configuration Management, The Missing Link in Web Engineering”, Artech House, 2000, Susan Dart. “Lessons Learned in Software Testing”, Wiley, 2002, Kaner, Bach, Pettichord “Managing the Testing Process”, Microsoft Press, 1999, Rex Black “I am a Bug”, AmiBug.Com, 2000, Robert Sabourin September 19, 2018 © Robert Sabourin, 2001

Thank You Questions? September 19, 2018 © Robert Sabourin, 2001