Testing - an Overview September 10,

What is it, Why do it? Testing is a set of activities aimed at validating that an attribute or capability of a program or system is able to meet its documented requirements Testing is done to detect failures so that defects may be uncovered and corrected 2

Common Types of Tests Type of Test DescriptionApplicable when… Unit Each unit (basic component) of the software is tested to verify that the detailed design for the unit has been correctly implemented a module or piece of functionality has been developed Integration Exposes defects in the interfaces and interaction between integrated components (modules) during and after the development of a new module System Tests a completely integrated system to verify that it meets its requirementsthe system development nears completion (project or phase) System Integration Verifies that a system is integrated to other systems defined in the system requirementsthe system is integrated with other systems or data sources Usability Verifies that the user interface is easy to use and understanduser interfaces are being developed Security Testing which confirms that the program can restrict access to authorized personnel and that the authorized personnel can access the functions available to their security level user roles are defined and being developed in the system Acceptance Can be conducted by the end-user, customer, or client to validate whether or not to accept the product. Acceptance testing may be performed as part of the hand-off process between any two phases of development development is done and the system testing has been successfully completed Regression After modifying software, either for a change in functionality or to fix defects, a regression test re-runs previously passing tests on the modified software to ensure that the modifications haven't unintentionally caused a regression of previous functionality modifications are made to an existing system Load / Stress / Performance Assures that the system performs with expected high volumes Requires the simulation of multiple users accessing the program’s services concurrently (relevant for multi-user systems) or large amounts of data high volumes (number of users / amount of data) are expected Parallel Old systems and new system are run in parallel and the results compared to detect unplanned differences a new system is replacing an old one with same functionality Recovery Assures that the system can be returned to an operational status after a failureevents that may affect an operational system need to be tested 3

Sample Testing Cycle Requirements analysis: Testing should begin in the requirements phase of the life cycle. During the design phase, testers work with developers in determining what aspects of a design are testable and with what parameters those test work Test Planning: Test strategy, test plan, test environment (testbed) creation. A lot of activities will be carried out during testing so a plan needs to be developed Test Development: Test procedures, test scenarios, test cases, test scripts to use during testing Test Execution: Testers execute the software based on the plans and tests and report any errors to the development team Test Results Analysis or Defect Analysis: Is done by the development team and may include the client to decide what defects should be treated, fixed, rejected or deferred Retesting the resolved defects: Once a defect has been dealt with by the development team, it is retested by the testing team Regression Testing: Retesting a previously tested program following modification to ensure that faults have not been introduced or uncovered as a result of the changes made Test Reporting: Once testing is completed, final reports on testing results are reviewed to determine whether or not the software tested is ready for release 4

Requirements Encryption Requirements Provides Full Disk Encryption (FDE) C Encryption and decryption of data can be configurable to be transparent to user (i.e. on the fly) or requires user intervention C Products will use only FIPS compliant algorithms for encryption, hashing, and signing C Can encrypt data on removable storage media/devices (hardware agnostic) D Ability to extract data from an encrypted source to an unencrytped destination C Allows encryption algorithm selection by an domain administrator D Provides FDE and EFS under a single product management console I Product uses an approved random number generator specified in FIPS Annex C for key generation I Product has a negligible effect on user productivity (boot up, file load and power down) C C = Critical D = Desired I = Important 5

Test Case Sample TEST CASE # (from requirements document) # 7 Provides Full Disk Encryption (FDE) SCENARIOThe contents on a supposedly encrypted hard drive will be examined using computer forensic software before and after encryption. Testing that will take place. A)Examine the laptop hard drive prior to encryption to confirm forensic software can read the contents. B)Encrypt the hard drive using the vendor’s software C)Examine the laptop hard drive after encryption and confirm forensic software cannot read the contents. D)Boot the laptop with the encrypted hard drive to confirm it is functional. ACCEPTANCE CRITERIA:Should the software not be able to read the contents of the drive and yet the laptop still function normally, the assumption is that the drive is encrypted using the vendor defined algorithm. SETUP Data – Identify any data that needs to be set up for this test. No data but workstation must be loaded with computer forensic software. Transactional Steps No.STEPSEXPECTED RESULTSTESTER INITIAL PASS/ FAIL ACTUAL RESULTS 1Examine hard drive with computer forensic software prior to encryption. The software will recognize all of the files on the hard drive and be able to display the file contents. DMSPass 2Examine hard drive with computer forensic software after encryption. The software will not recognize any file on the hard drive and hard drive data will appear to be random data. DMSPass. The forensic software considered all of the drive to be slack space. 3Perform regular boot of laptop with hard disk encrypted. Laptop will boot and load operating system from encrypted hard drive. DMSPass 6

Test Plan Document Overview: Overview of the focus of this testing Test Cases: List all the test cases that will be performed, their purpose and function, and who will be responsible for performing them Test Data: Indicate where the test data is coming from and who is responsible for providing it. How will it be created and accuracy ensured? Acceptance Criteria: Define the testing approval process. Identify the specific criteria that will be involved for acceptance of each test. Include severity definitions, pass/fail definitions Resolution Log: Describe the procedure for tracking problems. What are the items that will be included in the log? Special Test Issues: Describe any special or unique testing issues that should be considered in this system implementation Testing Schedule: Define the specific schedule for performing the testing. When will each test case be performed, and what are its prerequisites and dependencies? Environment: Identify the environment the test activities will occur in Test Results: Indicate how test results will be reported, documented or summarized 7