1. “Kuality” Assurance What does that look like? Scott Heise Indiana University KFS - Quality Assurance Manager Paul Sandoval University of Arizona KRA – Lead Testing Coordinator Kenton Hensley Cornell University KRA – Lead Business Analyst

2. Scott Heise Indiana University Kuali Financial System (KFS) Quality Assurance Manager

3. “Kuality” Assurance

4. Automated Unit Testing

5. Manual Testing

6. Testing

7. Code Review

8. Accessibility Review

9. License Review

10. Quality Assurance Framework

11. License Review

12. Licensing Software distribution licensing –Open Source Initiative (OSI) Educational Community License 1.0 (ECL) Original code –Individual Contributor License Agreements (CLAs) –Corporate Contributor License Agreements (CCLAs) –© Labeling the code Third-Party code –ECL-Compatible License –Follow license requirements –Acknowledgments –License folder

13. Usability – Before Coding Indiana UXG – Usability Group –Developed Standards/Guidelines for the UI –Initial Mocks reviewed by Kuali Financial Functional Council Decisions made: –Editable data entry lines –Page Level Help – no field level help –Usability sessions with users from each school via Breeze

14. Usability – During/After Marked as Usability Reviewed and Prioritized by Usability Prioritization Committee Usability Dev team formed during QA Usability issues that affect infrastructure, reviewed and time allocated during next release, e.g. – error handling framework

15. Accessibility Review

16. Accessibility Guidelines Test by sampling “Issues” approach to resolution

17. Paul Sandoval University of Arizona Kuali Research Administration System (KRA) Lead Testing Coordinator

18. KFS Testing

20. Testing Structure Testing Teams –One per module –Module Testing Coordinator Write/assign scenarios to testers Train testers on Kuali, Confluence, JIRA Review Bugs: Duplicate? Training? Usability? –Testers from all schools – 15 plus/module KFS Testing Coordinator –Monitors issues application wide –Assists Module TCs

21. Process eDoc/process ready for testing Scenario created/assigned – JIRA Bugs reported in JIRA Issues fixed marked as resolved Testing environment released weekly Release notes created - Confluence Testers test close/reopen Rinse and Repeat!

22. Tools - JIRA

23. Tools - Confluence

24. Lessons Learned Face to Face sessions Never too many testers Training for testers Pre-Testing to ensure a fairly stable environment Weekly meetings with testers Testing is the BEST way to learn!

25. KRA Testing and Usability

26. Process Functional Specs (SMEs) Code (Developers) Testing (Developers, Lead Testing Coordinator, Module Testing Coordinator, SMEs, Testers – includes Coeus consortium members)

27. Testing Module Testing Coordinators –One per module –Write/assign scenarios to testers from each school in JIRA –Train testers Testers test – report bugs in JIRA Module Testing Coordinators review –Bug, Duplicate, Training opp, Usability

28. Bugs Reported in JIRA Fixed by Developers Resolved with next Build Confluence used to help testers keep track of resolved bugs ready to be tested Testers Test – Close/Reopen

29. Lessons Learned One person was designated to write Spec Docs in the SME group in KFS. In KRA, everyone is taking an active role in writing Spec Docs. This promotes ownership in the process!! SME Members are going to take a more active role in testing. Face to Face testing – Training

30. KRA Quality, Timelines and Synergy Can these work together?

31. QA is Continuous! QA exists throughout the entire software development life cycle! –Planning and Communication Tools Microsoft Project, Confluence and Jira, Breeze, Skype, Face-to-face meetings –Analysis Functional and Technical –Licensing –Coding Code Reviews, Development Standards

32. QA is Continuous! –Testing Unit Testing, Regression Testing, Integration Testing –Usability and Accessibility HTML Mock Development, Focus Groups, Design Critiques, W3C Priorities, Accessibility Testing –Documentation Functional and Technical –Builds and Releases

33. Timelines

34. QA impacts Timelines Project planning is continuous along with QA Coeus represents the base functionality that will be delivered in KRA Enhancements are reviewed, estimated and submitted for functional approval. If accepted, the project plan is adjusted. Functional specifications are standard SME groups follow similar processes when interacting with SME group members and the development team A user interface group monitors HTML mock development and usability/accessibility issues

35. QA impacts Timelines Focus groups and design critiques are conducted with Faculty and staff roles across multiple schools HTML mocks and jsps are validated for W3C Priority 1 and 2 issues and corrected Code reviews are standard and code is reviewed weekly Unit testing is standard and a continuous test process runs daily Multiple test environments exist to ensure that automated and manual testing is performed Documentation is standard – functional specifications, technical gap documents, integration documents, user and help documents

36. Lessons Learned Developers should test the web application on a regular basis Code should be reviewed as functionality is developed and a final code review should be required for completion Unit Testing guidelines should be set early and followed Functional testers should test functionality as early as possible Project team members must embrace QA processes Coeus participation is essential! Coeus documentation is essential

37. “Kuality” Conclusions QA exists throughout the software development life cycle While processes may vary between projects, the goal is to ensure that the software is built in a timely manner and meets specified functional requirements Reviewing and making changes based upon lessons learned can improve the project

38. Questions?

39. Contacts Please feel free to contact the following individuals about the KFS, KRA or Coeus projects: Jim Thomas, Project Manager Indiana University – jthomas@indiana.edu Andy Slusar, Senior Project Manager Cornell University – as833@cornell.edu