1. Quality Assurance in Sakai Past, Present and Future
Pete Peterson, Sakai Foundation
Megan May, Indiana University

2. 10th Sakai Conference - Boston, MA, U.S.A.
Agenda
What is QA?
Past
Present
Future
Discussion/Questions?
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

3. 10th Sakai Conference - Boston, MA, U.S.A.
What is QA/QC?
Quality Assurance (QA) is the process of verifying or determining whether products or services meet or exceed customer expectations. QA considers all aspects of design, development, production, and service. When implemented correctly, QA provides a continuous and consistent improvement of processes within a product lifecycle.
Quality Control (QC), a subset of QA, is employed to ensure a certain level of quality in a product or service. The basic goal of QC is to ensure that the products, services, or processes provided meet specific requirements.
Often, Quality Control is confused with Quality Assurance. Though the two are very similar, there are some basic differences. QC is concerned with the product, while QA is process–oriented. Simply put, QA ensures that a product or service is developed and implemented in the right way; while QC evaluates whether or not the end result is satisfactory.
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

4. 10th Sakai Conference - Boston, MA, U.S.A.
What is QA/QC?
There are three essential questions that need to be answered and tested long before a product is placed into production, and QA/QC activities focus on answering these questions:
Does it do what we expect it to do?
Does it do what the end-user expects it to do?
Does it do “a” and “b” correctly and consistently?
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

5. 10th Sakai Conference - Boston, MA, U.S.A.
Key Points of QA/QC
Quality is not free: There are costs associated with implementing these additional steps into your projects and development processes—time, money, resources, etc. These costs, however, are small in comparison to the benefits of having increased end-user satisfaction, reduced support requirements, and functionally accepted applications and systems.
Repeatable: The QA/QC process should be repeatable and flexible, so it can be easily applied to other projects as they arise.
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

6. 10th Sakai Conference - Boston, MA, U.S.A.
Key Points of QA/QC
Comprehensive: QA/QC should be included in the project from the beginning to the end. The goal of this participation is to examine the project from all angles. This not only allows the QA/QC group to create better tests, but also to provide feedback and a global perspective to the developer community.
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

7. 10th Sakai Conference - Boston, MA, U.S.A.
Key Points of QA/QC
Ongoing: QA/QC doesn’t stop once a project is complete; in fact this is where much of the QA is applied. Examining pitfalls and past mistakes can improve design, development and testing, as can documenting procedural successes and areas that need improvement.
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

8. 10th Sakai Conference - Boston, MA, U.S.A.
Key Points of QA/QC
Autonomous: It is essential for the QA/QC group to have a high level of autonomy. Often the QA/QC process is performed in-house by the developers or project managers themselves. Although well-intentioned, this approach is almost always ill-advised and precarious. It is difficult to remove oneself, step back and see things from the diverse perspectives needed for an effective QA/QC analysis.
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

9. 10th Sakai Conference - Boston, MA, U.S.A.
Past
“Those who don't know the [past] are destined to repeat it.”
Edmund Burke
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

10. Sakai’s Testing & Release Management Process
Primarily At end of the cycle.
Mostly manual process
great emphasis on participation of RM. A lot of the release/product management is bleeds into the QA process.

11. Formulation of QA Network
Amsterdam Boston
Cambridge Cape Town
Columbia Indiana
rSmart Charles Sturt Georgia Tech
3 servers at MIT (mysql, oracle, hsdb) There are QA servers hosted at variety of institutions.
Deployment testing on wider variety of platforms
Tested conversation scripts by retaining data.
Recent additions of Nagoya, Marist & UMich

12. Past QA Workgroup Participation
2.2.1
23 People
10 Institutions
4 Countries
2.3.0
44 People
17 Institutions
5 Countries
2.4.0
93 People
37 Institutions
11 Countries
2.5.0
56 People
17 Institutions
4 Countries
- The QA WG is also a community of volunteers.
There is a core group in the WG testing
- During these releases high degree of emphasis placed on bug verification (~85%)

13. 10th Sakai Conference - Boston, MA, U.S.A.
Redefining processes
Defined milestones
No new features in maintenance releases
Better oversight of merges into maintenance release branches
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

14. Other Accomplishments
Expansion of test script library
Raised awareness of the importance of unit and performance testing throughout community
Availability of Static Code review
Integration of Accessibility WG activities in general QA cycle
Inclusion of 3rd party licensing review in QA process
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

15. 10th Sakai Conference - Boston, MA, U.S.A.
Sakai QA - Present
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

16. 10th Sakai Conference - Boston, MA, U.S.A.
Sakai QA - Present
Finalizing Sakai 2.6 release
Finalizing Sakai 2.5 release
So everything is done right?
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

17. 10th Sakai Conference - Boston, MA, U.S.A.
Sakai QA - Present
Sakai and coming soon to a QA server near you
Testing for maintenance branch merges
Updates of test plan
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

18. 10th Sakai Conference - Boston, MA, U.S.A.
Sakai QA - Present
The Links Slide QA
Release Management
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

19. 10th Sakai Conference - Boston, MA, U.S.A.
Resources K2
Sakai 3
Sakai 2.5 K1
Sakai 2.6
Sakai 2.7
July 2009
10th Sakai Conference - Boston, MA, U.S.A.

20. 10th Sakai Conference - Boston, MA, U.S.A.
Future
Automation
Inclusion of test cases in every jira
Continuous testing
July 2009
10th Sakai Conference - Boston, MA, U.S.A.