1. Coeus Upgrade Testing at Princeton University Kyle Burkhardt Coeus Coordinator

2. Coeus at Princeton Started using Coeus in 1998 2002/3: Dept-by-dept rollout with 3.8 12/2008: 4.2.3 7/2010: 4.4.1 1/2011: 4.4.3

3. Usage Premium – Proposal Development, IP, award, subcontract, IRB – s2s – Proposal hierarchy Lite – Faculty approvals – Encouraging smaller depts to use Lite Proposal Development – Looking at IRB Lite module

4. Who uses Coeus at Princeton? Academic department users – data entry – ~80 power users Administrative users – data viewing – ~20 power users ORPA – Data entry + data viewing Faculty members – Approvals only

5. Testing process Read release notes Decide, are we going to upgrade or not? Set timeline for release to production

6. Testing process Plain vanilla release – New functionality Have kick-off meeting with testers DVL: Test everything New functionality Our customizations All existing functionality QA: test some things New functionality Our customizations Release to PRD, training, and “copy of production” DBs Vanilla Development DVL QA Production PRD

7. Testing time Depends on the release, experience, number of testers – 3.8 to 4.2.3: 18 months DVL; 2 months QA – 4.2.3 to 4.4.1: 6 months DVL; 2 weeks QA – 4.4.1 to 4.4.3: 3 months DVL; 3 days QA

8. DVL Testers Me & ORPA staff – Test as ORPA staff Proposal Development, IP, awards, subs, IRB, feeds from award to financial system Rolodex, sponsor, inbox, adding users, business rules – Test like dept users Proposal Development Premium & Lite – s2s – Proposal hierarchy

9. Test scripts Stored on SharePoint Giant multi-tabbed spreadsheet – Every piece of functionality used at Princeton with testing scenarios ORPA staff spreadsheets Record testing results – s2s apps: store “before” and “after” Add to scripts as I learn

10. Motivation Testing treats ( bribes )

11. Bug? Is it a real bug? – Can it be reproduced? – Is it user error? – Is it because it’s test data? If it’s real – Can we live with it? – Is it a showstopper? – Can OIT fix it? – Can Coeus Consortium fix it? – Put it in Jira – Maybe send plea to the list-serv

12. Bugs Master JIRA bug spreadsheet Tab on big spreadsheet Basis for bugs/quirks link on Princeton Coeus home pagebugs/quirks link Princeton Coeus home page

13. New functionality & behavior changes Tab on big spreadsheet Basis for differences guide

14. Challenges Keeping to the timeline – Help desk, training gets in the way Testers have jobs too!

15. Suggestions Don’t customize! More testers?

16. Questions?