Olli Kulkki Markus Lappalainen Ville Lehtinen Reijo Lindroos Ilari Pulkkinen Helsinki University of Technology S Acceptability and Quality of Service

 Comparison of competitive services 1. Framework 2. Comparison  Improving acceptability for a service 1. Usability evolution process 2. Methods for process execution 3. Evolution continuity 4. Using measured data

Task: Collaborative Information Sharing Document writing Communication within group User Groups University student Office worker Services Google: Gmail + Google Docs MS Office: Outlook + Word

MS Office Purchaser is usually not same as user Different user needs addressed with different packages De Facto standard in office tools (and file types) Collaboration: Sending files and messages between stakeholders Google Free web based product family and personal user accounts Different services can be attached to user account Challenges Office products with extra features (web interface) Collaboration: Real time messaging and decument co- editing

Use Base Growth Use Base Growth Duplexity MS Office - Document formats become more common - First user defines document format - User base growth not directly improving service Google - Interactive features between users - Second-tier enhanced offerings - Service provider develops additional features as user base grows

MS OfficeGoogle Social acceptability Strong social pressure/infrastructure (Defacto environment) Large hype (cloud computing) Trying new things, talking about them Practical acceptability + Advanced platform + A lot of advanced users + Advanded features + Web based platform: + Flexibility + Collaborativity Usefulness+ Advanced features +Formatting + Parallel editing of documents ReliabilitySaving files to diskBrowser reliability Compatibility+Formats +Different versions + Works on top of any browser + Support for some document formats CostDifferent bundles available for different target audiences Advertisement funded (adsense) Usability+ New versions are focused to improve usability + Effective use requires training + Easy to learn + Basic functionality easy to find

Assessing and developing acceptability and quality for one of the previous applications

Usability Now Carry out Usability Measurements with current build Establish Baseline Target Usability Integrate User Needs and Baseline to define Target Usability Usability After Evolution Redo all Measurement with new build Compare results Has usability improved? Applied from Theofanos et al, A Practical Guide to the CIF: Usability Measurements, 2006

 Measure effectiveness, efficiency and satisfaction  Carry out participative usability tests  Focus on teamwork and document co-editing  Group tests with Think Aloud  Use representative tasks  Measure completeness, errors, time on task  Questionnaires to measure satisfaction

 How much we want (and how we measure)  Effectiveness  Efficiency  Satisfaction with intended users, tasks and context? Google Docs Users: Students doing course exercises together Tasks: Co-editing text, using source material etc. Context: Computer classrooms, home computers Google Docs Users: Students doing course exercises together Tasks: Co-editing text, using source material etc. Context: Computer classrooms, home computers

Inquiry Questionnaires Surveys Journaled sessions Inspection Benchmarking Heuristic evaluation Cognitive walkthrough Formal usability inspections Guideline checklists Perspective based evaluation Testing General concept (observation) Think-aloud Performance measurement Usability experiments Applied from the Usability method toolbox (

 Simplistic user interface  Two separate views for document management and text editing  Problems  Limited formatting tools  Access inconsistency with other Google apps  Tools for collaboration support are missing  Benefits  Easy to learn

Reverse quality Attractive quality Quantitative results Identify value enhancement Prototyping Feature deja vu Feature use case creation Simplified functionality Simulations Models Performance monitoring Passive measurement Technical statistics Usage statistics Automated data logging Statistical analysis Questionaires Collecting user needs for product development

 Record and analyze user behavior and preferences  Improve service quality and acceptability  Non-interactive, non-functional

Usage statistics Market share and valuation in ecosystem Technical statistics Shortening new feature time-to- market Performance management ROI Passive measurement Usage pattern recognition Log activity patterns Revenue management Revenue prediction New feature use cases Continuous innovation Serve user needs Feature deja vu Improve familiarity Reduce change resistance Simplified functionality Improved time to learn Identify value enhancement Attract new customers Quantitative results User profiling