1. Evaluation of Google Docs as a collaborative tool among UCI graduate students
Group 2
INF 231
12/2/2015

2. Primary Research Questions
What problems do small groups of college students (4-5 users) have while using Google Docs for collaborative writing?
How can we improve the functions and design associated with these problems?
Relevance
Collaborative writing is a core feature of Google Docs
Looking at a small user group (students) allows for a closer look at their behaviors and social interactions
May be useful for future developments with other related user groups
mention something about why looking at small groups is important for us -- feasible, allows for closer look at behaviors/social interactions, etc.

3. Overview of Study Methods
Goal: narrow down our focus to functions and UI elements that most commonly caused problems
Open-Ended Questionnaire
Focus Groups
Specificity of Information
Usability
Tests

4. Open-Ended Questionnaire
Distributed online using Qualtrics (N = 37)
Used to formulate specific research questions related to problematic functions and design elements for most common scenarios
Determine what collaborative writing scenarios (academic/non-academic) Google docs is used for among college students
List three most common scenarios (open ended)
How long and with how many people do they collaborate
Determine which functions and design elements users like and do not like
“What are the top 3 functions you like best about Google Docs?”
“What are the top 3 functions you least about Google Docs?”
Additional comments box for “Anything we missed?”

5. Open-Ended Questionnaire Preliminary Results
Collaborative Writing Scenarios
Cat 1 - Creating Reports / Written Documents
Cat 2 - Brainstorming
Cat 3 - Interview Prep
Cat 4 - Group Communication / Sharing
Cat 5 - Editing Process
Cat 6 - Event Planning
Cat 7 - Other

6. Open-Ended Questionnaire Preliminary Results
Average Number of Collaborators: 4.264

7. Open-Ended Questionnaire Preliminary Results
Disliked Features
Dislike 1 - Editing and Formatting
Dislike 2 - Sharing and Privacy
Dislike 3 - Portability (Integration with Google and other products)
Dislike 4 - Availability
Dislike 5 - User Interface Issues
Dislike 6 - Other

8. Open-Ended Questionnaire Preliminary Results
Liked Features
Like 1 - Editing and Formatting
Like 2 - Sharing and Privacy
Like 3 - Portability (Integration with Google and other products)
Like 4 - Availability
Like 5 - Concurrent Users
Like 6 - Revision History (Automatic Save)
Like 7 - Other

9. Open-Ended Questionnaire Preliminary Results
Users usually use Google Docs for academic purposes
The average number of collaborators on a Google Doc is about
The features in Google Docs that are commonly liked are also commonly disliked.
There is no discernable difference in the use of Google Docs for undergraduate and graduate students.

10. Focus Groups
Three focus groups were conducted (N = 12) to further investigate data from our questionnaire
Only UC Irvine graduate students were used due to lack of response from staff and undergraduate groups (approved by client)
Interviews were audio recorded so that direct quotes relevant to our research questions could be transcribed
Interviews were semi-structured around the following research questions:
R1. What is the process behind groups setting up and sharing Google Docs?
R2. What steps do groups take to creating a collaborative document?
R3. What issues do groups encounter when working simultaneously or asynchronously? In what types of situations do they prefer one over the other?
R4. Do people find the UI intuitive? If not, what issues do people have with the UI?
R5. What are common editing issues people run into when using Google Docs?
R6. Is the necessity of online connectivity a serious deterrent from using Google Docs?

11. Focus Groups Results (Part 1)
Users generally enjoy the collaborative workspace that Google Docs provides, however some felt pressured by writing in real time among peers (work offline and copy text over later)
All users had problems when multiple users typed on the same doc at the same time due to text being pushed down (open another word processor and copy text over or type in the same Doc on a distant page)
Users like the minimalistic style of Docs, but stated that they preferred more visible options in their word processors (e.g. sharing and editing)
Users did not like being notified by whenever comments were made involving them

12. Focus Groups Results (Part 2)
Users do basic text editing in Docs but commonly switch to Word for more extensive editing of tables, figures, and citations
We found a lack of knowledge of the Editing, Suggesting, and Viewing modes during the collaborative writing process, but interviewed users wanted to know more about these tools
Users were concerned with Doc security and were generally unfamiliar with how to modify security settings
Editing control among users could be improved by monarchical and democratic options

13. Usability Study: Tasks
Sharing and Privacy
Collaborative Writing
Formatting and Editing
Suggestion Mode and Comments
Total Features being tested: 4
Total Participants(Users): 10
Thanks to Dakuo and Ben for valuable comments in developing the tasks and analyses!

14. TASK 1: Sharing and Privacy
“Let's say that you and the test administrator were working on a new research idea.
You are about to start writing a paper about it.
Set up a Google Doc where you and the Test Administrator (Test Administrator’s address: can collaboratively start writing the paper.
The findings of your project are pretty ground breaking and as the Team Lead, it is your responsibility to ensure that the findings in the paper remain private until published.”

15. STEP1
STEP 2

16. TASK 1: Sharing and Privacy Results

17. TASK 1: Sharing and Privacy Results
Average Time to complete the task: 2 mins 20 secs

18. Task 2: Collaborative Writing
“...Assume you are telling a 12 year old about what your field of study is and try explaining a few basic concepts in about 3-4 lines...”
TEST ADMIN AREA
lorem ipsum...
USER AREA
I work on making computers talk to each other...

19. Task 2: Collaborative Writing

20. Task 2: Collaborative Writing Results
Average Time to complete the task: 3 mins 40 secs

21. Task 3: Formatting and Editing
Headings
Header, Footer, Page Number
Insert an Image
References

22. Task 3: Formatting and Editing
Average Time to complete the task: 12 mins 55 secs

23. Task 4 - Commenting and Suggestion Mode
“The test administrator will paste two interesting paragraphs about Google which he has already typed. Some grammatical mistakes might have creeped in while he was writing.
Your task for the first paragraph is to write suggestions about the grammatical corrections that need to be done.
For the second paragraph, add comments about grammatical correction which would help the author understand his/her mistake.”

24. STEP 1
STEP 2

25. Step I Alternative - I
Step I Alternative - II

26. Step I Alternative - III
Step II

27. Task 4 - Commenting and Suggestions results
Average Time to complete the task: 6 mins 48 secs

28. Mockups:
Designs were driven by interview responses and behaviors discovered from the usability tests
Four functions and their corresponding UI’s were redesigned:
Private Workspace (Sierra)
Sharing and Privacy Controls (Veronica)
Edit Mode UI (Mark)
Doc Name Popup (Abhijith)

29. Private Workspace Revisions
Users generally enjoy the collaborative workspace that Google Docs provides, however some felt pressured by writing in real time among peers
All users had problems when multiple users typed on the same doc at the same time due to text being pushed down

30. Current workflow

31. Current workflow

32. Current workflow
current
right click
menu

33. Proposed redesign

34. Proposed redesign
The private workspace stays put while I type! No one will see what is written here until I
press the submit button.
Submit
Copy
Cancel

35. Proposed redesign
The private workspace stays put while I type! No one will see what is written here until I
press the submit button. Then it will paste to the cursor.
Submit
Copy
Cancel

36. Sharing and Privacy Control Revisions
Users were concerned with Doc security and were generally unfamiliar with how to modify security settings
Editing control among users could be improved by monarchical and democratic options

37. Current Workflow

38. Current Workflow

39. Current Workflow

40. Current Workflow

41. Current Workflow

42. Proposed Redesign Share
Note: Yellow stars symbolize the symbol currently used by Google

43. Previous Menu Options

44. Redesign Menu Options (Owner)
Invite
Share Mode
Settings
Invite
Share Mode
Settings
Invite
Share Mode
Settings
Get Shareable Link
Link Sharing:
Default Mode
TjZPY5Hr0wEH5CQQ/edit?usp=sharing
Invite People:
Enter names or addresses…
Can Edit
Monarchy Mode
Who has access:
Democracy Mode
Person 1
Can Edit
Unanimous
All members required to approve suggestions
Person 2
Can Edit
Majority
Majority of members required to approve suggestions
Person 3
Can Edit
By Content
Members approve suggestions on their own content
Advanced Privilege Settings:
Person 4
Can Edit
Done
Done
Done

45. Redesign Menu Options (Non-Owner)
Get Shareable Link
Invite People:
Enter names or addresses…
Can Edit
Who has access:
Person 1
Can Edit
Person 2
Can Edit
Person 3
Can Edit
Person 4
Can Edit
Done

46. Edit Mode UI Revisions
Users did not like being notified by whenever comments were made involving them
We found a lack of knowledge of the Editing, Suggesting, and Viewing modes during the collaborative writing process, but interviewed users wanted to know more about these tools

47. Current Edit Mode UI

48. “Editing” Mode Dropdown (when clicked)

49. “Suggesting” Mode Selected
Turns green and bolded when selected

50. “Viewing” Mode Selected
Many toolbars become hidden
No color change when switched to this mode

51. Mode Selector shrinks to an icon when Docs window is small

52. Revised: “Editing” Mode Selected
The comment button has been moved next to notifications
The currently selected mode and options are now more obvious to the user

53. Revised: “Suggesting” Mode Selected

54. Revised: “Viewing” Mode Selected

55. Revised: Shrinking Icons
Edit modes remain visible and appear as buttons

56. “Comments” Button Interface
These are also buttons but do not look similar to the Comments button above
Confusion between “Comments” button and “Comment” button

57. Revised: “Notifications” Button Interface
All information and settings for doc comments are grouped under the notifications menu

58. Revised: “Notifications” Button Interface
Simple markup is useful when there are many comments to navigate through

59. Revised: “Notifications” Button Interface
This tab labeling makes finding and controlling notifications much clearer

60. Miscellaneous Editing Revisions
Users do basic text editing in Docs but commonly switch to Word for more extensive editing of tables, figures, and citations

61. Save before edit screen

62. Save before edit screen
This menu appears if the user starts to edit text but has not yet named the document
Better to ask the user to name the doc here, than when they go to share the doc. Otherwise, doc will automatically be saved as “Untitled Document”

63. Things to make more similar Word
Headers and footers
Citations
Image captions

64. Full Study of Revised Docs
The current study represents a small sample of graduate students at UCI
UCI had 30,757 students in 2014
A full study will require 589 students to ensure population responses are within a 4% confidence interval (p < .05)
Two groups: control (old design) and test (new design)
samples can be controlled for prior experience with Docs (normally distributed from no experience to highly experienced)
Examine data across groups by their level of Docs experience
Compare quantitative metrics (time to correct decision, mouse movement, etc.) directly between the control and test groups
Are people quicker and less confused with the new design?
Can decide this with statistics
Qualitative metrics can also be assessed, but should be coded to allow for statistical comparison (Likert, System Usability Scale)

65. TO DO: Team 1: Veronica, Mark, Sierra, Abhijith
Team 2: Saurebh, Urjit, Kasey, Zhe
Team 1:
Proof focus group writeup in final report
Team 2:
Finish questionnaire analysis and write up
Finish usability tests analysis and write up
Team 1 & 2:
Finish protocol writeup for full research study
Add images to report appendix