1. “Crowdsourcing” a Textbook: 120 Student Authors Writing on a Wiki
Edward F. Gehringer
Department of Computer Science
Dept. of Electrical & Computer Engineering
North Carolina State University
Supported by the National Science Foundation
Feedback: Web site:

2. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

3. Introduction
Wikis are one of the premier collaborative spaces on the Web.
Editing in place
No need to “disseminate” to others
Gehringer, “Crowdsourcing” a Textbook Feedback:

4. Why wiki textbooks?
Important for students to “take ownership” of learning
Need to evaluate different points of view
Compatible with constructivism
No scientific evaluation of textbook effectiveness
No comparative studies
Even buying textbook doesn’t help …
Contrast that with wiki textbooks, where research exists
Gehringer, “Crowdsourcing” a Textbook Feedback:

5. Why wiki textbooks, cont.
“Writing across the curriculum”
Students write for an audience of their peers
Feedback helps them improve
Allow direct dissemination without publisher
Combat high textbook prices
Important to learn to “find truth” on the Web
Gehringer, “Crowdsourcing” a Textbook Feedback:

6. Outline Introduction: Why wiki textbooks? The administrative burden
Our courses
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

7. The administrative burden
Chapters need to be spaced out over the semester
Precedence relationships must be obeyed
Students are usually allowed to choose what they write on
Electronic signup gives everyone an equal chance
Multiple deadlines needed for each chapter
Choose topic, submit, review, resubmit, etc.
Students must be reminded …
“Handles” for the wiki
Gehringer, “Crowdsourcing” a Textbook Feedback:

8. Outline Introduction: Why wiki textbooks? The administrative burden
Our courses
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

9. Spring course CSC/ECE 506, Architecture of Parallel Computers
Taken by all Computer Engineering students
Optional for Computer Science students
Small class …
15 students
New textbook
Gehringer, “Crowdsourcing” a Textbook Feedback:

10. The textbook supplement
Good book, but there is room to augment its coverage …
covers theory and design fundamentals,
a bit thin in showing how current machines realize them
Two goals
Cover topics not covered in text.
Apply concepts to current parallel machines
Gehringer, “Crowdsourcing” a Textbook Feedback:

11. Fall course CSC/ECE 517, Object-Oriented Design
4 textbooks for different topics
None cover all topics of interest
Very broad range of material
Large course
109 students finished the course
Gehringer, “Crowdsourcing” a Textbook Feedback:

12. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

13. Signing up Every two weeks, Depending on size of course
students sign up for chapters
Depending on size of course
There are more or fewer topics
2 to 10
More or fewer authors allowed to sign up for each topic
2 or 3
Students encouraged to work in pairs
Exercise 1: After this, have them log in & sign up for a topic.
Gehringer, “Crowdsourcing” a Textbook Feedback:

14. Exercise 1: Sign up Go to http://expertiza.ncsu.edu,
log in with “eli#” and password “eli”,
accept terms of use,
click on “ELI demo”,
& then “Signup sheet”.
Choose one of the topics.
Gehringer, “Crowdsourcing” a Textbook Feedback:

15. Writing on the wiki Same user-IDs on the wiki as on Expertiza
… unless handles are used!
We tell them how to name their pages.
ELI_demo/username
Exercise 2: Write on the wiki.
Gehringer, “Crowdsourcing” a Textbook Feedback:

16. Exercise 2: Create a Wiki Page
Go to
You will be redirected to this page:
Log in with “eli#” and password “eli”.
Return to ELI_demo, click on your user # and start editing.
Save your page.
Gehringer, “Crowdsourcing” a Textbook Feedback:

17. Exercise 2 (cont.): Submit your page
Go back to
Select “Others’ work”.
Click on the topic you want to review.
Gehringer, “Crowdsourcing” a Textbook Feedback:

18. Our first assignment writeup
Gehringer, “Crowdsourcing” a Textbook Feedback:

19. Which kind of feedback do you use?
a. Feedback by instructor b. Feedback by TA c. Peer feedback, not anonymous (e.g., face to face) d. Anonymous peer feedback e. Feeback by teammates
Gehringer, “Crowdsourcing” a Textbook Feedback:

20. Peer review
Submissions are peer-reviewed through our Expertiza system.
≈ 5 reviewers give feedback to each team.
Teams improve work based on feedback
Reviewers suggest a grade.
Instructor/TAs review student recommendations, assign final grades.
Exercise 3: Review others’ work.
Gehringer, “Crowdsourcing” a Textbook Feedback:

21. Exercise 3: Review a Wiki Page
After you save the page you were editing, go back to
Click on “Others’ work”, then on “Begin” for one of the reviews.
Assign scores and give text comments.
Gehringer, “Crowdsourcing” a Textbook Feedback:

22. Initial experience Students were not clear about the purpose …
Substantially duplicated the textbook
Did not concentrate on recent developments
As a result, we began pre-reviewing the student submissions.
Gehringer, “Crowdsourcing” a Textbook Feedback:

23. More hurdles
Our initial feedback served to tell students what they needed to re-do.
Students had just gone through the first few search hits.
They sometimes hit on info covered in next chapter of text.
So …
we met weekly to discuss upcoming chapters
made list of material to be covered
Gehringer, “Crowdsourcing” a Textbook Feedback:

24. Later assignment writeup
Gehringer, “Crowdsourcing” a Textbook Feedback:

25. Later assignment writeup, cont.
Exercise 2: Pause after this slide to have them write on the wiki.
Gehringer, “Crowdsourcing” a Textbook Feedback:

26. Differences in a large class
3 teams/topic instead of 2
“Too many” topics needed
Students suggest topics; instructor approves
No time to read all submissions …
Gehringer, “Crowdsourcing” a Textbook Feedback:

27. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

28. Progress?
Students’ second wiki chapters received almost the same average grade (82.7%) as the first (82.8%).
But reviews were more demanding.
Perceived quality of contributions is better.
Gehringer, “Crowdsourcing” a Textbook Feedback:

29. Survey results — contributions
Gehringer, “Crowdsourcing” a Textbook Feedback:

30. Survey results — process
Gehringer, “Crowdsourcing” a Textbook Feedback:

31. Survey results — software
Only the first question above received an unfavorable average response.
Since then, Expertiza has been modified to send reminders.
Gehringer, “Crowdsourcing” a Textbook Feedback:

32. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

33. Working with student authors
Discouraging plagiarism
Iterative improvement
Organization
Permission
Gehringer, “Crowdsourcing” a Textbook Feedback:

34. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

35. Lessons learned 1 Signup sheet is very useful
The only reliable way to manage topics in a large class.
Tells reviewer if author wrote on correct topic
Handles are not always a win
Students would change handles,
then forget and use their old handle.
 Expertiza wouldn’t find their work
Simpler just to turn off wiki “history.”
Gehringer, “Crowdsourcing” a Textbook Feedback:

36. Lessons learned 2 Students want to submit work explicitly
Confused by concept of Expertiza “finding” work on wiki
Review system sometimes confused about which assignment submission is for.
Students need reminders to stay on track
Otherwise, there are too many deadlines to remember
This may contribute to “too much rating for this course.”.
Gehringer, “Crowdsourcing” a Textbook Feedback:

37. Practical ideas Topic suggestions Social bookmarking
Start slowly—wiki only …
Gehringer, “Crowdsourcing” a Textbook Feedback:

38. Outline Introduction: Why wiki textbooks? The administrative burden
Our course
The process
Survey results
Working with student authors
The software and the process
Summary
Gehringer, “Crowdsourcing” a Textbook Feedback:

39. Summary A wiki is an excellent medium for collaborative writing.
With appropriate software support, a class can engage collaboratively in a large project.
Students in a parallel-architecture course made significant contributions to their textbook.
Gehringer, “Crowdsourcing” a Textbook Feedback:

40. Exercise 1: Sign up Go to http://expertiza.ncsu.edu,
log in with “eli#” and password “eli”,
accept terms of use,
click on “ELI demo”,
& then “Signup sheet”.
Choose one of the topics.
Gehringer, “Crowdsourcing” a Textbook Feedback:

41. Exercise 2: Create a Wiki Page
Go to
You will be redirected to this page:
Log in with “eli#” and password “eli”.
Return to ELI_demo, click on your user # and start editing.
Save your page.
Gehringer, “Crowdsourcing” a Textbook Feedback:

42. Exercise 2 (cont.): Submit your page
Go back to
Select “Others’ work”.
Click on the topic you want to review.
Gehringer, “Crowdsourcing” a Textbook Feedback:

43. Exercise 3: Review a Wiki Page
After you save the page you were editing, go back to
Click on “Others’ work”, then on “Begin” for one of the reviews.
Assign scores and give text comments.
Gehringer, “Crowdsourcing” a Textbook Feedback: