“Crowdsourcing” a Textbook: 120 Student Authors Writing on a Wiki Edward F. Gehringer Department of Computer Science Dept. of Electrical & Computer Engineering North Carolina State University Supported by the National Science Foundation Feedback: http://tinyurl.com/expertiza-form Web site: http://tinyurl.com/expertiza-site

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Introduction Wikis are one of the premier collaborative spaces on the Web. Editing in place No need to “disseminate” to others Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Why wiki textbooks? Important for students to “take ownership” of learning Need to evaluate different points of view Compatible with constructivism No scientific evaluation of textbook effectiveness No comparative studies Even buying textbook doesn’t help … Contrast that with wiki textbooks, where research exists Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Why wiki textbooks, cont. “Writing across the curriculum” Students write for an audience of their peers Feedback helps them improve Allow direct dissemination without publisher Combat high textbook prices Important to learn to “find truth” on the Web Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our courses The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

The administrative burden Chapters need to be spaced out over the semester Precedence relationships must be obeyed Students are usually allowed to choose what they write on Electronic signup gives everyone an equal chance Multiple deadlines needed for each chapter Choose topic, submit, review, resubmit, etc. Students must be reminded … “Handles” for the wiki Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our courses The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Spring course CSC/ECE 506, Architecture of Parallel Computers Taken by all Computer Engineering students Optional for Computer Science students Small class … 15 students New textbook Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

The textbook supplement Good book, but there is room to augment its coverage … covers theory and design fundamentals, a bit thin in showing how current machines realize them Two goals Cover topics not covered in text. Apply concepts to current parallel machines Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Fall course CSC/ECE 517, Object-Oriented Design 4 textbooks for different topics None cover all topics of interest Very broad range of material Large course 109 students finished the course Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Signing up Every two weeks, Depending on size of course students sign up for chapters Depending on size of course There are more or fewer topics 2 to 10 More or fewer authors allowed to sign up for each topic 2 or 3 Students encouraged to work in pairs Exercise 1: After this, have them log in & sign up for a topic. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 1: Sign up Go to http://expertiza.ncsu.edu, log in with “eli#” and password “eli”, accept terms of use, click on “ELI demo”, & then “Signup sheet”. Choose one of the topics. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Writing on the wiki Same user-IDs on the wiki as on Expertiza … unless handles are used! We tell them how to name their pages. ELI_demo/username Exercise 2: Write on the wiki. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 2: Create a Wiki Page Go to http://tinyurl.com/ELI-demo You will be redirected to this page: Log in with “eli#” and password “eli”. Return to ELI_demo, click on your user # and start editing. Save your page. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 2 (cont.): Submit your page Go back to http://expertiza.ncsu.edu. Select “Others’ work”. Click on the topic you want to review. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Our first assignment writeup Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Which kind of feedback do you use? a. Feedback by instructor b. Feedback by TA c. Peer feedback, not anonymous (e.g., face to face) d. Anonymous peer feedback e. Feeback by teammates Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Peer review Submissions are peer-reviewed through our Expertiza system. ≈ 5 reviewers give feedback to each team. Teams improve work based on feedback Reviewers suggest a grade. Instructor/TAs review student recommendations, assign final grades. Exercise 3: Review others’ work. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 3: Review a Wiki Page After you save the page you were editing, go back to http://expertiza.ncsu.edu. Click on “Others’ work”, then on “Begin” for one of the reviews. Assign scores and give text comments. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Initial experience Students were not clear about the purpose … Substantially duplicated the textbook Did not concentrate on recent developments As a result, we began pre-reviewing the student submissions. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

More hurdles Our initial feedback served to tell students what they needed to re-do. Students had just gone through the first few search hits. They sometimes hit on info covered in next chapter of text. So … we met weekly to discuss upcoming chapters made list of material to be covered Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Later assignment writeup Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Later assignment writeup, cont. Exercise 2: Pause after this slide to have them write on the wiki. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Differences in a large class 3 teams/topic instead of 2 “Too many” topics needed Students suggest topics; instructor approves No time to read all submissions … Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Progress? Students’ second wiki chapters received almost the same average grade (82.7%) as the first (82.8%). But reviews were more demanding. Perceived quality of contributions is better. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Survey results — contributions Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Survey results — process Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Survey results — software Only the first question above received an unfavorable average response. Since then, Expertiza has been modified to send e-mail reminders. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Working with student authors Discouraging plagiarism Iterative improvement Organization Permission Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Lessons learned 1 Signup sheet is very useful The only reliable way to manage topics in a large class. Tells reviewer if author wrote on correct topic Handles are not always a win Students would change handles, then forget and use their old handle.  Expertiza wouldn’t find their work Simpler just to turn off wiki “history.” Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Lessons learned 2 Students want to submit work explicitly Confused by concept of Expertiza “finding” work on wiki Review system sometimes confused about which assignment submission is for. Students need e-mail reminders to stay on track Otherwise, there are too many deadlines to remember This may contribute to “too much rating for this course.”. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Practical ideas Topic suggestions Social bookmarking Start slowly—wiki only … Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Outline Introduction: Why wiki textbooks? The administrative burden Our course The process Survey results Working with student authors The software and the process Summary Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Summary A wiki is an excellent medium for collaborative writing. With appropriate software support, a class can engage collaboratively in a large project. Students in a parallel-architecture course made significant contributions to their textbook. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 1: Sign up Go to http://expertiza.ncsu.edu, log in with “eli#” and password “eli”, accept terms of use, click on “ELI demo”, & then “Signup sheet”. Choose one of the topics. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 2: Create a Wiki Page Go to http://tinyurl.com/ELI-demo You will be redirected to this page: Log in with “eli#” and password “eli”. Return to ELI_demo, click on your user # and start editing. Save your page. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 2 (cont.): Submit your page Go back to http://expertiza.ncsu.edu. Select “Others’ work”. Click on the topic you want to review. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form

Exercise 3: Review a Wiki Page After you save the page you were editing, go back to http://expertiza.ncsu.edu. Click on “Others’ work”, then on “Begin” for one of the reviews. Assign scores and give text comments. Gehringer, “Crowdsourcing” a Textbook Feedback: http://tinyurl.com/expertiza-form