1. Improving the Classroom Experience
Hope Casey-Allen, Amy Liu, Yash Tambawala

2. Our team

3. Initial POVs

4. We met Wen, a PhD student at Berkeley
We met Wen, a PhD student at Berkeley. We were amazed to realize he sometimes felt more intimidated about participating in small classrooms than large classrooms. It would be game changing to build trust and a safe space in small classrooms.
We met Prof. Keith Schwartz, who teaches CS 103 at Stanford. We were amazed to realize that he often would go with his gut feeling rather than being analytical about the feedback for a course he taught. It would be game changing to build confidence among both teachers and students that learning objectives are being met.

5. Next step: more needfinding with deeper focus
We met Wen, a PhD student at Berkeley. We were amazed to realize he sometimes felt more intimidated about participating in small classrooms than large classrooms. It would be game changing to build trust and a safe space in small classrooms.
We met Prof. Keith Schwartz, who teaches CS 103 at Stanford. We were amazed to realize that he often would go with his gut feeling rather than being analytical about the feedback for a course he taught. It would be game changing to build confidence among both teachers and students that learning objectives are being met.
Next step: more needfinding with deeper focus

6. More needfinding Database Freelancer, former student
Used to feel timid about asking questions in large lectures
“I want to see and hear the professor resolve other students’ questions. It helps me learn”
Values honest feedback from professors and clients
Feedback was very important to Ashby. He specifically mentioned, “I didn’t like feedback back when I was a student, but it’s so important. I really value it now. At work, I have the best working relationship with people who give me honest feedback.”
Ashby helped us realize that feedback is important to both teachers and students, but perhaps as different needs. Teachers need feedback to evaluate if students are meeting course objectives; in particular, if they are learning skills that are required for downstream courses. On the other hand, students need feedback to improve their skills, unblock their progress, and to gauge how they’re doing in a class.

7. More needfinding Current student
“If you don’t get a confusion resolved during lecture, then the rest of it goes over your head.”
But was afraid of asking “dumb” questions in lecture.
“When my friends asked me questions during lecture, it helped me to formulate my thoughts and keep track of things.”
Lathrop Tech lounge
Now that Robbie is a sophomore and more experienced, he’s not as afraid of asking questions, unlike his freshman self.

8. Students uncomfortable participating in large and small lectures
Insights
Students uncomfortable participating in large and small lectures
Hearing and asking questions in lecture helps people learn
Feedback helps students grow
Image source:

9. Revising POVs

10. Revised POVs and HMWs
How might we...
make asking questions in class less intimidating?
make the classroom feel like a community?
empower students to engage without feeling judged?
Get the professor to check in on students’ understanding during the class?
Eliminate questions from the classroom?
We met Wen, a PhD student at Berkeley. We were amazed to realize he wouldn’t ask questions in class, even when he was confused, if he felt like other students were smarter than him. It would be game changing to lower the barrier for students asking clarifying questions in class

11. Revised POVs and HMWs
How might we...
find more teaching resources so that students can receive enough feedback?
encourage students to give feedback to each other?
enable students to gauge how they’re doing in a class?
tighten the feedback loop between students and educators?
Get students to reach out for help only when they really need it?
Make teachers more efficient at giving help in the classroom?
We met Michael, an educator at Palantir. We were amazed to realize that he felt there was not “enough of him” to effectively help students in his medium-sized (~40 people) class. It would be game changing to ensure that students get the help and feedback they need to succeed in medium to large-sized classrooms

12. Revised POVs and HMWs
How might we...
give teachers insight on how students are doing in downstream classes?
make course feedback more representative of the class as a whole?
get feedback about a course from past students?
motivate students to give accurate feedback?
enable student feedback to have direct impacts on their current course experience?
We met Prof. Keith Schwarz, who teaches CS 103 at Stanford. We were amazed to realize that he gauges whether or not students have met course objectives in a very ad hoc manner, relying on spontaneous feedback from teachers of downstream classes or past students. It would be game changing to facilitate getting reliable feedback about whether students are meeting course objectives.

13. Brainstorming

14. Testing and Experience Prototypes

15. How might we make asking questions in class less intimidating?
Icon created by Symbolon from Noun Project

16. How might we make asking questions in class less intimidating?
Anonymous real time question app for use among students during lecture.
During lecture, students submit a question and upvote questions from other students
Include a comment feature?
Icon created by Symbolon from Noun Project

17. Experience Prototype #1
Assumption:
When students are told that others in the class have the same question or think their question is good, they will ask the question in lecture.
Two methods to test this assumption

18. Experience Prototype #1, Method #1:
Student writes down questions they have during class
We didn’t tell student whether they should ask the question or not
Student had 6 questions in 90 minute lecture
On the second and fourth questions, we indicated that we had the same question

19. Experience Prototype #1, Method #1:
Student only asked 1 out of 6 questions...and it was when we validated their question!
Interesting observation:
The second time we said we had the same question, student asked us to ask instead
“I just asked a question”
Insights / qualified assumption
Students more willing to ask a question in class if a peer has the same question.
But not more than once?
Minimum waiting period between asking questions?

20. Experience Prototype #1, Method #2:
Student writes down questions they have during class
For each question, they looked at a post-it with a number x and imagined x people liked their question. “Anonymous” validation.
Student had 2 questions in 90 minute lecture (they knew the topic well)

21. Experience Prototype #1, Method #2:
Student writes down questions they have during class
For each question, they looked at a post-it with a number x and imagined x people liked their question. “Anonymous” validation.
Student had 2 questions in 90 minute lecture (they knew the topic well)
When 1 person “liked” question -> didn’t ask
When 10 people “liked” -> asked question

22. Experience Prototype #1, Method #2:
Insights / qualified assumption
Peer validation helps when asking questions, but the number of validations seems to matter.
More is better?
Friend validation > anonymous?
Was 1 person liking the question not enough? Why?
Threshold for the min # of likes before person will ask?

23. How might we find more teaching resources so that students can receive enough feedback?
Icon created by Symbolon from Noun Project

24. Platform that incentivizes past students to help current students
How might we find more teaching resources so that students can receive enough feedback?
Platform that incentivizes past students to help current students
Past students (“helpers”) answer questions about course content, provide feedback, and hold office hours.
Low time / financial commitment for teachers and helpers
Icon created by Symbolon from Noun Project

25. Experience Prototype #2, Method #1
Assumption:
Past students are interested in helping current students, with some incentivization.
Placed a poster in the Huang basement, offering standard financial incentive for past students to help current students
Students tear off one of the contact tabs at the bottom if they are interested
and/or

26. Experience Prototype #2, Method #1
Result - only 1 person interested. Assumption didn’t hold well.
Because of inadequate incentive? Lack of credibility?
1 person interested -> perhaps more would be interested with extensive publicity / better credibility?
Prototype was not as accurate as it could have been for simulating our desired interaction
Idea of “micro-commitments” not captured

27. Experience Prototype #2, Method #2
Tried a new mechanism for testing the assumption.
Instead of pricing based on hours spent, we based it on doubts solved
Asked it to 2 students, doing their Master’s in EE and another in CS
They felt that - “I feel the TA session should be enough for solving any doubts. But if someone needs my help and is ready to pay, then I can do it.”
While the “micro-commitment” strategy may work in case of an innovative pricing model, there were still concerns
The students said “Why not just go to office hours?”
Another concern which was raised was regarding time availability. They felt that the payment probably didn’t justify the time commitment but still would consider the option.

28. How might we give teachers insight on how students are doing in downstream classes?
Icon created by Symbolon from Noun Project

29. How might we give teachers insight on how students are doing in downstream classes?
A “1 year later” or ”1 quarter later” course evaluation platform where students assess how well a course prepared them for later classes.
Incentivize students to fill out evaluations by letting them see other students’ evaluations
Professors can see if students are meeting course objectives, revise course accordingly
Different from existing course evaluation platforms:
As Keith had mentioned to us, current course evaluations are pretty unreliable
Students like courses if they like the teacher, not necessarily if the course was useful
Also, students fill out evals right at the end of the quarter. They’re not going to know how useful this course was for a later one until much later.
That’s why Keith relies on teacher/students downstream who decide to give him feedback
Icon created by Symbolon from Noun Project

30. Experience Prototype #3
Assumption:
Students are willing to write class evaluations in exchange for access to an evaluation database that could inform their future class choices.
Ask students to write an evaluation of a course they took, in exchange for access to other evaluations

31. Experience Prototype #3
Out of 4 students, 3 agreed enthusiastically and 1 agreed conditionally
Insisted there be at least a 1:1 ratio in evaluations he fills out to the set of evaluations per class that he receives in return
The 2 students in the picture engaged in a follow up discussion after they finished the course evaluations. When they had finished writing the evaluations, one of them realized that they were actually planning on taking the course that the other student evaluated, so they then engaged in a follow-up conversation.

32. Experience Prototype #3
Insights / qualified assumption
Students are willing to write class evaluations in exchange for access to other evaluations...if the exchange is good enough.
What is good enough? Varies by student?
Surprise: everyone loved the idea
Surprise: the level of enthusiasm
Surprise: the followup conversation

33. Experience Prototype #3
“If you could develop a good platform that could reliably tell me whether the course I taught students was adequately preparing them for downstream courses, projects, whatever, I would pay money for that.”
-Michael, educator at Palantir

34. Wrap Up
In class questions promising, but already existing structures like Piazza, chat messages, etc that could be tweaked to fill this need
Downstream class evaluations useful, but
-need more research on exactly what information is most useful to educators
-how to motivate initial pool of evaluators

35. Thank you!
Questions?