E-Mission + Team of undergraduates = ???

Background and motivation

Open source travel diary collection (M) Extension to survey data collection (P) Aggregate modelling and analysis (A) If you have a project in one of these areas, come talk to me! Happy to deploy (with adequate support) and generate results.

Motivation, overview and structure

Scale and complexity of problems is large. Lots of work is interdisciplinary, which makes it hard because you have to deal with non core CS aspects (I say so!) Need for long-term and sustained system building to tackle large-scale social problems (David Patterson says so!) Many CS students have programming experience when entering. How to engage them, provide challenging work and deepen skills? (David Corman says so!)

Researcher Help with routine tasks Skills that researcher does not have (e.g. UI design) Interdisciplinary skills (e.g. behavioral econ) Larger pool of people to avoid design bias Throwing away final projects year after year seems wasteful Students Build something real and end-to-end that is used after class is done Learn to identify and split up real-world problems If publishable, improve grad school admission chances

15 students max. 12 finally enrolled CS majors, but outside minors related to behavior, transportation or sustainability a plus Made an exception for a business major, didn’t work so well Grades in lower division courses Made an exception for a B student really interested in sustainable transportation, worked pretty well

First half, skill building. In 2-3 person teams, fix a bug related to the skill of the week. Phone based sensing and interaction Service architecture and backend People Data integration Exploratory Data Analytics Second half, form small project teams and work on three related projects that contribute to e-mission (their choice) Tour models User models and recommendations UI display and gamification

Evaluation and lessons learned

Experience of planning for class was great Forced lit review Forced articulation of ideas to group of non-experts Forced cleaning up code to allow others to run it Forced improved documentation Students implemented a recommendation system that worked end to end Divided up work into teams User model, tour model, alternatives, UI Everything “worked” together at the end (with a little help) Class evaluation was positive (6.6/7 overall)

Clustering trips to generate tour models (needs to work for both short and long)

Ideas are good, but quality is low (demo/prototype) Recommendations are not necessarily meaningful Evaluation? We don’t need no stinkin’ evaluation Why? Shared global state and read after write dependencies State now distributed across project groups Student B depends on Student A’s code, but Student A has a midterm this week Student B depends on Student A’s code, but Student A writes crappy, ill-tested code. Now Student B has to play janitor.

Undergraduate students cannot be expected to write serious infrastructure “Testing takes too much time” in spite of many efforts Hacky code to “get things to work” Prepare to do a ton of “glue” work to get the pieces to work together Integrate early and often to force them to do more Don’t make students build complex systems with lots of dependencies Much easier to work on small, bite-size pieces that can be scheduled around other commitments

Build the infrastructure and evaluation system first One example that can be copied Dashboard/autograder/evaluation system Contests! Focus around multiple alternatives to the example Run contests for greater motivation Require deployment for evaluation Similar to kaggle, but with ongoing data collection Errors = bad results = bad score Will try this out in Spring or Fall 2016 and report back…

I’d love to hear from you if you have tried something like this.