The Struggles of New College Graduates in their First Software Development Job Andrew Begel, Human Interactions in Programming, MS Research Beth Simon *, University of California, San Diego * Visiting researcher at Microsoft Andrew Begel, Human Interactions in Programming, MS Research Beth Simon *, University of California, San Diego * Visiting researcher at Microsoft March 14, 2008SIGCSE 2008

Study Questions How do new developers learn their jobs? What are the critical skills they need for learning and productivity? How well does university prepare CS graduates for a first job in software engineering? Goals:Improve CS educational pedagogy Improve developer training at Microsoft How do new developers learn their jobs? What are the critical skills they need for learning and productivity? How well does university prepare CS graduates for a first job in software engineering? Goals:Improve CS educational pedagogy Improve developer training at Microsoft

Study Methodology 8 new Microsoft developers  4 BS, 1 MS, 3 PhD  2 USA education, 2 Chinese, 1 Pakistan, 1 Australia, 1 Kuwait, 1 Mexico. All PhD education took place in USA.  Only one had work experience prior to Microsoft (2 years).  Hired between Oct 2006 and March new Microsoft developers  4 BS, 1 MS, 3 PhD  2 USA education, 2 Chinese, 1 Pakistan, 1 Australia, 1 Kuwait, 1 Mexico. All PhD education took place in USA.  Only one had work experience prior to Microsoft (2 years).  Hired between Oct 2006 and March 2007.

But wait, this is Microsoft! Surely, this study couldn’t be relevant for my students. But wait, this is Microsoft! Surely, this study couldn’t be relevant for my students.

The Takeaway New developers’ problems are mainly due to poor communication, not technical skills.

Study Methodology Observations 4-5 times over a two month period (concentrated in early April and late May 2007) Each observation was 2-3 hours. Total of 85 hours. Participants recorded 3-5 minute videos to answer reflective questions (which we provided). Each recorded around 27 videos. Total of 216 videos. $50 Amazon gift certificate paid for each week of participation. Observations 4-5 times over a two month period (concentrated in early April and late May 2007) Each observation was 2-3 hours. Total of 85 hours. Participants recorded 3-5 minute videos to answer reflective questions (which we provided). Each recorded around 27 videos. Total of 216 videos. $50 Amazon gift certificate paid for each week of participation.

TimestampDescription Task TypeSubtask Type Task Type #2 Subtask Type #2 11:45:43 AMreruns copy script. DebuggingReproducing 11:46:18 AMscript done. checks over script output to make sure it looks right. Says that the script is complaining that the files aren't signed. with source directory says that they are signed. Weird. copied successfully, but binaries aren't signed. DebuggingReproducingMeta- Cognition Struggling with Tools 11:47:26 AMShakes head. T is confused. Team lead says they’re signed. But empirical evidence says they're not. DebuggingReproducingMeta- Cognition Confusion 11:48:11 AMT says maybe he wants to sign the binaries himself. Meta- Cognition Planning 11:48:36 AMT mutters to himself “bad bad very bad” Meta- Cognition Reflection

Analysis Methodology Each observation, record action at the moment with timestamp. Each day, code tasks and summarize observations. Each week, abstract activities into categories. Later, resort activities into categories. Recode actions into events a third party could have seen. Each observation, record action at the moment with timestamp. Each day, code tasks and summarize observations. Each week, abstract activities into categories. Later, resort activities into categories. Recode actions into events a third party could have seen.

What Do New Developers Do? Programming (reading >> writing, commenting, proofreading) Working on bugs ((repro, reporting, triage) >> debugging) Testing (writing, running) Revision Control (check in, check out, revert) Documentation (reading >> writing, search) Specifications (reading >> writing) Tools (discovering, finding, installing, using, building) Communication (asking questions, persuasion, coordination, , meetings, meeting prep, finding people, managers, teaching, learning, mentoring) Programming (reading >> writing, commenting, proofreading) Working on bugs ((repro, reporting, triage) >> debugging) Testing (writing, running) Revision Control (check in, check out, revert) Documentation (reading >> writing, search) Specifications (reading >> writing) Tools (discovering, finding, installing, using, building) Communication (asking questions, persuasion, coordination, , meetings, meeting prep, finding people, managers, teaching, learning, mentoring)

The Good and the Bad New devs are good at  Programming  Design  Peer mentoring  Using many strategies when debugging They interact naïvely with other team members when  Asking questions  Triaging and resolving bugs  Attending meetings New devs are bad at  Engaging a large, legacy codebase without using written docs  Reproducing bugs  Using new tools  Reflecting on their own progress (knowing when they are stuck) New devs are good at  Programming  Design  Peer mentoring  Using many strategies when debugging They interact naïvely with other team members when  Asking questions  Triaging and resolving bugs  Attending meetings New devs are bad at  Engaging a large, legacy codebase without using written docs  Reproducing bugs  Using new tools  Reflecting on their own progress (knowing when they are stuck)

Beliefs of New Developers Perfectionism 1.I must prove I am smart, productive and write quality code from day 1. 2.I must do everything by myself. 3.I must fix all bugs I personally find in the code. 4.Don’t ask questions. Delusion 1.It compiles, therefore it works. If it has bugs… well, that’s what testers are for. 2.I know when I get stuck. Self Esteem 1.It can’t be the bug repro instructions, it must be me. 2.My self-worth is directly related to how much I coded today.

Recommendations for CS Educators Scaffold and support social interactions in software teams  Employ legitimate peripheral participation, communities of practice Design higher-fidelity software engineering courses  Work with larger, legacy codebases, pre-existing teams, minimal docs  Make bug fixes, implementation changes, or non-critical new features  Deliver capricious management directives Meta-cognitive skill development: Am I stuck now? Scaffold and support social interactions in software teams  Employ legitimate peripheral participation, communities of practice Design higher-fidelity software engineering courses  Work with larger, legacy codebases, pre-existing teams, minimal docs  Make bug fixes, implementation changes, or non-critical new features  Deliver capricious management directives Meta-cognitive skill development: Am I stuck now?

Future Work on this Study Describe differences between novice and expert professional developers. Recode observation logs with “functional coding” schema. Analyze video diaries: What do new developers say when reflecting on their own learning? Describe differences between novice and expert professional developers. Recode observation logs with “functional coding” schema. Analyze video diaries: What do new developers say when reflecting on their own learning?

Future Research Plans Improve “onboarding” process at Microsoft Mentoring, mentoring, mentoring Modeling appropriate behavior Teach managers how to teach Study new hires in Other roles (requirements engineering, testing) Non-USA locations (e.g. China, India) Globally distributed development Mass onboarding (corporate acquisitions) Improve “onboarding” process at Microsoft Mentoring, mentoring, mentoring Modeling appropriate behavior Teach managers how to teach Study new hires in Other roles (requirements engineering, testing) Non-USA locations (e.g. China, India) Globally distributed development Mass onboarding (corporate acquisitions)