Dimensions Characterizing Programming Feature Usage by Information Workers Christopher Scaffidi, Andrew Ko, Brad Myers, Mary Shaw Carnegie Mellon University VL/HCC 2006

2 Motivation Understanding how people use features is essential to improving those features. Researchers have studied various populations’ use of programming features: –Web developers –Users in the home –Children We have surveyed another important population, information workers, who are… –Typically in an office environment –One of the largest end user groups

3 Talk outline Sample Respondent characteristics Factor analysis Interpretation Primary result: We found 3 distinct clusters of programming features based on feature usage.

4 Sample We surveyed over 800 readers of Information Week. Information Week ed 125,000 readers who opted in 831 completed the survey (816 retained for analysis) We asked if respondents or their subordinates had in the past 3 months used 23 programming features in 5 tools: –Web server scripts –Web pages –Databases –Spreadsheets –Slide editors / word processing

5 Respondent characteristics Respondents were predominantly end users. Most were managers: Information Week advertises to “information managers”. 76% of respondents had at least one subordinate. Most were in non-programming industries or jobs: Only 10% came from IT vendors; others were education, business services, government, manufacturing, finance Only 23% were IT or networking staff Most majored in business, science or engineering: Only 22% majored in comp sci or computer-centric fields

6 Respondent characteristics They were end user programmers. Most were familiar with programming constructs: 79% were familiar with all four programming terms (variables, subroutines, conditionals, and loops) 35% actually created all four constructs in the past year Most reported usage of each programming tool: Web server scripts: 52% Web pages: 69% Databases: 79% Spreadsheets: 93% Slide editors / word processing: 96%

7 Factor analysis We prepared the data for factor analysis (FA). We performed several main cleaning steps: –Filtering out people with missing feature variables: 168 left (we checked generalizability of factors, discussed later) –Scaling each feature variable to mean 0.0, std 1.0 (as not every case of feature usage is equally “significant”) –Subtracting a per-user, per-tool mean from each feature variable with that tool (to remove tool “bundling” effects) –Removing 4 features with low communality (< 0.1) We now had 168 people with 19 features showing interrelated usage tendency.

8 Factor analysis Our 3 factors (“clusters”) were Macros, Linked Structure, and Imperative A tendency to use each feature typically co-occurred with a tendency to use other features in the same factor. Macros Factor: –Recording & editing of spreadsheet & document macros Linked Structure Factor: –Creating & linking database tables –Creating web forms; creating page static includes –Creating spreadsheets & linking cells with functions Imperative Factor: –Using “new MyObj()” in Perl, PHP, and JavaScript –Creating JavaScript functions –Creating database triggers

9 Factor analysis We checked our factors several ways. We repeated factor analysis… –with different extraction algorithms –with only data from respondents with 0 subordinates … and consistently found the same qualitative structure. We checked generalizability over all 816 respondents 1.Constructed a scale from each factor 2.Checked Cronbach’s alpha for each (0.82, 0.62, 0.64)

10 Implications What does the existence of factors (feature groups) tell us? If users vary widely, can we create broadly useful features? Fortunately, 3 dimensions seem to characterize usage. What programming features are most widely used? Features in Linked Structure factor; see paper for details. Why do these feature groups exist? Possible hypotheses: Common work patterns require using features in concert. Learning a feature makes it easy to learn certain others. Mastering a feature disinclines people to use others.

11 Thank You To VL/HCC for this opportunity to present. To Rusty Weston and Lisa Smith at Information Week for their partnership. To James Herbsleb, Irina Shklovski, and Sara Kiesler for design and analysis assistance. To our paper reviewers for helpful suggestions. To NSF, Sloan, and NASA for funding.