Welcome! Three Approaches to Measuring STEM Education Innovations: Moving Toward Standardization and Large Data Sets Wendy DuBow, PhD, National Center for Women & Information Technology Betsy Bizot, PhD, Computing Research Association Tom McKlin aka Shelly Engelman, PhD, Sage Fox Group Discussant: Jason Ravitz, Google

What You’re In For… All instruments will focus on psycho- social constructs All were used in a Computer Science context All of our constructs can be used independently All are able to be shared for free (with acknowledgement)

Why This Panel

A Social Cognitive Career Theory Survey Wendy DuBow, Ph.D. NCWIT Director of Evaluation

My Professional Context Serve as an internal evaluator for Nat’l Center for Women & IT, & conduct research to further our mission, & serve as external evaluator on other NSF-funded projects. One small part of my job is to: Conduct research and evaluation on CS educational and outreach interventions in school settings

Social Cognitive Career Theory One way to explain how academic and career interests are developed… Premises: Self-efficacy in a given field and expectations of positive outcomes from continued involvement lead to increased interest. Interest then directly predicts goal setting and choices that ultimately lead to a particular career. Perceived supports & barriers communicated by parents, peers, and significant others also influence interest and goals.

Constructs We Measure 1.Interest in computing 2.Self-efficacy in computing 3.Outcome expectations 4.Perceived social supports/barriers 5.Intent to persist

Sources: Social Cognitive Career Theory Lent, R. W., Miller, M. J., Smith, P. E., Watford, B. A., Lim, R. H., Hui, K.,... Williams, K. (2013). Social Cognitive Predictors of Adjustment to Engineering Majors Across Gender and Race/Ethnicity. Journal of Vocational Behavior, 83(1), Lent, R. W., Lopez Jr., A. M., Lopez, F. G., & Sheu, H. B. (2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73,

Development History Based on Lent’s SCCT survey Adapted to our needs with 3 programs in mind Piloted with Talk Aloud Used in a variety of MS and HS programs Ran EFA and CFA on 2 data sets

Used in this Context: Respondents: middle/high school students Setting: computing outreach programs in school and out of school Timing: post-only but has been used as pre/post

Self-efficacy Construct Right now, how confident are you in your ability to… design computer games use new computer software (e.g., wiki pages, web pages, games) solve software problems (e.g., software crashes, slow programs) fix or build computers or other technologies out of parts program computers or other devices (in other words, write code) think of new technology inventions (e.g., new apps or software, improved tablets or MP3 players)

Perceived Social Supports/Barriers Important people in my life think it’s good for me to learn about technology. I believe people like me can do well learning technology. My family likes me to learn about technology. My friends think it’s cool that I learn about technology. I believe people like me can do well in technology jobs. I believe people like me can create new technology inventions.

Request the entire survey…