Inference: Bayesian, Cursed, … Michael Conlin Michigan State University Stacy Dickert-Conlin Michigan State University Jeff Wooldridge Michigan State University 2009 Summer Meetings

Optional SAT Policies “I SOMETIMES think I should write a handbook for college admission officials titled “How to Play the U.S. News & World Report Ranking Game, and Win!” I would devote the first chapter to a tactic called “SAT optional.” The idea is simple: tell applicants that they can choose whether or not to submit their SAT or ACT scores. Predictably, those applicants with low scores or those who know that they score poorly on standardized aptitude tests will not submit. Those with high scores will submit. When the college computes the mean SAT or ACT score of its enrolled students, voilà! its average will have risen. And so too, it can fondly hope, will its status in the annual U.S. News & World Report’s college rankings.” Colin Diver, President of Reed College, New York Times, 2006

U.S. News & World Report (Criteria and weights for rankings colleges)

Research Question What is the college’s inference for applicants who choose not to submit their SAT I scores?

College Data Application data for 2 liberal arts schools in north east Submitted SATI Scores, Submitted SATII Scores, Submitted ACT Scores, High School GPA, Private High School, Race, Gender, Residence, Legacy Acceptance and Enrollment Decisions. Performance Measures for those who Enroll.

College Board Data SAT I scores for those who elected not to submit them to the college. SAT II scores Student Descriptive Questionnaire (SDQ) Self Reported income High school GPA High school activities

Summary Statistics 15.3 percent of the 7,023 applicants to College X choose not to submit SAT I scores. 24.1 percent of the 3,054 applicants to College X choose not to submit SAT I scores.

Summary Statistics for College X N=324 N=122 N=5216 N=895

Summary Statistics for College X

Conclusions from Prior Research (based on estimates from reduced form) College admission departments are behaving strategically by more (less) likely accepting applicants WHO DO NOT SUBMIT their SAT I scores if submitting their scores would decrease (increase) the average SAT I score the colleges report to the ranking organizations. Applicants are behaving strategically by choosing not to reveal their SAT I scores if they are below a value one might predict based on their other observable characteristics. These reduced form results are robust to different assumptions regarding college’s inference for those applicants who do not submit.

Voluntary Disclosure Example Student i has the following probability distribution in term of SAT I scores. When disclosure is costless, Bayesian Nash Equilibrium results in every type except the worst disclosing and the worst being indifferent between disclosing and not disclosing. SAT I Score Probability 1300 0.2 1200 0.4 1100 0.3 1000 0.1 Expected SAT I Score 1300(.2)+1200(.4)+1100(.3)+1000(.1)=1170

Voluntary Disclosure Models Comments: Distribution depends on student characteristics that are observable to the school such as high school GPA. With positive disclosure costs, the “unraveling” is not complete and only the types with the lower SAT I scores do not disclose. Assumptions: Common Knowledge. Colleges use Bayesian Updating to Infer SAT I Score of those who do not Submit/Disclose Colleges’ incentives to admit an applicant is only a function of his/her actual SAT I score (not whether the applicant submits the score)

Voluntary Disclosure: Theory SAT I Score Probability 1300 0.2 1200 0.4 1100 0.3 1000 0.1 Eyster and Rabin (Econometrica, 2005) propose a new equilibrium concept which they call a Cursed Equilibrium. College correctly predicts the distribution of the other players’ actions but underestimates the degree these actions are correlated with the other players’ private information. “Fully” Cursed Equilibrium (χ=1)– College infers if applicant doesn’t disclose that his/her expected SAT I score is 1300(.2)+1200(.4)+1100(.3)+1000(.1)=1170 “Partially” Cursed Equilibrium (χ=.4 for example)– College infers if applicant doesn’t disclose that his/her expected SAT I score is (1-.4) [(1100(.3)+1000(.1))/.4]+ (.4)1170 = 1113

Model and Structural Estimation

Notation Known to the applicant at the time she submits her application Known to the applicant at the time of enrollment decision μ(Xi)+εap +εen ,expected utility from attending College X for applicant i UR ,expected utility if applicant does not attend College X and does not apply early decision at College X (normalized to zero). UR-C , expected utility if applicant does not attend College X and does apply early decision at College X. εs , unobserved cost of submitting SAT I

Whether to Apply Early Decision and/or Submit SATI Score

College’s Objective Function To account for the college’s concern for the quality of its current and future students and the understanding that future student quality depends on the college’s ranking, we allow the college’s objective function to depend on the perceived ability of the incoming students, the “reported” ability of these students, and the demographic characteristics of the student body.

College accepts applicant i if: Unobserved (to us) quality of applicant i College accepts applicant i if: Pe(Xi,k,l) [ΠP(X+iP)+εqi + ΠR(X+iR)+ ΠD(X+iD)] +(1- Pe(Xi,k,l)) [ΠP(X-iP)+ ΠR(X-iR)+ ΠD(X-iD)] >ΠP(XriP)+ ΠR(XriR)+ ΠD(XriD) Or Pa(Xi,k,l) =Prob{εqi>ΠP(X-iP)-ΠP(X+iP)+ΠR(X-iR)-ΠR(X+iR) +ΠD(X-iD)-ΠD(X+iD)+f(YRri)-f(YRai)/Pe(Xi,k,l) } X’s are expected characteristics of incoming class Probability i attends (function of whether apply early and submit)

Can relate X+ij-X-ij to Xij for j=P,R,D College accepts applicant i if: Pa(Xi,k,l) =Prob{εqi>βP•XiP+βR•XiR+βD•XiD+ βYR/Pe(Xi,k,l)} What about perceived quality of applicant who does not submit SAT I? Depends on what college infers.

What about perceived quality of applicant who does not submit? College accepts applicant i if: Pa(Xi,k,l) =Prob{εqi>βP•XiP+βR•XiR+βD•XiD+ βYR/Pe(Xi,k,l)} Based on Eyster and Rabin for those who don’t submit, assume SATiP = χ SATi,unconditional + (1- χ)SATi,conditional where χ is a parameter to be estimated.

Applicant’s Decision to Enroll if Accepted An accepted applicant who applies early decision will enroll if μ(X)+ εap+εen> -C An accepted applicant who doesn’t apply early decision enrolls if μ(X)+ εap+εen> 0 (Probability that these conditions hold determine Pe(Xi,k,l).)

Estimation Assuming εap,εen,εq,εs~N(0,1) and are independent, can derive a maximum likelihood function and obtain parameter estimates of, among other things, (χ,C,βYR) using SML. χ =0.266 (for College X)

Future Work Consider “Counterfactuals” What if χ=0? How would that affect admission decisions and quality of student body? What if the College had to report their “true” average SAT I scores to the ranking organizations? How would that affect their admission decisions and quality of student body?