1. Nigel Ward University of Texas at El Paso Fifth International Conference on Intelligent Technologies December 3, 2004 Dealing with Uncertainty in a Model of Computer Science Graduate Admissions

2. (a 12 minute pre-banquet talk at a small 3-day gathering of soft-computing researchers)

3. The Problem 10,000+ CS grad school applicants a year many wasted applications some disappointed applicants A Solution enable applicants to predict acceptance decisions, using a web tool a model of applicant strength + models of admissions criteria

4. demonstration The Acceptance Estimator Concept

5. How to Combine GRE Scores? Two common styles: avg/sum and min: “we expect a GRE V+Q+A of at least 2100” “we expect at least 600 V, 700 Q and 650 A” A compromise: stronger scores weighted less, but not zero* 1.33 for weakest, 1.0 for middle,.67 for strongest (an ordered weighted averaging operator) * cf Carlsson, Fuller and Fuller in Yager and Kacprzyk, 1997

6. Sample Computation raw value (RV) normalized value NV rank R ranking factor RF contribution level CL verbal600100#1.6767 quantitative6500#31.330 analytical writing 4.562#21.0062

7. Explaining Apparent Diversity admissions policy for department x standard model of applicant strength > GQ department- specific threshold x omissions simplifications guesses fog X’s published admissions policy and statistics spin

8. Estimating the Scaling Parameters To apply OWA, we must normalize scores first what is the GRE Q score corresponding to a 3.7 UTEP GPA? GRE Composite GPA JNTU Mumbai U. Texas at El Paso

9. Weighting the Scores factor GRE V IW.7 GRE Q1.0 GRE AW.7 GPA (if US)2.5 GPA (JNTU, Madras)2.5 GPA (other Indian)2.0 … letters of recommendationvaries ∑ CL x IW i ∑IW ii i i CGRE =

10. Complexities in Recommendations commeasurate with GREs and GPA can be a plus or a minus are fundamentally optional are not expected to have specific points so no ranking factors vary in influence so the importance weight computation is vital

11. Modeling Recommendations Leading recommender is adescribing you as a = weight scaling factor = warmth score

12. Summary of the Computation Subtract Baseline and Scale Raw to get Normalized Value: Weight and Sum: Order Normalized Values and apply Ranking Factors to get Contribution Levels: ∑ CL x IW i ∑IW ii i i GQ = NV = (RV - BV ) x SF iiii CL = NV x RF iii where r is rank, n is number of scores RF = 2 r - 1 3 n - 1 ( 1+ ) i

13. Factors in Admissions Decisions In the Model GREs GPA in-major or recent GPA major letters of recommendation statement of purpose scholarships group membership Not in the Model undergrad school GRE subject test (CS) TOEFL nationality/culture specific coursework research match publications etc.

14. Evaluation 55 UTEP applicant datafiles accept / reject compare compute GQ score > -25? applicant featuresaccept/reject decisions 51/55 successes with failures explicable

15. Modeling Other Departments compute GQ score applicant data > accept / reject threshold for school X published data for school X compute GQ score

16. Does the Model Work for Departments?

17. Thus selectivity, as measured by the model, correlates with desirability, somewhat

18. The Diversity Behind the Numbers Minimum scores of 550, 600 and 3.5 on the verbal, quantitative, and analytical writing sections, respectively (U. of Delaware) Most students admitted have earned scores in excess of 650 for the Analytical and Quantitative parts (Columbia) Average scores of successful applicants to this program for Fall 2002: GRE: 560 verbal, 770 quantitative (U. of Houston)

19. Averages, Minimums, and Thresholds inferred threshold

20. Averages, Minimums, and Thresholds inferred threshold threshold vs. min: ~30 (0.15 GPA points) ==> departments don’t take risks (?) avg vs. threshold: ~20 (0.1 GPA points) ==> departments don’t have much variety (?)

21. A View of the Applicant Pool Number of Applicants Overall Applicant Strength (GQ score) minimum average acceptees

22. A Blurred View Number of Applicants Applicant Strength measured by GREs only minimum average acceptees

23. Modeling Other Departments compute GQ score applicant data > accept / reject threshold for school X published data for school X compute GQ score

24. Modeling Other Departments compute GQ score applicant data > accept / reject threshold for school X published data for school X compute GQ score adjustment 3010soft minimums 4010hard minimums 20-20average 40-30most above marginadjustmentdescription

25. Presenting Uncertainty

26. Some Sources of Uncertainty user interface errors lack of information about the applicant incorrect fundamental assumptions incorrect GQ-model parameters incorrect modeling of departments’ criteria inadequate information on departments

27. Try it Yourself! http://www.cs.utep.edu/admissions/

28. Future Work verification on data from more departments better parameter estimates on more data a more parameterized version to model different departments better a centralized clearinghouse?

29. Benefits for UTEP better informs potential UTEP applicants increases site traffic, and applicant pool? increases Google score shows we understand student needs makes the world a better place