Predicting Undergraduates’ Grades from the Survey of Academic Orientations and from the General Information Scholastic Test Prof. Hall P. Beck.

Slides:



Advertisements
Similar presentations
Survey Methodology Reliability and Validity EPID 626 Lecture 12.
Advertisements

Research Methodology Lecture No : 11 (Goodness Of Measures)
What is a Good Test Validity: Does test measure what it is supposed to measure? Reliability: Are the results consistent? Objectivity: Can two or more.
AIR 2005 Student Research & Information Student Affairs William Armstrong, PhD & Heidi Carty, PhD UC San Diego 1 Comparing the Academic Transition of Freshmen.
Social Demographic Predictors of Graduation Rate Completers Prof. Irmannette Torres-Lugo Assistant Researcher Office of Institutional Research & Planning.
Is College Success Associated With High School Performance? Elizabeth Fisk, Dr. Kathryn Hamilton (Advisor), University of Wisconsin - Stout Introduction.
June 19, Proposal: An overall Plan Design to obtain answer to the research questions or problems Outline the various tasks you plan to undertake.
Looking Good, Teaching Well? Linking Liking, Looks, and Learning Regan A. R. Gurung, Kristin M. Grudzielanek, and Christina J. Tosh Attractiveness is a.
Reliability and Validity of Research Instruments
MULTIPLE REGRESSION. OVERVIEW What Makes it Multiple? What Makes it Multiple? Additional Assumptions Additional Assumptions Methods of Entering Variables.
LINEAR REGRESSION: Evaluating Regression Models. Overview Assumptions for Linear Regression Evaluating a Regression Model.
LINEAR REGRESSION: Evaluating Regression Models. Overview Standard Error of the Estimate Goodness of Fit Coefficient of Determination Regression Coefficients.
Attachment as a moderator of the effect of security in mentoring on subsequent perceptions of mentoring and relationship quality with college teachers.
Introduction Method  Evaluation of ability to provide social support yielded scores with good internal consistency reliability.  There was moderate agreement.
Examination of Holland’s Predictive Pattern Order Hypothesis for Academic Achievement William D. Beverly and Robert A. Horn Northern Arizona University,
Reliability and Validity
Assessing Achievement and Aptitude
Assessing Achievement and Aptitude: Applications for Counseling Chapter 8.
Chapter 7 Correlational Research Gay, Mills, and Airasian
Chapter 7 Evaluating What a Test Really Measures
Measurement Concepts & Interpretation. Scores on tests can be interpreted: By comparing a client to a peer in the norm group to determine how different.
Student Achievement and Predictors of Student Achievement in a State Level Agricultural Mechanics Career Development Event Edward Franklin Glen Miller.
When the Tail Wags the Dog: Perceptions of Learning and Grade Orientations in, and by, Contemporary College Students and Faculty Howard R. Pollio University.
Aron, Aron, & Coups, Statistics for the Behavioral and Social Sciences: A Brief Course (3e), © 2005 Prentice Hall Chapter 3 Correlation and Prediction.
METHODS Study Population Study Population: 224 students enrolled in a 3-credit hour, undergraduate, clinical pharmacology course in Fall 2005 and Spring.
Enhancing Parents’ Role in Higher Education Assessment Anne Marie Delaney Director of Institutional Research, Babson College.
Development and Validation of a Biodata Tool for Civilian Army Personnel The U.S. Army developed the Civilian Leader Improvement Battery (CLIMB) containing.
Reducing Anxiety Christine Velardi. The Power of Positive Recollections: Reducing Test Anxiety and Enhancing College Student Efficacy and Performance.
Is PeerMark a useful tool for formative assessment of literature review? A trial in the School of Veterinary Science Duret, D & Durrani,
AFT 7/12/04 Marywood University Using Data for Decision Support and Planning.
Instrument Validity & Reliability. Why do we use instruments? Reliance upon our senses for empirical evidence Senses are unreliable Senses are imprecise.
MGTO 324 Recruitment and Selections Validity II (Criterion Validity) Kin Fai Ellick Wong Ph.D. Department of Management of Organizations Hong Kong University.
When the Tail Wags the Dog: Perceptions of Learning and Grade Orientations in, and by, Contemporary College Students and Faculty Howard R. Pollio University.
Copyright © Allyn & Bacon 2008 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public.
Correlation and Regression PS397 Testing and Measurement January 16, 2007 Thanh-Thanh Tieu.
Chapter 4: Test administration. z scores Standard score expressed in terms of standard deviation units which indicates distance raw score is from mean.
S&T Student Course Evaluations: Beyond Question 7 – Part 2 Dan Cernusca, Ph.D. Instructional Design Specialist Missouri University of Science and Technology.
Reliability vs. Validity.  Reliability  the consistency of your measurement, or the degree to which an instrument measures the same way each time it.
Quantitative SOTL Research Methods Krista Trinder, College of Medicine Brad Wuetherick, GMCTE October 28, 2010.
1 Self-Regulation and Ability Predictors of Academic Success during College Anastasia Kitsantas, Faye Huie, and Adam Winsler George Mason University.
MGS3100_04.ppt/Sep 29, 2015/Page 1 Georgia State University - Confidential MGS 3100 Business Analysis Regression Sep 29 and 30, 2015.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Measurement Models: Exploratory and Confirmatory Factor Analysis James G. Anderson, Ph.D. Purdue University.
OpInform 2015 College Preparation Beyond GPA and Standardized Tests.
Appraisal and Its Application to Counseling COUN 550 Saint Joseph College For Class # 3 Copyright © 2005 by R. Halstead. All rights reserved.
Chapter 8 Validity and Reliability. Validity How well can you defend the measure? –Face V –Content V –Criterion-related V –Construct V.
Lecture 02.
Stat 112 Notes 9 Today: –Multicollinearity (Chapter 4.6) –Multiple regression and causal inference.
A Comparison of General v. Specific Measures of Achievement Goal Orientation Lisa Baranik, Kenneth Barron, Sara Finney, and Donna Sundre Motivation Research.
Practices and Predictors of the Use of Accommodations by University Faculty to Support College Students with Disabilities Leena Jo Landmark, M.Ed., and.
Chapter 9 Correlation, Validity and Reliability. Nature of Correlation Association – an attempt to describe or understand Not causal –However, many people.
SOCW 671: #5 Measurement Levels, Reliability, Validity, & Classic Measurement Theory.
Grades: Their Effects on Students as Measures of Achievement.
 The Knowledge, Attitudes, Experience and Confidence of MPRNs and Advance Directives (ADs)
Online students’ perceived self-efficacy: Does it change? Presenter: Jenny Tseng Professor: Ming-Puu Chen Date: July 11, 2007 C. Y. Lee & E. L. Witta (2001).
Early Identification of Introductory Major's Biology Students for Inclusion in an Academic Support Program BETHANY V. BOWLING and E. DAVID THOMPSON Department.
Educational Research Chapter 8. Tools of Research Scales and instruments – measure complex characteristics such as intelligence and achievement Scales.
Measurement Chapter 6. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts.
Survey of Academic Orientation Scores Predict Retention of College Freshmen Hall P. Beck Mary Francis Hill Appalachian State University William D. Davidson.
The SAT: Yay or Nay Hao Li. Outline 1. The problem --“SAT” optimal movement 2. Misconceptions about the SAT 3. Predictive validity of SAT score: predicting.
Results Reliability Consistency and stability of cluster solution across different samples In both years, three distinct cluster groups identified thus.
Interpreting Test Results using the Normal Distribution Dr. Amanda Hilsmier.
1 Measurement Error All systematic effects acting to bias recorded results: -- Unclear Questions -- Ambiguous Questions -- Unclear Instructions -- Socially-acceptable.
Jenn Shinaberger Corey Lee Lee Shinaberger Coastal Carolina University
Dr. Nina O’Brien Department of Management
Assessment in Career Counseling
College Applications How to Make the Most of Your College Experience
Learning online: Motivated to Self-Regulate?
MGS 3100 Business Analysis Regression Feb 18, 2016
Presentation transcript:

Predicting Undergraduates’ Grades from the Survey of Academic Orientations and from the General Information Scholastic Test Prof. Hall P. Beck

Predictors of Grades Academic Predictors: High School Record, Teacher Recommendations, Standardized Tests Personality Tests: I-E, Test Anxiety, Tolerance of Ambiguity Perceptions of Academic Environment: SAO

Purposes of Investigation Determine if the GIST is predictive of GPA Determine if the SAO is predictive of GPA Determine if the GIST and SAO given concurrently explain greater variance in GPA than either instrument alone Determine if the GIST and SAO improve the prediction of GPA after taking PGPA into account.

GIST And SAO Measures GIST-I and GIST-M Structure Dependence, Creative Expression, Reading For Pleasure, Academic Efficacy, Academic Apathy, Mistrust of Instructors Adaptiveness Index

Representative GIST-I Items Which hormone helps the body respond to physical and emotional stress? a.Saliva b.Philogiston c.Estrogen d.Insulin e.Adrenaline The Magna Carta was signed in a.Germany b.France c.England d.Italy e.United States

Representative GIST-M Items The square root of.50 is: a.Greater than.50 b.Equal to 2.00 divided by 4 c.Less than.50 d.Equal to.50 squared e.None of the above To neutralize her chemical solution Keri must mix her acid in a 3:1 ratio with a strong base. If Keri measures 4.5 ml of acid into a beaker, how much base will be required to neutralize the solution? a.1.5 ml b.1.75 ml c.2.25 ml d.2.5 ml e.None of the above

Representative SAO Items Structure Dependence: I like questions that have definite right and wrong answers. Creative Expression: Learning new things is thrilling. Reading For Pleasure: I like to look through the library for books that spark my interest even when I am not working on a specific course assignment. Academic Efficacy: I sometimes wonder if I am college material. Academic Apathy: I try to work just hard enough to get the grade that I need in a class. Mistrust of Instructors: Some instructors enjoy giving students poor grades.

The General Information Scholastic Test: Some Validity Data (Cloherty, 1998) Significant, positive correlations were found between GIST-I and SAT-V (.63) and GIST-M and SAT-M (.61) When corrected for restriction of range the correlations increased to.79 and.76, respectively

Development and Validation of Scores on a Measure of Six Academic Orientations in College Students Davidson, Beck and Silver, 1999

SAO Alpha Values

SAO Test-Retest

Construct Validation of the SAO Personality Profiles: Correlate SAO factors with personality indices Work Preference Inventory Personal Need For Structure 16PF O and L Scales NEO-Openness California Personality Inventory Ai Scale LOGO II

Methodology 1.Administered SAO and GIST to 264 first semester ASU freshmen 2.Obtained permission to retrieve first semester GPA from the registrar and regressed the six SAO orientations as well as the GIST-I and GIST-M with GPA

Regression of Variables on GPA (R 2 ) SAOGISTSAO + GIST.15**.07**.20** **p <.01

Regression of Variables on GPA (R 2 ) PGPAPGPA+GIST.12**.13 **p <.01

Regression of Variables on GPA (R 2 ) PGPAPGPA + SAO.12**.23** **p <.01

Discussion: GIST Brief and accurate measure of academic ability and/or achievement for research purposes. Assisting admission personnel in evaluating non- traditional students. Assisting high school counselors in determining if their students’ SAT scores are atypical.

Discussion: SAO and/or GIST Provide universities with a better means of predicting Academic and Social adjustment than is currently available. Provide counselors with information regarding perceptions of their students. Allow university personnel to focus their services on students who need it the most. Serve as a guide for the development of therapeutic interventions.