Presentation by Patsy Dougherty

Slides:



Advertisements
Similar presentations
Katherine L. Hughes & Melinda Mechur Karp
Advertisements

Class Advisement Surveys Class Advisement Surveys Northview High School Results Report Fall 2012.
The American Freshman: National Norms for Fall 2005 Sylvia Hurtado & John H. Pryor January 26, 2006 Cooperative Institutional Research Program (CIRP) Higher.
Independent and Dependent Variables
General Information --- What is the purpose of the test? For what population is the designed? Is this population relevant to the people who will take your.
Impact of Career Services Retention, Graduation, Confidence, SLOs & SDOs.
Experimental Research Designs
Business Research for Decision Making Sixth Edition by Duane Davis Chapter 7 Foundations of Measurement PowerPoint Slides for the Instructor’s Resource.
Factors Influencing Indices of Career Development Jessica Lazenby Research for the Ronald E. McNair Post-Baccalaureate Achievement Program Faculty Mentor:
The Development of Life Purpose in Pepperdine University Undergraduates Cindy Miller-Perrin Don Thompson Pepperdine University Faculty Conference October.
Making Better Career Decisions Itamar Gati The Hebrew University of Jerusalem.
Itamar Gati The Hebrew University of Jerusalem
Gauging the Impact of Student Characteristics on Faculty Course Evaluations Laura Benson Marotta University at Albany – SUNY.
Part 4 The PIC Model: Supporting Evidence or: Does it really work?
Psychometric Properties of the Job Search Self-Efficacy Scale Investigators: Jeff Christianson Cody Foster Jon Ingram Dan Neighbors Faculty Mentor: Dr.
Survey Research & Understanding Statistics
Research Methodology Lecture No :27 (Sample Research Project Using SPSS – Part -A)
Sampling and Data Collection
Baseline Data for Assessment of Academic Advising Initiative Janine M. Allen, Ph.D.  Professor of Education  Portland State University  Cathleen L.
Research Methods in MIS
Assessment in Career Counseling
Assessment in Career Counseling
Notes for Social Sciences Constructing Survey Questions.
Personality and Individuality
Quantitative Research
Student Technological Mastery: It's Not Just the Hardware Wm. H. Huffman, Ph.D. Ann H. Huffman, Ph.D.
1 Copyright © 2011 by Saunders, an imprint of Elsevier Inc. Chapter 10 Clarifying Measurement and Data Collection in Quantitative Research.
Test Validity S-005. Validity of measurement Reliability refers to consistency –Are we getting something stable over time? –Internally consistent? Validity.
Measurement Concepts & Interpretation. Scores on tests can be interpreted: By comparing a client to a peer in the norm group to determine how different.
Study announcement if you are interested!. Questions  Is there one type of mixed design that is more common than the other types?  Even though there.
Chapter 12. Observational and Survey Research Methods Chapter Objectives Distinguish between naturalistic and participant observation methods Articulate.
Stress Survey Jamie Blondina Chelsea Leaman MaryJane Fuhrer Allie Robbins.
Predicting Marital Success with PREPARE: A Predictive Validity Study Article by B.J Fowers and D.H Olson Presentation by: Aylin Atabek Elissa Vaidman Qiana.
Results: SEM was used to test fit and examine relations among variables for six models corresponding to each theme of the Holland code. All models, with.
Copyright © 2001 by The Psychological Corporation 1 The Academic Competence Evaluation Scales (ACES) Rating scale technology for identifying students with.
Perceived Constraints by Students to Participation in Campus Recreational Sports.
Chapter Five: Nonexperimental Methods II: Ex Post Facto Studies, Surveys and Questionnaires, Sampling and Basic Research Strategies.
Perceptions and attitudes of fifth year medical students of Stellenbosch University regarding the Rural Clinical School in Worcester.
Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell Chapter 13 Survey Designs.
Correlational Research
COPYRIGHT © 2014 Brooks/Cole*Wadsworth Publishing Company A division of Cengage Inc. 1 Applying Career Development Theory to Counseling, 6e by Richard.
AFT 7/12/04 Marywood University Using Data for Decision Support and Planning.
Exploring the Relationships Among College Students’ Goal Orientations, Perfectionism, and Academic Self-Efficacy Hannah Geis and Brittany Weber, Faculty.
Student Engagement Survey Results and Analysis June 2011.
 Collecting Quantitative  Data  By: Zainab Aidroos.
The Psychologist as Detective, 4e by Smith/Davis © 2007 Pearson Education Chapter Five: Nonexperimental Methods II: Ex Post Facto Studies, Surveys and.
Evaluating a Research Report
Research Problem In one sentence, describe the problem that is the focus of your classroom research project about student learning: Biology undergraduates.
Psychometrics & Validation Psychometrics & Measurement Validity Properties of a “good measure” –Standardization –Reliability –Validity A Taxonomy of Item.
Descriptive Research Study Investigation of Positive and Negative Affect of UniJos PhD Students toward their PhD Research Project Dr. K. A. Korb University.
Selecting a Sample. Sampling Select participants for study Select participants for study Must represent a larger group Must represent a larger group Picked.
Session 4 Reliability and Validity. Validity What does the instrument measure and How well does it measure what it is supposed to measure? Is there enough.
Instrumental Behaviors Following Test Administration and Interpretation: Exploration Validity of the Strong Interest Inventory Presented by: Ana AriasAlheli.
Ch 9 Internal and External Validity. Validity  The quality of the instruments used in the research study  Will the reader believe what they are readying.
An Action Research Project Chevonne Byas EDUC 7021 T Fall 2010 Overeating and Underachieving: The Effects of Physical Fitness on the Academic Achievement.
COPYRIGHT © 2014 Brooks/Cole*Wadsworth Publishing Company A division of Cengage Inc. 1 Applying Career Development Theory to Counseling, 6e by Richard.
Advancing Assessment of Quantitative and Scientific Reasoning Donna L. Sundre Amy D. Thelk Center for Assessment and Research Studies (CARS) James Madison.
Chapter Eight: Quantitative Methods
HELEN ROSENBERG UNIVERSITY OF WISCONSIN-PARKSIDE SUSAN REED DEPAUL UNIVERSITY ANNE STATHAM UNIVERSITY OF SOUTHERN INDIANA HOWARD ROSING DEPAUL UNIVERSITY.
Career Counseling: A Holistic Approach
DESIGNING GOOD SURVEYS Laura P. Naumann Assistant Professor of Psychology Nevada State College.
Assessment and Career Planning
The Behavior Assessment System for Children (BASC)
–Anonymous Participant
Interest and Vocational Testing
Chapter Eight: Quantitative Methods
Quasi-Experimental Design
Assessment in Career Counseling
Demographic Characteristics of the Sample DISCUSSION/CASE STUDY
Presentation transcript:

Presentation by Patsy Dougherty Career Decision Scale Test created by Samuel H. Osipow, Clarke G. Carney, Jane Winer, Barbara Yanico, and Maryanne Koschier Presentation by Patsy Dougherty

Try out the sample question! Pass out handouts. Have everyone complete the sample question on the handout… These are the types of questions included on the Career Decision Scale.

More samples: Exactly like me Very much like me Only slightly like me Not at all like me I have decided on a career and feel comfortable with it. I also know how to go about implementing my choice. 4 3 2 1 I know what I’d like to major, in but I don’t know what careers it can lead to that would satisfy me. None of the above items describe me. The following would describe me better: (write your response below). ________________________________________________________________________________________________________________________________________________ The first question is one of the two for the Certainty Scale. The other one only differs by replacing “career” with “major.” The second question counts toward the Indecision Scale. The third question is the last question on the assessment.

Career Decision Scale The Career Decision Scale was designed as a “rapid and reliable instrument for surveying high school and college students about their status in the decision-making process.” It “provides an estimate of career indecision and its antecedents as well as an outcome measure for determining the effects of interventions relevant to career choice and development.” Also… This scale provides a simple way of identifying the principal barriers which interfere with an individual's ability to make a required educational or vocational decision.

Career Decision Scale High School and College students Development: Special Populations: Adults in Continuing Education Programs and Women Returning to College Specialty Versions: Spanish, medical students, rewording for graduate students, rewording for high school students Development: Part of a proposed modular system to promote self-counseling about career indecision 16 items (The Indecision Scale) 18 items (The Certainty Scale) 19 items (Open-ended Questions) -> Normed according to Grade and Gender… Also includes Age norms from validity studies, manual doesn’t use age because of the correlation with grade… but neighter of these are very accurate norm groups - Specialty versions NOT available for use…. Used in validity studies. Any rewording was modest ( and the HS rewording was deemed unnecessary since it was already normed on Hsers)

Factor Structure Original Study VS. Further Examination Stability? Lack of Structure and Confidence External Barrier to a Preferred Choice Difficulty Choosing among Options Personal Conflict Original Study VS. Further Examination Stability? Differences may be due to item interpretation Still, the study has a good predictability! Factor Analysis only conducted for the Indecision Scale - All factors are associated with decision-making! (Not just lack of structure in life)

Reliability – Test Manual Osipow, Carney, and Barak (1976) Individual Items and Indecision Scale Test-retest correlation, 0.9 and 0.82 Item correlations for Certainty and Indecision Scales .34-.82 Mostly 0.60-0.80 Slaney, Palko- Nonemaker, and Alexander (1981) Test-retest over 6 weeks Certainty and Indecision Scale Item correlation from 0.19-0.70 CDS total = 0.70 ** Not alternate forms *** .8 = acceptable .7 = can still be useful DEF room for improvement

Validity – Test Manual Group Comparison and Correlations with Other Instruments Assessment of Career Decision Making (ACDM) Occupational Alternative Question Career Maturity Inventory (CMI) - Attitudes Scale Holland and Holland Scale Treatment Studies Career Counseling Interventions College Residential Career Exploration Program Career Planning Discussion Groups Career Exploration Class Career Development Workshops Relationships with Other Personality Variables External Locus of Control Fear of Success Relationships with Demographic Variables Age Differences Grade Level Sex Differences Ethnic Difference GPA Achievement Type Aptitude Area of Study

Validity – Test Manual Methods Results Slaney (1980): Group Comparisons and Correlations with Other Instruments Occupational Alternatives Question First choice and no alternatives First choice plus alternatives Methods N = 232 Male and female college students Comparison of scores for both tests No first choice with alternatives Neither a first choice or alternatives Results Concurrent Validity: Clearly differentiated subjects into same categories in both tests Concurrent Validity… which as we learned in class can be a double edged sword because it assumes the other instrument is valid. May not be the best test of validity, but it is a strong example of the validity that the CDS has.

Predictive Validity of the Career Decision Scale Administered to High School Students Hartman, Fuqua & Hartman (1983) N = 205 Chicago, urban, middle income community High school seniors enrolled in specific coursework Two categories: Decided-No change Undecided-Still Undecided Methods In-person administration Follow-up phone interview Final phone interview Results Predictive Validity: strong, increasing over time Factor Structure: After one year, only 3 relevant. After two years, only 2. Problem with this study is the focus on using the CDS in high schools, when the CDS is already normed for high schools. Actually relatively useless study. **No treatment Thought this would be an interesting study to analyze because it is mentioned in the manual as a specialty version, and represents yet another conflicting finding of the factor structure of the CDS. This is PART TWO (the second year) of the study that is not included in the manual.

Proposed Validity Study 500 students in First Year College Program, NCSU Treatment Group: 250 students Control Group: 250 students Pre-test: August Post-test: December Treatment/Intervention: Bi-weekly group workshops and individual advising meetings Topics including: Career Information Decision-Making Strategies Interest Inventories Convenience sampling (use the students available at state)… Ask for volunteers in the First Year College Program  Wanted to look at this type of validtiy study because there were some conflicting results reported on these category of studies in the manual. Would assume that the results of this study would indicated PREDICTIVE VALIDITY that students who received the treatment/intervention would have increased certainty ratings and decreased indecision ratings.

Results of Administration 2 = Male 3 = Female 5 = College 1 Freshman (Male) 1 Sophomore (Male) 1 Junior (Female) 2 Seniors (Females) Individual administration Convenience sampling

Scoring Procedure Total 1-2 Total 3-18 Normative Group % ile Cer Ind This is what the scoring box looks like. You add up the scores for the two scales and place them in those boxes. Next you input the norm group applicable to the subject, (such as Female, College, Senior). Last, you look up the percentiles associated with the subject’s scores for each scale. These percentiles are located in the back of the test manual Suggested that you look at individual items in order to plan proper interventions more specific to the needs and what factors are significant

Percentile Distribution Uncertain Certain Decisive Indecisive Medium Further need for assessment Explain that there are two distributions that should be inversely correlated. If you score the same range in each, then it is likely that the data is inaccurate. If you score opposing scores than your results are more clearly defined as to whether you need interventions (Cer-Low, Ind-High) or don’t (Cer-High, Ind-Low). If you fall in the middle for either scale, you require more assessment. Unspecified whether you would benefit from intervention. Low High 16 84

Participant 1 6 58 28 Senior 78 Female, College Total 1-2 Total 3-18 Normative Group % ile Cer 6 Female, College 58 Ind 28 Senior 78 Interpretation **** Participant one falls in the Middle range for both scores. These scores are about average, and the participant would benefit from additional testing

Participant 2 8 100 23 Senior 52 Female, College Total 1-2 Total 3-18 Normative Group % ile Cer 8 Female, College 100 Ind 23 Senior 52 Interpretation **** 8 = 100th percentile… didn’t know you could have that.

Participant 3 5 21 34 Junior 84 Female, College Total 1-2 Total 3-18 Normative Group % ile Cer 5 Female, College 21 Ind 34 Junior 84 CER – Actually 1 SD below the mean for her Grade! Could be signficantly low for certainty…. Is almostttt significantly high for Indecision High for Indecision, VERY close to Low for Certainty…. “High likelihood of need for intervention”

Participant 4 8 100 30 Freshman 55 Male, College Total 1-2 Total 3-18 Normative Group % ile Cer 8 Male, College 100 Ind 30 Freshman 55 Interpretation **** 100 on the certainty scale, again, is a strange percentile Also, while he was VERY extroidinarily high on certainty, he was in the middle for Indecision, where as these two are intended to be inversely correlated. This means that Participant 5 would likely benefit from additional assessment in order to more effectively analyze his needs.

Participant 5 8 100 25 Sophomore 42 Male, College Total 1-2 Total 3-18 Normative Group % ile Cer 8 Male, College 100 Ind 25 Sophomore 42 Interpretation ****

Means and Standard Deviations Manual Includes: High School (Sample S and Sample W) Sex Grade Age ** College Sex Grade Age ** College of Study ** Age – Asked on the test booklet, but it is not normed. Only mentioned in the standard deviations College of Study - NOT mentioned in the manual or asked for on the test… ** -> comes from various validity studies listed in manual

Means – College Females Biggest differences are Participant 2 who is low on Indecision, which is good.. That means she is likely more decisive While Participant 3 is about 7 scale points HIGHER than the mean on Indecesion

Means – College Males

Problems Successes Response Rate Out-of-date Instruction Scoring Time commitment Successes Response rate: had to ask around to a few people before receiving 5 completed assessments Out-of-date: THIS WAS TEST-TAKER REPORTED. Some of the questions apparently seemed unclear, and the wording is somewhat out-of-date. Instruction: Very easy instruction, and easily self-administered Time commitment: Very short assessment, so people were more willing to take and complete the assessment

Summary Positive Aspects: Reliable instrument Validity studied extensively Proven predictability Points for Consideration: Certainty Scale unclear Factor instability Aged test instrument

References Hartman, B. W., Fuqua, D. R., & Hartman, P. T. (1983) Predictive validity of the Career Decision Scale administered to high school students. Psychological Reports, 52, 95-100. doi: 10.2466/pr0.1983.52.1.95 Osipow, S. H. (1980). Career decision scale: Manual. (3rd rev.) Columbus, OH: Marathon Consulting & Press, 1980. Osipow, S. H., Carney, C. G., Winer, J. L., Yanico, B., & Kochier, M. (1976). The Career Decision Scale (3rd revision). Odessa, FL: Psychological Assessment Resources, Inc.

Questions?