Itamar Gati The Hebrew University Jerusalem Evidence-Based Practice: Applying Decision-Theory to Facilitate Individual’s Career Choices Itamar Gati The Hebrew University Jerusalem

Choosing a Career as a Decision-Making Process: Unique Features Amount of Information: Often large N of alternatives Large N of considerations and factors Within-occupation variance Practically unlimited Quality of Information Soft, subjective Fuzzy Inaccurate or biased

Unique Features of Career Decisions (continued) Uncertainty about the individual’s future preferences about future career options unpredictable changes and opportunities the implementation of the choice Non-cognitive Factors emotional and personality-related factors necessity for compromise actual or perceived social barriers and biases

CDM Difficulties of 15,000 surfers on the Future Directions website (Gati & Meyers, 2003) Are you experiencing difficulties in making your career decision?

Implications and Conclusion Many factors contribute to the complexity and difficulties involved in the career decision-making process Career counseling may be viewed as decision counseling, which aims at facilitating the clients' decision-making process, and promoting better career decisions By adopting decision theory and adapting it to the unique features of career decisions, theoretical knowledge can be translated into practical interventions to facilitate individuals’ career choices

How can Theoretical Knowledge and Empirical Methods be used for Developing Counseling Instruments? Today’s Presentation The three bases of career counseling: Locating the focuses of the client’s decision-making difficulties (CDDQ) Guidance in the decision-making process The three-stage model (PIC) Identifying the client’s stage in the process Characterizing the client’s decision-making style (DS)

Career Decision-Making Difficulties The first step in helping individuals is to locate the focuses of the difficulties they face in making career decisions Gati, Krausz, and Osipow (1996) proposed a taxonomy for describing the difficulties (see Figure 1), based on: the stage in the decision-making process during which the difficulties typically arise the similarity between the sources of the difficulties the effects that the difficulties may have on the process and the relevant type of intervention

Figure 1: Locating Career Decision-making Difficulties based on the taxonomy of Gati, Krausz, & Osipow (1996) Prior to Engaging in the Process Lack of Readiness due to Lack of motivation Indeci-siveness Dysfunc-tional beliefs During the Process Lack of Information about Cdm process Self Occu- pations Ways of obtaining info. Inconsistent Information due to Unreliable Info. Internal conflicts Externalconflicts

The Career Decision-making Difficulties Questionnaire (CDDQ) The Career Decision-making Difficulties Questionnaire (CDDQ) was developed to test this taxonomy and serve as a means for assessing individuals’ career decision-making difficulties Cronbach Alpha internal consistency estimates: .70-.90 for the 3 major categories, .95 for the total CDDQ score

Empirical Structure of the Difficulties (N= 10,000; 2004) Lack of motivations Indecisiveness Dysfunctional beliefs Lack of info about self Lack of info about process LoI about occupations LoI about addition sources of help Unreliable Information Internal conflicts External conflicts

Computerized Assessment of Career Decision-Making Difficulties The CDDQ was incorporated into a career-related self-help-oriented free of charge Internet site (www.cddq.org). Research has shown that the Internet and the paper-and-pencil versions of the CDDQ are equivalent (Gati & Saka, 2001; Kleiman & Gati, 2004). The CDDQ was found suitable for different countries and cultures and has been translated into 18 languages.

Interpreting the CDDQ results Measuring career decision-making difficulties is not enough – interpretation is very important Interpretation is part of face-to-face counseling and is crucial for Internet-based assessment of career decision-making difficulties, where no expert counselor is available The proposed interpretation procedure is aimed at locating the individual’s salient difficulties and recommending ways to deal with them (with added reservations when needed)

The Four Stages of Interpretation Ascertaining Credibility, using validity items and the time required to fill out the questionnaire Estimating Differentiation based on the standard deviation of the 10 difficulty-scale scores Locating the Salient, moderate, or negligible difficulties, based on the individual's absolute and relative scale scores Determining the need to add reservations to the feedback provided (based on doubtful credibility, partial differentiation, or low informativeness)

The 4 Stages of Interpretation 1 Evaluating Credibility Not Credible Doubtful Credible 2 Estimating Differentiation Low Questionable High 3 Locate Salient Difficulties Aggregate Reasons to Add Reservation (RAR) Compute Informativeness (B /W ) B/W < 1 RAR = 3 B/W > 1 RAR ≤ 2 4 Add Reservation to Feedback Receives Feedback No Feedback

Interpreting the CDDQ results The goal: empirically testing a four-stage model for interpreting the CDDQ profiles of individuals The interpretation is based on the within-client relative salience of the difficulties as well as their absolute salience, augmented by quality-assurance measures Career counselors' expert judgments were used to validate the proposed procedures of analyses

5 Studies Study 1: Ascertaining the Credibility of Responses to the CDDQ, based on validity items Study 2: Estimating the Differentiation of Responses, based on the SDs of the 10 scale scores Study 3: Determining the Relative Salience of Difficulties (salient, moderate, negligible) Study 4: Determining the Need to Add Reservations to the Feedback

Studies 1-4 Career counselors' expert judgments were used in the four studies for validating the proposed procedures Method Participants: career counselors and graduate counseling students Questionnaires: in studies 1,4 - all possible cases; in studies 2,3 - responses of 16 actual clients Results: High similarity between experts’ and students’ judgments, as well as within-groups judgments High similarity between the experts’ judgments and the proposed algorithm at each stage

Study 5 – Testing the Applicability of the Proposed Model Method: Analyzing the CDDQ data of four groups (N = 6,192) Hebrew paper-and-pencil version – 965 university students Hebrew Internet version - 4030 individuals surfing the Future Directions Internet site (www.kivunim.com) English paper-and-pencil version - 452 US College students English Internet version - 745 individuals who filled out the CDDQ on the Internet ( www.cddq.org ) Results: see Figures 3 & 4

Figure 3: The Distribution of the Three Levels of Difficulties (negligible, moderate, salient difficulty) in the Ten Difficulty Categories and in Four Groups (N = 6192; H-Hebrew, E-English, p-paper and pencil, I-Internet) Difficulty category

Figure 4: Distribution of types of feedback in the four groups

Conclusions The incorporation of a middle level of discrimination increases the usefulness of the feedback and decreases the chances and implications of potential errors Adding reservations when appropriate is essential for providing meaningful feedback and decreasing the chances of misleading conclusions

General Feedback on the CDDQ

Detailed Feedback on the CDDQ

Among the salient difficulties is “lack of information about the career decision-making process” (4) The Distribution of the Three Levels of Difficulties (negligible, moderate, salient difficulty) in the Ten Difficulty Categories and the Four Groups (N = 6192; H-Hebrew, E-English, p-paper and pencil, I-Internet)

Guidance in the decision-making process The PIC model (Gati & Asher, 2001) which separates the career decision- making process into 3 distinct stages: - Prescreening - In-depth exploration - Choice

Prescreening Goal: Locating a small set (about 7) of promising alternatives that deserve further, in-depth exploration Method: Sequential Elimination Locate and prioritize aspects or factors Explicate within-aspect preferences Eliminate incompatible alternatives Check list of promising alternatives Outcome: A list of verified promising alternatives worth further, in-depth exploration

Steps in Sequential Elimination Locating and prioritizing aspects or factors Explicate within-factor preferences in the most important factor not yet considered Eliminate incompatible alternatives yes Too many promising alternatives? no This is the recommended list of occupations worth further, in-depth exploration

A Schematic Presentation of the Sequential Elimination Process (within aspects, across alternatives) Potential Alternatives 1 2 3 4 . . . . N Aspects a (most important) b (second in importance) c . n Promising Alternatives

In-depth exploration Goal: Locating alternatives that are not only promising but indeed suitable for the individual. Method: collecting additional information, focusing on one promising occupation at a time: Is the occupation INDEED suitable for me? verifying compatibility with one’s preferences in the most important aspects considering compatibility within the less important aspects Am I suitable for the occupation? probability of actualization: previous studies, grades, achievements fit with the core aspects of the occupation Outcome: A few most suitable alternatives (about 3-4)

Choice Goal: Choosing the most suitable alternative, and rank-ordering additional, second-best alternatives Method: comparing and evaluating the suitable alternatives pinpointing the most suitable one Am I likely to activate it? if not - selecting second-best alternative(s) if yes - Am I confident in my choice? if not: Return to In-depth exploration stage if yes: Done! Outcome: The best alternative or a rank-order of the best alternatives

Still… But luckily, information and communication Career decision-making requires collecting a vast amount of information Complex information-processing is needed But luckily, information and communication technologies are available The use of a computer-assisted career guidance system based on a theoretical model can help overcome human cognitive limitations There are several computer-assisted career guidance systems available, most of them on the Internet

However, although Internet-based, career-related self-help sites are flourishing, these sites, as well as “stand-alone” computer-assisted career-guidance systems, vary greatly in quality. Hence, it is very important to investigate the utility and validity of these self-help programs.

Stand-Alone, Internet-Based Career-Planning Systems Possible Solutions Desirable Features CDDQ Assessment of needs Steps (PIC), factors to consider, dealing with compromises and uncertainty Providing guidance concerning the process potential alternatives, their characteristics, training Providing relevant and accurate information

Stand-Alone Internet-Based Career-Planning Systems (continued) Possible Solutions Desirable Features User’s input- continuous feedback, outcome – sensitivity analysis Monitoring the dialogue on the Internet or elsewhere Guiding the user toward additional sources of information informative summary of the dialogue Directing the user to face-to-face counseling when needed

MBCD Making Better Career Decisions MBCD is an Internet-based career planning system that is a unique combination of a career-information system a decision-making support system an expert system Based on the rationale of the PIC model, MBCD is designed to help deliberating individuals make better career decisions

MBCD – Goals Advancing the user’s career decision-making by locating a small set of promising occupational alternatives on which s/he may focus and collect more detailed information. Increasing the user’s readiness and motivation to make a career decision. Presenting a practical model of career decision-making that can be implemented in future career decisions as well as other decisions.

MBCD – System’s Features Prescreening Promising alternatives are located using the Sequential-Elimination model (Gati, 1986), which takes into consideration those career aspects that are most important to the counselee. MBCD includes 28 career factors

MBCD’s Key Features (cont.) Eliciting both facets of the individual’s preferences: (a) the optimal level (b) additional levels that the user regards as acceptable (reflecting the user’s willingness to compromise)

MBCD’s Key Features (cont.) Each occupation is characterized by a range of levels within each aspect, reflecting the within-occupation variance. The system provides detailed feedback and recommendations according to the user’s input and its effect on the search results. The dialogue is flexible and the users can change their responses at any point.

MBCD’s Key Features (cont.) Promising alternatives are located by the Sequential-Elimination search model (Gati, 1986). But the user can also use a compensatory-model-based search.

Compensatory model-based search Goal – locating the most compatible occupations Rationale - advantages of occupations may compensate for their disadvantages Steps of the compensatory search Locate gaps between preferences and the characteristics of the occupation for each factor Sum the gaps, weighted by importance of factors Locate occupations with minimal sum of gaps

The Conjunction of the Two Lists Sequential elimination-based list Compensation-based list Conjunction list Users are advised to focus on the occupations that were included in the recommended list of both search models in the in-depth exploration

MBCD’s Key Features (cont.) Options to check the quality of the list of “promising occupations”, including: “Almost compatible occupations” (i.e., sensitivity analysis) “Why not” “What if” “Similar occupations” “Compare Occupations” During the dialogue users can ask which occupations have been eliminated from their list of promising alternatives due to only one small discrepancy with their reported preferences (this is the “almost compatible occupations” option). The results of the sensitivity analysis are automatically included in the (personalized) dialogue-summary printout.

MBCD’s Features (cont.) Initial in-depth explorations is offered by detailed occupational descriptions During the dialogue users can ask which occupations have been eliminated from their list of promising alternatives due to only one small discrepancy with their reported preferences (this is the “almost compatible occupations” option). The results of the sensitivity analysis are automatically included in the (personalized) dialogue-summary printout.

MBCD’s Features (cont.) At the end of the dialogue the user receives a printed summary to take along for further processing of the information. The printout also provides information for the counselor. The user’s preferences are saved under a personalized code for future interactions.

Making Better Career Decisions Does it really work?

END of PART 1

Making Better Career Decisions Does it really work?

Prescreening Based on Elimination: Descriptive Validity (Gati & Tikotzki,1989) The monitored dialogues of 384 career counselees with a computer-assisted career information system were analyzed. Results: most users (96%) employed a non-compensatory strategy during all or at least a part of the dialogue: many options considered at a previous stage of the dialogue were not considered at the following stage, showing that individuals tend to use a prescreening strategy based on eliminating alternatives

Criteria for Testing the Benefits of Making Better Career Decisions Examine users' perceptions of MBCD Examine changes in user’s degree of decidedness Examine perceived benefits Locate factors that contribute to these variables

METHOD Participants 247 males and 465 females who filled out both a pre-dialogue and a post-dialogue questionnaire Mean age 22.8; mean years of education 12.6 4% high-school students 6% recent graduates from high school 58% recently completed their military service 9% considering an alternative to their current major 3% college graduates deliberating a job choice 8% considering a career transition 12% "other"

Decidedness Increased No change Decreased 355 (50%) 266 (37%) 91 (13%) Mean Perceived Benefit (MPB) and Willingness to Recommend (WR) the Use of MBCD to a Friend (%) as a Function of the Difference in Decidedness after the Dialogue of MBCD (N=712) Decidedness   Increased No change Decreased Frequency 355 (50%) 266 (37%) 91 (13%) MPB 3.12 2.57 2.52 WR% 93.5 74.8 72.5 Measure

Decidedness Before the Dialogue Frequencies of Degree of Decidedness Before and after the Dialogue with MBCD Decidedness After the Dialogue Decidedness Before the Dialogue 1 2 3 4 5 1- no direction 34 7 6   2 - only a general direction 41 66 15 9 3 - Client is considering a few specific alternatives 27 58 84 30 4 - would like to examine additional alternatives 23 51 35 54 5 - would like to collect information about a specific occupation 20 21 28 6 - sure which occupation to choose 16

Decidedness Before the Dialogue with MBCD Willingness to Recommend (WR) the Use of MBCD to a friend as a Function of the Degree of Decidedness Before and After the Dialogue with MBCD (N=712) Decidedness After the Dialogue with MBCD Decidedness Before the Dialogue with MBCD   1 2 3 4 5 1- no direction 38 14 17 29 -- 2 - only a general direction 85 73 67 100 3 - considering a few specific alternatives 93 82 97 4 - client would like to examine additional alternatives 92 5 - would like to collect information about a specific occupation 90 98 89 6 - Client is sure which occupation to choose 81

MBCD’s Effect on Reducing Career Decision-Making Difficulties (d, Cohen, 1992) Scale .31 .13 .29 .16 Lack of Readiness Motivation General indecisiveness Dysfunctional Beliefs .72 .48 .45 .78 .20 Lack of Information About The Process The Self Occupational Alternatives Additional Sources .11 .18 .01 -.13 Inconsistent Information Unreliable Information Internal Conflicts External Conflicts .65 Total CDDQ

MBCD’s Effect (d, Cohen, 1992) on Reducing Career Decision-Making Difficulties (Gati, Saka, & Krausz, 2003)

Predictive Validity of MBCD Design: Comparing the Occupational Choice Satisfaction (OCS) of two groups: those whose present occupation was included in MBCD’s recommended list those whose present occupation was not included in MBCD’s recommended list

Method Participants The original sample included 123 clients who used MBCD in 1997, as part of their counseling at the Hadassah Career-Counseling Institute Out of the 73 that were located after six+ years, 70 agreed to participate in the follow-up: 44 women (64%) and 26 men (36%), aged 23 to 51 (mean = 28.4, SD = 5.03)

Method Instruments MBCD Questionnaire: clients were asked to report their field of studies, their satisfaction with their present occupational choice (scale of 1 – 9): “low” (1-4), “moderate” (5-7), “high” (8-9) Procedure the located clients were interviewed by phone, six+ years after visiting the career-counseling center

Results Frequencies of Occupational Choice Satisfaction by Acceptance and Rejection of MBCD's Recommendations, Based on Sequential Elimination

Frequencies of Occupational Choice Satisfaction by the Search-Model Whose Recommendations Were Accepted

Conclusions Accepting the recommendations of the sequential-elimination-based search of MBCD produces the best outcomes (i.e., highest levels of satisfactions with the occupation) The data does not support the effectiveness of the compensatory-based search The data does not support any advantage of using the conjunction list over using only the sequential-elimination-search list

Alternative Explanations Differences in the lengths of the lists No difference was found in the OCS between clients whose list included 15 or fewer occupations and clients whose list included more than 15 occupations. Therefore, this explanation can be ruled out.

Alternative Explanations (cont.) Clients who accepted MBCD’s recommendations are more compliant, and therefore more inclined to report a high level of satisfaction. However, following the compensatory-model-based recommendations did not contribute to the OCS. Therefore, this explanation can be ruled out too.

Gender Differences in Directly and Indirectly Elicited Career-Related Preferences Gadassi and Gati 2006 Method Participants. 226 females (74.1%) and 79 males (25.9%) who entered the Future Directions Internet site Age: 17-30, mean=22.84 (median = 22, SD = 3.34) Years of education: mean=12.67 (median 12, SD = 1.48)

Instruments Future Directions (http://www.kivunim.com) Making Better Career Decisions (MBCD, http://mbcd.intocareers.org) The preference questionnaire: this questionnaire imitated the preference elicitation in MBCD. Participants were presented with 31 aspects, and were asked to rank-order them according to importance, and to report their preferences in all 31 aspects

Preliminary analysis Lists of occupations. We used MBCD to generate three lists of occupations according to: (1) sequential-elimination (2) compensation, and, for 235 participants, (3) the list based on the conjunction between the sequential elimination and the compensatory search lists.

Preliminary analysis Lists of occupations. We used MBCD to generate three lists of occupations according to: sequential-elimination compensation and, for 235 participants, the list based on the conjunction between the sequential elimination and the compensatory search lists

Preliminary analysis Determining the degree of gender-ratings of occupations was based on the judgments of 10 undergraduate students. 1 – “most (that is, over 80%) of the individuals who work in this occupation are women” 5 – “most (that is, over 80%) of the individuals who work in this occupation are men – over 80%" The inter-judge reliability was .96, We computed the mean gender-ratings of the lists of occupations for each participants

Preliminary analysis Lists of occupations. We used MBCD to generate three lists of occupations according to: sequential-elimination compensation and, for 235 participants, the list based on the conjunction between the sequential elimination and the compensatory search lists

Gender Differences in Directly and Indirectly Elicited Preferred Occupations (Gadassi & Gati, 2007)

Summary of Major Findings PIC is compatible with people’s intuitive ways of making decisions (Gati & Tikotzki, 1989) Most users reported progress in the career decision-making process (Gati, Kleiman, Saka, & Zakai, 2003) Satisfaction was also reported among those who did not progress in the process Users are “goal-directed” – the closer they are to making a decision, the more satisfied they are with MBCD The list of Recommended Occupations are not sex-type biased (Gadassi & Gati, 2006)

Identifying the Client’s Stage in the Process It is possible to start the PIC process from “the middle” – according to the client’s needs However, it is recommended to start the process from the beginning, in order to: Strengthen confidence in the occupational alternatives considered by the client Eliminate inadequate alternatives considered by the client Offer additional alternatives that were not considered by the client so far Teach decisions skills: aspect-based instead of occupation-based approach

The stage in the PIC model decision-process of pre-academic programs students, at the beginning and end of the program (N=386) The stage in the decision-making process – beginning of programs total 4 3 1 2 The stage in the dcm process – end of programs 13 1 2 3 7 1-before pre-screening 77 5 17 11 44 2-before in-depths exploration 93 7 29 12 45 3- before choice 203 60 50 8 85 4 – after choice 386 73 98 34 181 Total - over rows (55%) 211 made progress in the process (35%) 136 stayed in the same stage (10%) 39 moved backwards

Tailoring the Intervention to the Client’s Decision-Making Style There is an advantage in tailoring the counseling intervention to the client’s decision-making style Previous research typically characterized individuals by the most dominant characteristic of their decision-making style (e.g., intuitive, dependent). we suggest that a multidimensional analysis should be used to uncover a comprehensive decision-making style-profile of clients. A theoretical framework based on ten dimensions related to the career decision-making process was developed for characterizing individuals' career-decision making styles

The Ten Dimensions The degree of analytic vs. holistic information-processing The level of effort invested in the process The degree of comprehensiveness in gathering and integrating the information The degree of consultation with others The degree of realism (willingness to compromise) Internal vs. external locus of control The speed of making the final decision The degree of procrastination The degree of dependence on others The degree of acceptance to others’ wills

Testing the Proposed Model To empirically test the proposed taxonomy we developed the career Decision-making Style Questionnaire (DSQ), in which each of the proposed dimensions was represented by a few statements. The questionnaire was uploaded to a career-related, self-help oriented Internet site (www.kivunim.com ) A cluster analysis supported the proposed differentiation between all ten dimensions.

Locating Repeated Profiles of Decision-Making Styles Based on a cluster analysis of the participants, we located homogeneous groups of participants with similar career decision-making style profiles We found five groups of participants with similar decision-making styles These results were discussed in terms of the hypothesized ten dimensions and the previously identified career decision-making styles

The Means of the Located Groups in Terms of the 10 Dimensions Red = Low; Green = High 5 4 3 2 1 Dimension 2.4 2.3 4.4 3.4 4.5 Analytic 2.2 3.3 4.2 3.9 4.6 Effort 3.5 Comprehens. 3.1 Consulting 3.7 4.0 2.7 3.2 3.6 Realistic 2.6 1.9 4.1 2.9 Locus of Speed Procrastin 4.9 Dependence 2.0 Acceptance

General Average of the Located Groups Sd M Group 0.43 2.91 4 0.50 3.17 5 0.48 3.82 2 0.36 3.87 1 0.23 4.03 3

To sum up, I presented and discussed: The CDDQ for locating the focuses of the individual’s decision-making difficulties, and the design and testing of a systematic procedure for interpreting its results A general framework for cdm – the PIC model MBCD – a unique combination of career information, expert, and a decision-support system DSQ – A taxonomy and a questionnaire for a multidimensional analysis of client’s decision-making styles

To sum up Career choices are decision-making processes, therefore career counseling is also decision counseling Decision theory can be translated into practical interventions aimed at facilitating individuals’ career decision-making Many tools were transformed into user-friendly Internet-based systems, which can be incorporated into counseling interventions The theory-based interventions can and should be empirically tested for theoretical validity as well as practical effectiveness

END Sofsof

Figure 2: Receives Feedback Ascertaining Credibility credible doubtful high partial Locate Salient Difficulty Categories Add Reservation to Feedback low No Feedback Compute Informativeness (Bv/Wv) Receives Feedback B/W > 1 B/W < 1 Estimating Differentiation Ascertaining Credibility non Aggregate Reasons to Add Reservation (RAR) RAR ≤ 2 RAR = 3 Figure 2:

Results: Compared Means of the Femininity-Masculinity Score According to Type of List and Gender

The Empirical Structure of the 10 Dimensions