© CogniMetrics, Inc., 2005 PROGNOST A Web-Based Personnel Assessment System

Content navigator PROGNOST: a web-based assessment system What is PROGNOST? For whom PROGNOST is designed PROGNOST: Features Quality Selection Decisions Local Accessibility User Languages: English and Spanish Expandable with non web-based tests PROGNOST: Applications Selection of Terminal Control Room Operator Selection of Executive Assistent Selection of Retail & Specialty Product Marketing PROGNOST: Specifications IT Aptitude Battery Wonderlic Personnel Test Personality Characteristics Inventory Decision Support System Pricing Availability The compass icon appears on many slides in the slide show. Click on one of its arrows to return to the navigator page.

PROGNOST : web-based assessment system PROGNOST a web-based assessment system for personnel selection. PROGNOST consists of:  low-cost customizable decision-support system.  standard (but expandable) battery of web-delivered tests. What is PROGNOST?

PROGNOST : web-based assessment system PROGNOST is designed for companies that: give a high priority to the quality of their selection and promotion decisions. expect accountability of human resources manage- ment. have employees working in workplaces that are spread out over a region, nation, or internationally. have employees with ethnic, cultural and linguistic different backgrounds. RETURN For whom PROGNOST is designed

The main goal of our web-based assessment system is to improve the quality of personnel decisions. We will introduce PROGNOST by discussing potential problems with the quality of personnel decisions. PROGNOST features

Quality Selection Decisions Psychological assessment is a systematic effort to collect and to combine relevant information for the purpose of diagnosis or prognosis of human behavior. PROGNOST features

The quality of psychological assessment often is sub- optimal because of the following:  Most HR-professionals restrict the systematic effort part to collecting information (i.e., testing and interviewing). To combine the information experts use so-called “clinical judgment”.  Over 50 years of research has shown that combining this information using “clinical judgment” is highly inaccurate and subject to distortion. Quality Selection Decisions PROGNOST features

 Until recently no user-friendly system was available to replace “clinical judgment”.  Traditional statistical techniques were too restrictive and were not user-friendly. Quality Selection Decisions PROGNOST features

In 2003 CogniMetrics, Inc. developed PROGNOST a method to build customized low-cost decision-support systems for selection decisions. A decision support system is an expert system that uses all available scientific knowledge to predict success of current or future employees for a particular job or training, based on available information (such as their test scores). What follows is a short introduction to PROGNOST: Quality Selection Decisions PROGNOST features

collecting information to predict probability of success in job / training combining this information into a single statement expressing the probability of success tests questionnaires decision- support system RISK ASSESSMENT PER INDIVIDUAL Psychological Assessment

tests questionnaires collecting information to predict probability of success in job / training combining this information into a single statement expressing the probability of success decision- support system RISK ASSESSMENT PER INDIVIDUAL Psychological Assessment

tests questionnaires collecting information to predict probability of success in job / training combining this information into a single statement expressing the probability of success decision- support system RISK ASSESSMENT PER INDIVIDUAL Psychological Assessment

PROGNOST decision support system What do we need it for? Human experts (psychologists, medical doctors) are incapable of reliably combining information of multiple tests into a single decision outcome (e.g., a hire / no-hire). This problem is particularly serious when many information sources (such as tests) have to be considered.

PROGNOST decision support system How does it work? A decision-support system combines results of scientific studies, common-sense knowledge and observational evidence to come up with a prediction of the probability of success in training or job.

PROGNOST decision support system Expert system (BN-model) Theoretical insights Empirical observations PREDICTIONS Common sense How does it work?

PROGNOST decision support system We build decision support systems using Bayesian Networks, an artificial intelligence technology. Bayesian Networks propagate information derived from objective tests (and subjective ratings if required) to a selection or placement advise for an individual applicant. Bayesian Networks are more powerful and more robust than traditional regression techniques. How does it work?

PROGNOST decision support system RETURN PROGNOST produces the following output: 1. Odds of success: a simple quantitative index. If odds > 1.00 candidate is likely to succeed on the particular task, else (s)he is more likely to fail. 2. A more detailed report with the candidates’ strong and weak points. How does it work?

TESTS collecting information CogniMetrics, Inc. offers a standard battery of tests and questionnaires that together predict over 50 % of the variance in training and job performance. all tests are web delivered languages: English, Spanish economical: low assessment costs per candidate the standard battery can be expanded or tests can be tailored to specific Alcoa needs.

The standard battery of PROGNOST administers two main groups of tests / questionnaires: Intelligence Personality traits TESTS collecting information

TESTS intelligence Eighty years of selection research has proven tests of intelligence to be the strongest predictors of success in training and job performance. The standard battery comprises two different types of intelligence tests: Wonderlic Personnel Test (WPT - Wonderlic, Inc.) IT Aptitude Test (ITAB - CogniMetrics, Inc.) RETURN

IT Aptitude Battery General : The ITAB tests measure an aspect of intelligence that has become increasingly important since the advent of Information Technology. ITAB tests measure the ability to interact efficiently with systems that have a dynamic of their own. Such systems vary in complexity from MS Office software tools to complex information systems monitoring industrial processes. The cognitive skill required for using such devices is called procedural knowledge, or procedural skill.

IT Aptitude Battery Features : The ITAB test reports deliver: a multifaceted PERFORMANCE SCORE, measuring Procedural Knowledge Aptitude, and scores on two causative factors: INTELLIGENCE: efficiency in information processing SYSTEMATIC APPROACH: steady approach and careful inferences. RETURN

Wonderlic Personnel Test The Wonderlic Personnel Test (WPT) is a classical test of general cognitive ability (general intelligence). It measures primarily so-called crystalyzed intelligence: a product of past learning.

Wonderlic Personnel Test Features : Short test of general intelligence (test time 12 minutes) with reliabilities and validities comparable with much longer tests. 50 items in ascending order of difficulty. Web delivered. RETURN

TESTS personality Certain core personality traits are a predisposition to long- term commitments, such as needed to complete a training program..

PC Inventory The Personality Characteristics Inventory evaluates candidates on five basic dimensions of personality: Conscientiousness Extraversion Agreeableness Openness Stability

PC Inventory The 150 items can be rearranged in several combinations so as to generate various scores: 12 subscales: efficiency, dependability, achievement striving, etc. success scales: commitment to work, integrity, learning, orientation, teamwork. occupational scales:manager, sales, production worker, driver, clerical employee.

PC Inventory Features: No time limit (generally takes minutes to complete). 150 Questions. Valid, reliable, and legal. RETURN

CogniMetrics’ tests are web-delivered, which makes test administration possible at virtually any place of the globe. This is ideal for companies that have geographically spread work locations. An additional advantage of our system is that it guarantees an equal weighing of test information throughout the company. Local Accessibility PROGNOST features

All tests of the proposed standard battery are available in both English and Spanish. PROGNOST features User languages

PROGNOST features Expandable system PROGNOST can be designed to deal with information gathered through observation, interview, and paper-and- pencil tests. For example, the HR specialist may want to add behavior ratings obtained from observing Assessment Center exercises. PROGNOST can be made to accept this information and weigh it appropriately.

Applications examples PROGNOST can be designed to optimally predict any category of employment positions. CogniMetrics reviews empirical studies involving the prediction of success in those positions and estimates the population correlation between predictors and criterion. translates the outcomes into the structure of the Bayesian Network (conditional probabilities). The following slides show a number current job openings in large San Antonio based companies (Alcoa, Inc.; H.E.B., VALERO Energy Corp.). The purpose is to demonstrate that PROGNOST with the ITAB tests has an edge in predicting success in work environments of the 21 th century.

Applications examples Terminal Control Room Operator (VALERO, Job ID: ) Position description: Monitors and supports all load rack activities related to the terminal automation system. Monitors terminal automation systems and advise field personnel when a terminal is in need of attention. Tracks product movements to help reduce the chance of terminal outages or possible overfills. Supports troubleshooting of the terminal control system with the assistance of Information Services and/or field personnel. Etc. Experience and Educational requirements: High school diploma or GED required. Some college background preferred. Minimum two years of experience in terminal operations or industry experience required. Basic knowledge of petroleum products and industry terminology required. Excellent PC skills, with proficiency in Microsoft products required. Effective analytical and problem-solving skills. Relevant tests: The ITAB tests measure ability to deal with procedural knowledge, a crucial aptitude in work environments of the 21 th century.ITAB tests

Applications examples Selection of Executive Assistant (ALCOA, Job-ID: 2410BR) Function description: Must possess excellent computer skills (Microsoft Word, Excel, PowerPoint, Outlook etc.) Responsible for travel planning of the Senior Management Team. The chosen candidate must be able to multi-task, work with confidential information, be self motivated and a team player. Excellent organizational skills required Experience and Educational Requirements: high school degree or equivalent, bachelors preferred. Requires 10 years of general administrative and secretarial experience with emphasis in working independently towards results and goals. Most relevant tests: ITAB tests measure a crucial aptitude for the 21 th century work environment.

Applications examples Retail & Specialty Product Marketing (VALERO, Job ID: ) Position Description: Performs maintenance and upkeep of Retail facilities. Diagnoses and repairs a variety of refrigeration equipment problems to include HVAC, walk-in coolers, icemakers, and freezers. Diagnoses and repairs a variety of merchandising and fast-food equipment problems such as coffee makers, … cappuccino machines, and related equipment. Maintains and repairs building structures, including roofing, plumbing, drywall, flooring, ceilings, glazing, doors, hardware, electrical, and related building items. Experience and Educational requirements: High school diploma or GED required. Vocational training in craft- related areas required. Basic computer skills required. Must have experience in selecting proper tools and repair methods. Etc. Relevant tests: Many jobs require technical troubleshooting (diagnoses and repair, see above), a form of procedural knowledge handling. See: ITAB tests.ITAB tests.

PROGNOST Specifications PROGNOST consists of:  a Bayesian NetworkBayesian Network  a standard battery of tests:  ITAB:IT Aptitude BatteryIT Aptitude Battery  WPT: Wonderlic Personnel TestWonderlic Personnel Test  PCI:Personality Characteristic InventoryPersonality Characteristic Inventory

Measurement claim:procedural knowledge handling (aspect of general intelligence) Number of tests:2 (HTT and BT) Administration time:10 minutes per test (average) Languages:English, Spanish, German, Dutch Test Publisher:CogniMetrics, Inc. Published:March 2004 Price (if purchased separately):see: Reliabilities:HTT (0.88 – 0.97) BT (0.80 – 0.87) Validities: construct validity studies:see: FOR MORE INFORMATION predictive validity studies:currently under way with: - Air Force Research Lab, Brooks City Base; - Navy Directorate Selection and Classification, Washington, D.C. FOR MORE INFO... Download ITAB manuals: Report on construct validity: (paper 066; powerpoint presentation 066). Specifications Standard Test Battery IT Aptitude Battery

Wonderlic Personnel Test Measurement claim: general intelligence Number of tests:1 (50 questions) Administration time:12 minutes Languages:English, Spanish Test Publisher:Wonderlic, Inc. Published:1937 Reliabilities:ranging from 0.82 – 0.94 Validities: construct validity studies:see Manual WPT predictive validity studies:numerous (see Manual WPT). Specifications Standard Test Battery

Personality Characteristics Inventory Measurement claim: basic personality dimensions Number of tests:5 scales, or 12 scales Administration time:25 – 30 minutes Languages:English, Spanish Test Publisher:Wonderlic, Inc. Published: Reliabilities:ranging from 0.82 – 0.87 Validities: construct validity studies:see Manual PCI predictive validity studies:see Manual PCI

Specifications decision support system PROGNOST PROGNOST is a decision-support system build on an expert system shell, a Bayesian Belief Network system. Manufacturer (Belief NetworkNorsys Software Corp. technology and API software):Vancouver, BC. Canada Network development:CogniMetrics, Inc. San Antonio,TX Further software development:CogniMetrics, Inc. San Antonio,TX Further software development includes: Test manager (administering the tests to applicants; transfers test data to database). User interface decision support system (available to counselors to read decision outcomes and to add further information). Report writer

Pricing prognost Software Development$ 22,500 test manager (incl. interface) user interface decision support system API programming report writer Network Development$ 6,500 These prices hold for the development of a network based on the proposed standard test battery and 5 different job categories to be predicted. Modifications of the standard system, for example, additional tests, and/or additional job categories are possible. System development

Pricing standard test battery Two options: 1. Quantity-based License: For example, a license for 100 test administrations of the standard test battery costs: $6,000 + tax. This license can be renewed at any time. Payment in advance. Bulk discounts can be negotiated. 2. Time-limited License: We offer interesting discounts for licenses for a full year of unlimited use of the standard battery. CogniMetrics, Inc. can provide an estimate of cost savings resulting from implementation of PROGNOST as basis of your selection process: direct savings on costs per individual assessment indirect savings as a result of increased productivity, or improved success ratios in training programs as a result in improved validity of the prediction. Tests licensing

Availability CogniMetrics, Inc. can develop the proposed standard system within three months.

Thanks for reviewing this CogniMetrics proposal For further information, Please contact: Dr. Martin J. Ippel (210) 481 – 0261

CogniMetrics, Inc.  2004 IT Aptitude Battery: In 2004 CogniMetrics, Inc. published the IT Aptitude Battery, the world’s first battery of fully interactive tests of human intelligence. The ITAB tests measure ability to deal with procedural knowledge, a crucial aptitude in work environments of the 21 th century. The ITAB tests can be purchased as part of PROGNOST, or separately. For more vendor information, please visit: select.

CogniMetrics, Inc.  2003 PROGNOST: In 2003 CogniMetrics, Inc. developed PROGNOST a method to build customized low-cost decision-support systems for selection decisions. A decision support system is an expert system that uses all available scientific knowledge to predict success of current or future employees for a particular job or training, based on available information (such as their test scores).