Data Collection Through Analysis in the U. S. Army’s Enlisted Common Soldier Task (ECST) Project Johnny J. Weissmuller Metrica Bob Gosc Institute for Job & Occupational Analysis

IMTA - 11 Nov 1999Data Collection Though Analysis 2 Background F The Institute for Job and Occupational Analysis (IJOA) was incorporated in 1993 as a not-for-profit agency pursuing scientific and educational goals in OA.

IMTA - 11 Nov 1999Data Collection Though Analysis 3 Background F Although IJOA undertakes operational endeavors in the OA community, the goal is to develop better methods and tools -- and spread these (not project data) to academic, government, and private sector users around the world.

IMTA - 11 Nov 1999Data Collection Though Analysis 4 U.S. Army Enlisted Common Soldier Task (ECST) Project F Goal: Survey all Enlisted Grades, across diverse Military Occupational Specialties (MOSs), to identify COMMON training needs F Output: to support the Army-wide Critical Task Selection Panel in revising the U.S. Army’s Soldier’s Handbook

IMTA - 11 Nov 1999Data Collection Though Analysis 5 ECST Technological Challenges F 847 Items, 47 Duties, 20,000 cases worldwide F Army’s need for new survey program features: –Sort by Duty Importance –Screen by Skill Level –Modify by Position F Using IJOA’s “Author” system for generating this AND 8 clones for TE surveys

IMTA - 11 Nov 1999Data Collection Though Analysis 6 Stages in the Occupational Analysis (OA) Process  Inventory Development  Survey Preparation  Field Administration  Data Aggregation  Data Processing  Analysis, Report Writing, and End-User Assistance

IMTA - 11 Nov 1999Data Collection Though Analysis 7 Inventory Development F Word 97 used to integrate task lists from diverse sources F Word 97/ used to coordinate around the world (track revisions) F Word 97 used to note target skill levels, source citations, etc.

IMTA - 11 Nov 1999Data Collection Though Analysis 8 Word 97 to HTML Format

IMTA - 11 Nov 1999Data Collection Though Analysis 9 Survey Instrument Preparation F Word 97 documents saved in Hyper Text Markup Language (HTML) format F Converted Task & Duty Statements using HTM2Flat F Converted Collateral data using HTM2Data F OASurv Electronic Survey Administration disks ready 15 minutes after receiving “GO” .

IMTA - 11 Nov 1999Data Collection Though Analysis 10 Word 97 to HTML Format

IMTA - 11 Nov 1999Data Collection Though Analysis 11 HTML Format to atCODAP

IMTA - 11 Nov 1999Data Collection Though Analysis 12 HTML Format to OASurv/CODAP/GenSurv

IMTA - 11 Nov 1999Data Collection Though Analysis 13 Survey Instrument Preparation F The changing abilities of the OASurv Program F Allowing many users on a single diskette requires effective virus checking in the field F Capturing a “computer inventory” to determine what computer platforms in what quantities are in the field

IMTA - 11 Nov 1999Data Collection Though Analysis 14 OASurv - The Survey Engine Upgrades & Support Programs F Extract “Target Skill Level” for OASurv & reports (HTM2Data) F Format “Target Skill Level” information for new OASurv Logic (PlusOne) F Rate Duty Importance, sequence & screen tasks based on those ratings

IMTA - 11 Nov 1999Data Collection Though Analysis 15 HTML Format to OASurv/atCODAP Data List

IMTA - 11 Nov 1999Data Collection Though Analysis 16 Field Administration F Master Disk sent to a commercial duplicator who copied, labeled, and returned 20,000 disks within 72 hours. F Disks shipped to 60 locations worldwide. F APOs Express Mail where available and Priority Mail for others. F FedEx for normal packages

IMTA - 11 Nov 1999Data Collection Though Analysis 17 Field Administration Returns - Job Incumbent Surveys  19,600 surveys distributed across all components  7,689 disks returned  6,682 usable cases returned  Percent of distributed surveys returned Active 67.27% Guard 23.71% Reserves 26.76%

IMTA - 11 Nov 1999Data Collection Though Analysis 18 Field Administration Training Emphasis (TE) Surveys

IMTA - 11 Nov 1999Data Collection Though Analysis 19 Data Aggregation  Upload Spread across many PCs  Combined Respondent Data, Logs, & Computer Inventory  Separated & Linked Write-In-Text (*.WIT)  Later employed an automated reader

IMTA - 11 Nov 1999Data Collection Though Analysis 20 Data Processing Data Processing was accomplished by atCODAP , a commercially available PC version of the CODAP system – i.e., the Comprehensive Occupational Data Analysis Programs. Microsoft Access  data bases and reports, along with other products were prototyped using partial data as “status reports” four times prior to the final cutoff of survey returns. All final atCODAP products, including Microsoft Office  export products from Access , were generated within 96 hours after data cutoff. This included subsampling for 72 Training Emphasis Inter-rater reliability analyses and job descriptions for 72 groups reporting annual frequency and percent members performing.

IMTA - 11 Nov 1999Data Collection Though Analysis 21 Analysis, Interpretation, Report Writing & Aiding End-Users atCODAP Programs and reports were used as “Data Reduction Tools” and “Audit Tools” Summary Data Vectors were exported to Access using the OADecks program Tasks & Duty Affiliations were migrated to Access Access was used to produce “Reference Reports” Access was used to produce “Focus Reports” Access Forms were used to interactively capture decisions of Army Researchers and finally, of top Army policy makers.

IMTA - 11 Nov 1999Data Collection Though Analysis 22

IMTA - 11 Nov 1999Data Collection Though Analysis 23

IMTA - 11 Nov 1999Data Collection Though Analysis 24

IMTA - 11 Nov 1999Data Collection Though Analysis 25 Conclusions F Required Technology is “Commercial Off- The-Shelf” (COTS) or Government-owned software F Logistics have been optimized for rapid, large-scale usage F Psychometric & Human Factors Issues have been addressed

IMTA - 11 Nov 1999Data Collection Though Analysis 26 F Any Questions?