1. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California ONR/NETC Planning Meeting 18 July, 2003 UCLA/CRESST, Los Angeles, CA ONR Advanced Distributed Learning Eva L. Baker Intellectual and Social Capital: Opportunities  2003 Regents of the University of California UCLA CRESST National Center for Research on Evaluation, Standards, and Student Testing

2. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 1 Intellectual Capital Fund of knowledge, skills, and cognitive propensities— analogous to ROI CRESST focus for ONR –Content understanding –Problem solving –Teamwork –Communication –Metacognition

3. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 2 Intellectual Capital Content Understanding Problem Solving Teamwork and Collaboration MetacognitionCommunication Learning

4. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 3 Social Capital Team performance and skills Networking and information sharing Trust Efficacy (can do) Adaptive communication Environmental stress management Leadership roles Effort

5. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 4 Organizational Development and the Creation of Social Capital Both intellectual and social capital needed Social capital is the sizzle Social capital interacts with individual differences Individual differences in the past has been conceived as general traits or traits that come into play on certain occasions

6. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 5 Trust EfficacyNetworks Effort Adaptive Communication Learning Organization Leadership Social Capital Teamwork

7. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 6 Measurement of Social Capital In institutions (e.g., schools), as a component of effectiveness In programs, as the context and outcome of quality instruction In organizations, to improve effectiveness

8. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 7 What’s New? Not trait-, state-, personality-focused Componential and therefore instruction-sensitive Interactive with intellectual capital formation Measurement should be iterative and focus on bottom-up cycles of reporting Organizations put self forward like awardees

9. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 8 Program Design analytic framework for evaluating social capital, development, duration, predictive elements Contrast high-functioning with low- functioning systems to discriminate key predictors Create measures for self-assessment and improvement of individuals, teams, and organizations

10. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 9 Components of Social Capital Individual behaviors (motivated, persistent, resilient, altruistic) Individual meta processes (intellectual and emotional awareness and control) Knowledge of how to make social systems work

11. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 10 Measurement of Team Performance –Knowledge maps for team cognition Content knowledge –Team process measured by pre- specified messages or online text analysis, e.g., latent semantic analysis E-Rater (ETS) –Automated generated and scored After- Action Reviews Opportunities

12. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 11 Opportunities (cont.) Dynamic Testing of Team Performance –Vygotsky (ZPD), feedback (O’Neil), help seeking (Webb) Cognitive Demands of Psychomotor Tasks (rifle marksmanship and SEALS) –Brief SEALS August 14, 2003, for possible joint program

13. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 12 Opportunities (cont.) Comparative Testing of Authoring Systems – Top Down Ontology, task analysis, experts –Bottom Up “Repurposing” simulations – Dependent measures Outcome of instruction is learning Outcome of assessment is reliable and valid decisions Online Analysis and Reporting

14. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 13 Psychometrics of Simulation Psychometric Issues –Validity and accuracy –Design economies –Technical comparability and equating –Estimation of difficulty –Generalization and characteristics across topics, contexts, prior knowledge –Sampling –Scaling –New functional approaches to norming Online administration, diagnostic/ diagnostic/ scoring, reporting prescriptive prescriptive (AAR) Individual Training Collective Training

15. Baker ONR/NETC July 03 v.4  2003 Regents of the University of California 14 Opportunities (cont.) Standardizing Quality Alignment (R&D) Linguistics Search capacity Evaluation (program assessment) 5 Vector mixed models