Industrial Technologist’s Toolkit For Technical Management (ITTTM) Assessment Overview: Orientation Tutorial Presentation 4 1. Explanation of this presentation, context 2.Courseware process structure, portfolio 3.Assessment levels, types, outcomes 4.Portfolio outcome assessment matrix (POAM) 5.Review, broader ITTTM relationships Dr. John W. Sinn, Professor, Bowling Green State University

Explanation Of This Presentation, Context This is the fourth of four related presentations used to help you understand and use ITTTM courseware. The four ITTTM orientation tutorials can help you see relationships in core knowledge and technical management as part of ITTTM. The four related presentations are: 1. ITTTM Introduction, Overview 2. ITTTM Content Overview 3. ITTTM Process Overview 4. ITTTM Assessment Overview

Documentation Tools, Lean Data Tools, Six Sigma Service, Tools Manufacturing, Non-Manufacturing Synchronous Tools, New Product Development Applied Research, Project Team Cultural Change, Industrial And Academic Core Values Quality Management System Product Realization Measurement Analysis Improvement Resource Management Analysis, Global Views, Values Integration, Pragmatic Field Information Disciplined Knowledge, Research And Development Interaction, Problem Solving, Decision Making Communication, Internal, External, E-commerce Management Responsibility ISO Total Quality, Baldridge Leadership Context, Continuous Improvement Professional Outputs Primer Inputs Model System: Industrial Technologists’ Toolkit For Technical Management (ITTTM) Explanation Of This Presentation, Context

Six sets define core, overview 42 tool courseware system:  Primer Tools  Cultural Tools  Data Tools  Documentation Tools  Service Tools  Synchronous Tools  Tool set is about a course, used as base for assessment  Electronic structure is flexibile for strong communication  MS Word Office Suite tools aid in assessment Explanation Of This Presentation, Context Portfolio development, assessment, six tools……… startup activity in course is where assessment begins

Courseware forms have important portfolio structure:  SDA’s, RCA’s, documents, grows portfolio around project  Initial individual responses by each become grand forms  Format guides project, details structure of portfolio  Analysis, reflection, writing are introspective assessment  RCA’s evolve with project, iteratively document progress  SDA’s, content assessments of tool applications in project  Systematic, disciplined, structure for communication RCA and SDA format as key assessment vehicle……… Courseware Process Structure, Portfolio

1. Regular Critique Assessment (RCA’s) 2. Standard Dedicated Audit (SDA’s) 3. RCA’s Grows, SDA Different Each Tool 4. Grow Knowledge, Assess Project Work 5. Analyze, Synthesize, Question Project 6. RCA’s Improve, Format As Portfolio RCA’s, SDA’s help team assess project progress, improve based on ITTTM tools, systematic portfolio development Courseware Process Structure, Portfolio

ITTTM courseware facilitates team project with technical management content applied for assessment in portfolio:  Course format grows technical management system  Forms require team analysis, management via PPDPOA  Content areas researched, detailed, “do” project processes  Form questions assess ISO/QS 9000 project relationships  Work posted by each is one content analysis, interpretation  At phase review enhanced SDA’s as grand form, portfolio  Phase portfolio documents progress compared to startup All available electronically for portfolio assessment……… Courseware Process Structure, Portfolio

1. Tools Completed, Assessed, Improved 2. Phase I Portfolio Total Long Form Work 3. RCA’s, SDA’s Done Continuously In Each Tool 4. Growth In Knowledge, Team, From Assessment 5. Total Long Form, Phase II, Final Portfolio 6. Long Form Does Project, Improvement Assessed Courseware facilitates a long form report, portfolio developed with the SDA and RCA forms, tools……… Courseware Process Structure, Portfolio

1. Short Form Power Point Grown Iteratively 2. As tools, Phases Done, Short Form Evolves 3. Short Form Is An Executive Summary 4. Explains Growth In Knowledge, All Functions 5. Uses Slides, Not Duplicating Long Form 6. Short Form Stands Alone, No One To Explain…… Team, project facilitates a short form, Power Point compatible with portfolio, to explain long form……… Courseware Process Structure, Portfolio

ITTTM interfaces with various assessments, outcomes:  Self assessment, goals set by individuals at start, various points along the way to guide progress  Course assessment, course outcomes, objectives provide assignments reflecting curriculum  Collegial, university assessment, broad-based values, curricular requirements relating to general education  Professional assessment, advisory groups, professional bodies to validate, certify, accredit skills, knowledges Assessment Levels, Types, Outcomes ITTTM electronic courseware interfaces, documents various technical management outcomes for assessments.

1. Syllabus Identifies Outcomes, Content 2. Content Leads To Rubrics, Metrics 3. Learning Activity Defined As Matrix 4. Syllabus Matrix Blend Assessment, Courseware 5. Matrix In Syllabus Guides Portfolio 6. Content Synthesis, Feedback, Portfolio Assessment Levels, Types, Outcomes ITTTM courseware is course level assessment vehicle, via syllabus, statement of anticipated student outcomes

Course outcomes drive the assessment process, defining key deliverables, syllabus focus. Examples in a course include: Assessment Levels, Types, Outcomes 1.Change, improvement and people. Change, improvement in services is studied, focused on “human issues”. 2. Global, local quality, technology service relationships. Student teams do service projects focused on assessment. 3. Auditing ISO/QS 9000 services. ISO/QS 9000 systems provide a platform, documentation systems for audit. Usually three to five course outcomes identified, content expands the definition of what actually gets studied. All of this is part of the typical syllabus related to ITTTM.

Course outcomes are further defined in the POAM, a matrix assessment system with all desired requirements:  Course outcomes stated in achievable terms  Content tied to course outcomes, relationally  Values reflecting broader educational, professional norms  All basic, general requirements to be met The POAM matrix provides a mechanism to assess work done by teams, and give feedback for improvement at the course level, part of the typical syllabus related to ITTTM. POAM is shown in next slide with all elements as original. Portfolio Outcome Assessment Matrix (POAM)

General Information: The POAM is a vehicle designed to provide instructor feedback with each posting cycle for work (approximately 2 weeks in a standard 16 week semester) submitted by teams and individuals. The POAM is tied to course outcomes in the syllabus, with categories providing anticipated rubrics and functions to help guide all in submission of critiques each posting cycle, moving toward a phase portfolio, under development. The POAM is also tied heavily to courseware use, particularly SDA’s and RCA’s from application sections of each tool right after content coverage. Outcome 1: Change, improvement and people. Category general focus is disciplined pursuit and organization of literature in the quality field assembled to support and develop ideas. Responsible quality systems applications and knowledge are reflected in literature assembled. Cultural relationships considered, alongside global views and various values reflect broad quality system principles and knowledge to build on tool content emerging and being understood. Rubrics showing outcome, general assessment, reflected in team portfolio, 2 RCA’s, QMS, possible sub-total of 7/14 points.PossibleActual o RRAA detailed, analysis/synthesis, QMS disciplined approach, robust assessment questions, chat discussion based on posted agenda. o Cultural, global view, change, reflect tool content, evolving QMS in each critique, cumulatively becoming portfolio of all project work. o IRABCA grand forms show all work compiled by researchers, contributions by all reflecting toolkit content, project focus. o IRABCA source diversity, professional views connected integrating, reflect and apply professional field in appropriate SDA’s. o First as critique, then cumulatively as phase portfolio, IRABCA, RRAA articulates, integrates, synthesizes to address project/team goals. 2/4 1/2 2/4 Other feedback to improve/transition to phase portfolio, reflect outcome 1, ISO elements, broader outcomes: Outcome 2: Auditing ISO/QS 9000 services. Category general focus is on organizing and documenting knowledge as findings based on a systematic search for answers and relationships leading to a structured understanding of how to change and improve product and people. Data and documentation-driven tools are completed by individuals and collectively as team to reflect solutions to problems in project, moving toward variation reduction, improvement. Rubrics showing outcome, general assessment, reflected in team portfolio, 4 SDA’s, QMS, possible sub-total of 7/14 points.PossibleActual o Objective decisions, recommendations from SDA data, findings documented as applications/reflections/conclusions in matrices, tables. o SDA’s grow QMS knowledge, reflecting ISO rubrics/systems as product realized in team project becoming grand forms, “best practices”. o All specific content outcomes shown in syllabus are addressed and reflected in team SDA’s, compiled and contributed by all. 2/4 3/6 2/4 Other feedback to improve/transition to phase portfolio, reflect outcome 2, ISO elements, broader outcomes: Outcome 3: Global, local quality and technology service relationships. Category general focus is wise use of resources, well managed to solve problems, make cost effective decisions for improvement alongside various values reflected within broad quality system principles and knowledge being grown. Effective communication observed and documented in web-based course infrastructure inherent in work and methods developed by individuals and team. Rubrics showing outcome, general assessment, reflected in team portfolio, 2 RCA’s, QMS, possible sub-total of 7/14 points.PossibleActual o PPDPOA shows QMS, team focus emerging/evolving, responsibilities for all work, all rotating through various RCA compiling functions. o Solid format, no typos, fonts/borders same, all boxes done in forms from compiler improved individual researcher work in forum. o TPPMA grand form intentionally measures and reflects improvement as internal and external numerical analysis, assessment questions. o Electronic chat discussions, threads, work posted in team forum reflect tips from front of course, numerical ratings/names in portfolio. o Power Point short form grown incrementally, done by all on rotating basis, to explain use of courseware, project, key findings as QMS. 2/4 1/2 2/4 Other feedback to improve/transition to phase portfolio, reflect outcome 3, ISO elements, broader outcomes: Individually scored items and points awarded in the scorecard based on individual team member postings in forum during the first half of posting cycle. o SDA’s completed by each researcher in forum, contributions reflect different iteration of tool/SDA content, project focus. o RCA/SDA’s completed/presented by each in forum, compiled by researchers, equitable contributions by all show tool/project focus. o Overall quality and completeness of each SDA/RCA is done by each researcher to aid all as compilers, build QMS. 3/6 points Team Assessed:Assessor: JWSWork Assessed:Date:Team Points: 30/60

Portfolio Outcome Assessment Matrix (POAM) General use of POAM stated, broader context defined, part of ITTTM Top and bottom of form, is important general information, and how to assess persons on team individually as needed: General Information: The POAM is a vehicle designed to provide instructor feedback with each posting cycle for work (approximately 2 weeks in a standard 16 week semester) submitted by teams and individuals. The POAM is tied to course outcomes in the syllabus, with categories providing anticipated rubrics and functions to help guide all in submission of critiques each posting cycle, moving toward a phase portfolio, under development. POAM is also tied heavily to courseware use, particularly SDA’s and RCA’s from application sections of each tool right after content coverage. Individually scored items and points awarded in the scorecard based on individual team member postings during first half of posting cycle. o SDA’s completed by each researcher in forum, contributions reflect different iteration of tool/SDA content, project focus. o RCA/SDA’s completed/presented by each in forum, compiled by researchers, equitable contributions by all show tool/project focus. o Overall quality and completeness of each SDA/RCA is done by each researcher to aid all as compilers, build QMS. 3 points Team Assessed:Assessor:Work Assessed:Date:Team Points: Work done by individuals, posted in discussion area, assessed as separate points General team, project assessment information, total points awarded

Portfolio Outcome Assessment Matrix (POAM) Outcome 1: Change, improvement and people. Category general focus is disciplined pursuit and organization of literature in the quality field assembled to support and develop ideas. Responsible quality systems applications and knowledge are reflected in literature assembled. Cultural relationships considered, alongside global views and various values reflect broad quality system principles and knowledge to build on tool content emerging and being understood. Rubrics showing outcome, general assessment, reflected in team portfolio, 2 RCA’s, QMS, possible sub-total of 7 points.PA o RRAA detailed, analysis/synthesis, QMS disciplined approach, robust assessment questions, chat discussion based on posted agenda. o Cultural, global view, change, reflect tool content, evolving QMS in each critique, cumulatively becoming portfolio of all project work. o IRABCA grand forms show all work compiled by researchers, contributions by all reflecting toolkit content, project focus. o IRABCA source diversity, professional views connected integrating, reflect and apply professional field in appropriate SDA’s. o First as critique, then cumulatively as phase portfolio, IRABCA, RRAA articulates, integrates, synthesizes to address project/team goals Other feedback to improve/transition to phase portfolio, reflect outcome 1, ISO elements, broader outcomes: Outcome is stated, context explained Rubrics, specific areas, items assessed, described as courseware, project portfolio relationships Possible (P) points for rubric, outcome identified and actual (A) space for assigned value Feedback for improvement potentials is explained here Each outcome, defined in POAM, provides response from instructor, assessment to requirements, and points earned:

Portfolio Outcome Assessment Matrix (POAM) ISO Quality Elements, BGSU Core Values, Metrics Synthesized As General Outcomes Context ISO Quality Element BGSU Core Values, College/University Anticipated General Outcomes, Integration And Articulation, For Assessment Purposes Quality management system Analysis, cultural, global views, values Cultural relationships considered, global views and various values reflected within broad quality system principles and knowledge documented in a team portfolio, all presented in the context of a team-based project. Management responsibility Integration; pragmatic field information applied Disciplined pursuit and organization of literature in the quality field assembled to support ideas under development. Responsible quality systems applications and knowledge are reflected in literature assembled. Product realization Disciplined knowledge; R & D; problem solving Organizing and documenting knowledge as findings based on a systematic search for answers and relationships leading to a structured understanding of how to change and improve product and people. Measurement analysis, improvement Communication, decision making, systems Communication observed and documented in web-based course infrastructure as part of broad quality systems inherent in work and methods developed by individuals and teams. Resource management Interaction, articulated knowledge, organized improvement Wise use of resources, well managed to solve problems, make cost effective decisions for improvement, all reflected in quality systems knowledge and applications documented in a team-based portfolio. Context of course outcomes, broad relationships in ITTTM, reflecting ISO quality elements, BGSU core values, to derive anticipated general programmatic outcomes, rubrics………

Review, Broader ITTTM Relationships This was the third of four presentations to help understand, use ITTTM courseware. The four presentations in the ITTTM Orientation Tutorial folder include: 1.ITTTM Introduction, Overview 2.ITTTM Content Overview 3. ITTTM Process Overview 4. ITTTM Assessment Overview After viewing these, best in order, you can review or use the courseware provided in the ITTTM 42 Tools folder. Give feedback by using forms in the ITTTM Research Folder. Contact the author at