The Effective Management of Student Teams Using the CATME/Team-Maker System: Practice Informed by Research Capstone Conference 2010 Workshop 1 Everyone please make your wireless connections.

The development team is multi-institutional and multidisciplinary. Richard Layton, Rose-Hulman Misty Loughry, Georgia Southern Matt Ohland, Purdue University Hal Pomeranz, Deer Run Associates Lisa Bullard, NC State Cindy Finelli, U Michigan Rich Felder, NC State Doug Schmucker, Trine University

By the end of the workshop, participants should be able to… List key findings from team-formation and peer-evaluation literature including recommendations for distributing skills and attributes formative assessment and interventions summative assessment Use Team-Maker to create, administer, and evaluate a team-formation survey Use CATME to create, administer, and evaluate a peer evaluation survey

We have a 3-hour agenda with breaks about every 40 minutes Min.Activity 10 :Introductions, workshop goals, wireless connections 15 :Activity: Problems encountered in assigning teams 15:Activity: Connecting your practices to scholarship 5 :Hands-on: Setting up Team-Maker 10 :—Break— 10 :Hands-on: Practice using Team-Maker 15:Demonstration: Evaluating results using Team-Maker 20 :— Break — 10 :Activity: Problems encountered in peer evaluations 15:Activity: Connecting your practices to scholarship 10 :Hands-on: Practice using CATME 10 :— Break — 10 :Hands-on: Practice using CATME 15 :Demonstration: Evaluating results using CATME 10 :Discussion, feedback, assessment

What problems have you encountered in assigning teams? Individually Think about team-forming views / practices Write an outline of your practices and what problems you’ve encountered In pairs Discuss, reach consensus on two or three most important items As a group Relate to Team-Maker features / gaps

Instructor selection is better than self-selection and random assignment What does your experience tell you? How do you address student concerns with instructor selection?

There are negative consequences of ignoring race,gender, and ability in forming teams What does your experience tell you? How do you address student concerns when using these criteria? Lack of diversity can make it impossible to consider race and gender Schedule compatibility may surpass all of these

Set up a practice account to use Team-Maker Connect to test server at “catme.org” A copy of the real-world system Test data won’t appear in the real system Real account set up separately Go to “request faculty account” we will assign the password “catme” for all Download sample CSV file from catme.org/sample.csv to your machine (desktop or "My Documents")

Break Check the time Please be back in 10 minutes

Team-Maker Practice Login to your faculty account Create and issue a survey (fake student CSV) The survey as the student sees it Login as one of the students in the survey you created For your survey, responses of fictitious students are auto-generated View survey results Form teams

Team-Maker helps you make decisions about how your teams are formed Review team make-up Look for extreme cases Reality check Reconsider your priorities Adjust weights Re-form teams Repeat as necessary

Break Check the time Please be back in 20 minutes

What problems have you encountered by using or not using peer evaluations? Individually Think about peer evaluation views/practices Write an outline of your practices and what problems you’ve encountered In pairs Discuss, reach consensus on two or three most important items As a group Relate to CATME features / gaps

Students need guidance and structure on task and process What does your experience tell you? How do you address student concerns when using these criteria? Training on providing feedback Multiple ratings First rating low stakes

Students must be accountable but safe in rating their teammates What does your experience tell you? If ratings have no consequences, raters will try to please the audience Confidential vs. anonymous Faculty judgment is critical

CATME Practice Use same faculty account Create and issue a survey (to previous teams) Teams can be loaded w/o using Team-Maker The survey as the student sees it Login as one of the students in the survey you created For your survey, responses of dummy students are auto-generated

Break Check the time Please be back in 10 minutes

CATME helps you discover what might be happening in your teams View feedback to students and faculty Raw scores / comparisons Exceptional conditions Manipulator (red) Overconfident (yellow) Underconfident (yellow) High performer (green) Low performer (yellow) Personality conflict (red) Clique (red)

Discussion Individually Think about advantages, disadvantages, potential research uses In pairs Compare, prepare to share with group As a group Discuss

Workshop Assessment What features do you like about the system? What missing features would make the system more useful to you or others? Do you think you will use the system? Why or why not? Any suggestions for improving the workshop?

Instrument / NSF Support Web-based instrument / tools found at This material is based upon work supported by NSF Award “Design of a Peer Evaluation Instrument that is Simple, Reliable, and Valid” and , “SMARTER Teamwork: System for management, assessment, research, training, education, and remediation for teamwork”