CIRTL Network Administrative Meeting Wednesday, September 2, :00-1:00 ET/ 11:00-12:00 CT /10:00-11:00 MT/ 9:00- 10:00PT This meeting will be recorded Telephone Bridge Call-in number: 1 (571) Passcode: # Announcements: Fall meeting in Boston GLHEC SOWs Cross-Network Update Topic 1: Cross-Network Learning Community – “Supply Side” a)In-kind Contributions (Mathieu) b)Needs and Resource Allocation Framework (Goldberg, Spilios)

Cross Network Courses Courses (registration closes today) about 200 registered to date O Diversity in the College Classroom O Service-Learning in STEM Classrooms: Introduction to Pedagogy and Practice O Creating Assessments and Evaluation Plans (Short Course) O Developing a Teaching Portfolio (FULL) O Effective Teaching with Technology (FULL) O Students reading real science: bringing primary literature into the undergraduate classroom (FULL) CIRTL MOOC (registration open, enrollment unlimited): O An Introduction to Evidenced-based STEM Teaching (MOOC)

UW-Madison is submitting an NSF Advancing Informal STEM Learning Exploratory Pathways proposal (due Nov. 3 rd ) We are proposing to take Delta’s face-to-face Improv to Improve Communication and Teaching course and convert it to an online course and assess the efficacy of instruction. We are looking for 2 CIRTL Network institutions to partner with us. We are expecting that each collaborating CIRTL Network campus will: have a cohort of no fewer than four students in the course; instructors will commit to teaching the course at least twice during the duration of the grant; instructors will attend the Facilitator's training at UW-Madison during the summer immediately before teaching the course. We are considering using CISCO telepresence rooms to teach the course.

CROSS-NETWORK LEARNING COMMUNITY – SUPPLY SIDE Leadership Team, CNOG, Greenler, Goldberg/Spilios

Purpose of Discussion – Summer/Fall 2015  Teaching-as-Research on the CNLC after 3 years  Responsive to charge - Spring 2015 Network meeting  Plan  Summer 2015 (LT, CNOG) – Data collection and review  August Network meeting – Overview of current state  September Admin meeting – “Supply Side” discussion  September Network meeting – “Demand Side” discussion  October In-Person meeting – Plan for moving forward

Full Semester (15+ hours) Shorter Term (4-8 hours)Total AY AY AY AY Total CIRTLCastCoffee Hour Journal Club TAR CapstoneTotal AY AY AY AY Total CIRTL Network Participation, Overview CIRTL Network Completions in Cohort-based CN Events (Full Courses, Short Courses) CIRTL Network Participations in Drop-In CN Events (CIRTLCasts, Coffee Hours, TAR Capstone, Journal Club) Note: Not at full capacity; impact can be increased without further cost ≈ 100 / yr ≈ 800 / yr

University0.67 FTE instructor (2 Cat 1 / 3 semesters) CIRTL Central 1.75 FTE CNOG Annual Cost ( ) Annual Benefit University (average) Cohort-based 8 students Drop-in44 students Range across Universities3 – 17 students

Benefit High-moderate and high engagement levels in TD during the doctoral program had the most positive effects on how frequently early career doctorates’ used certain teaching strategies; each of these engagement levels had a significant, positive effect on teaching behaviors in all six college teaching components. Low moderate TD engagement also had effects on multiple teaching components, and even low TD engagement had a positive effect on how frequently early career doctorates used different techniques to assess student learning. Formal courses were the only format to have an effect on doctorates’ teaching practices as a long-term effect of TD participation. This finding is consistent with those in the rest of the study, which generally found that participation in intensive TD programs or formal courses yielded the most positive effects. - Longitudinal Study of Future STEM Scholars Connolly, Lee, Savoy 2015 (in prep) Finally, we found that the amount of teaching experience during the doctoral program had a consistent, positive effect on doctorates’ self- efficacy beliefs and teaching behaviors across all six of the college teaching components we assessed. Clearly, the influence of applied teaching experience in preparing doctoral students and early career academics cannot be overlooked, and must be integrated with well- designed TD program offerings.

Impact of Network Expansion ?  Expansion Team: On-boarding plan to provide CNLC academic opportunities to new members in the first year *before* new partners contribute fully  There will be a huge change in potential Cat 1 supply in two years  Unknown whether will be an equal increase in demand, but certainly a substantial increase  Forecast: More Cat 1’s total, but not to scale Less Cat 1’s per institution More demand on Cat 1 offerings

Findings re CNLC Policies  Broad support for Cat 1 contributions including  Network leadership (from CNOG) Service on major committees and operations groups Currently would be 15 Cat 1’s, 9 universities  On-boarding initiatives (from Expansion Team)  Strong affirmation for great flexibility in Cat 1 academic contributions - make use of the diverse assets of the Network;  But … “CNLC should support production of Associates and Practitioners” => core “curriculum”, including MOOC

Expanding in-kind contributions (type here or in the chat box) Pros Cons

 Based on IPs and SOW data, CNLC opportunities are not integrated into most Associate and Practitioner pathways  CNLC is “on the side”, an additional opportunity  ISU has factor 3 higher participation; deep integration Connection of CNLC with Local Goals

 Why are cohort participation levels not at capacity –  Multiple conjectures, but we don’t really know why Quality ? NB: Not evident in evaluations of those who complete Needs are satisfied locally ? NB: CNLC College Classroom had highest enrollment NB: MOOC enrollment from Network ≈ 300 Online Barrier ? Synchronous vs. Asynchronous ? Awareness ? Are we actually connecting with 100,000 STEM graduate students Integration and Promotion ? Not integrated into local learning communities Key Questions – Demand Side

BU CIRTL Needs Assessment and Resource Allocation Framework Bennett Goldberg, co-leader, Physics Kathryn Spilios, Biology September 1, 2015

BU CIRTL Why CIRTL should implement a process of needs assessment and resource allocation CIRTL is a complex network of 21 institutions growing to 50+, each different. – A needs assessment collects and combines data, identifies gaps, and creates a unified picture of network needs that can then be analyzed, prioritized and resourced. CIRTL has developed a large and growing number of cross-network activities – A needs assessment maps cross-network activities to CIRTL outcomes, as well as maps cross- network activities to local learning community needs. CIRTL institutions would have greater commitment and buy-in if they knew more about the overall network, its operations and outcomes. CIRTL Network expansion will require a new set of cross-network activities 15

BU CIRTL Framework Principles Transparency – the process, assessment matrix, and institution’s contributions will be published on the CIRTL website. Inclusiveness – the process includes all voices at all steps. Data driven – the outcomes of all network activities, opinions of activity providers and participants will drive decisions through actionable data.

BU CIRTL Sequence of actions in needs/resource framework PURPOSE: to reach a collective (not necessarily a consensus) understanding and prioritization of the needs of the CIRTL network 17 Needs assessment, resource costs and activity prioritization Resource description, categorization, and valuation. Mapping resources to needs Data collection and feedback CURRENT RESOURCES: the time, effort, and skill set of the Institutional Leaders and their local teams who contribute to cross-network activities MAPPING: uses existing CIRTL Network model of category contributions; allows local institutional choice and optimized collective choices DATA: outcomes of Network goals and the value to those providing the activity. Data should be actionable - an activity is so valuable we offer it more often; or an activity is so valueless we decide not to resource it.

BU CIRTL Details of the Needs/Resource Process 1.Pre-meeting gap analysis, cross-network activity suggestions and cost reporting by institutions: prior to in-person meeting each institution will perform a gap analysis, list their top ten cross-network resources needs, and assign a resource cost to these activities. CIRTL Central will help us make a gap analysis tool combining the IP and CN existing matrices. The gap analysis data will be collected and collated by CIRTL central, averaged and modified with CNLT/CNG experience and reported back to the network as a single compendium in advance of the in-person discussions. 2.Needs assessment using a matrix that lists activities as a function of their occurrences, costs, value to network outcomes, and data on activity from prior year to ultimately determine prioritization. The tables will be prepopulated with data, and will become more populated year by year. Small group 60 min sessions with pre-populated large white boards and sticky notes; Following breakout, all 6 whiteboards are rolled into main meeting room, teams have 2-3 minutes to present while everyone crowds around; and we collect and respond to general questions using polleverywhere. 3.Cost – Priority matrix is built from the needs assessment, where each activity has a cost in human resource and a priority level representing value to the network. Friday morning the cost-priority matrix is displayed discussed. The total number of activities is explored with respect to the fixed resources of the network – 42 Cat 1 and 84 Cat 2 units. Some activities are moved. 18

BU CIRTL 4.Resource allocation and mapping to needs using a two-step process. All institutions fill out an online rank ordering of their choices of all category one and two activities. We then maximize the overall number of first and second choices, and minimizes the sum total of all choice rankings. We propose doing this on Friday morning as a test run, and instantly displaying the resource allocation to the community as a test of the process and to open discussion. In actuality, institutions will have 1-2 months to confer with their local colleagues and make the choices. 19 Details of the Needs/Resource Process Cost of resource Value or prioritization CAT 2 CAT 1 Don't Do

BU CIRTL CIRLT Needs Assessment Matrix Category of activityActivity# of occurences Person-hours per instance; or high, med, low Outcomes data from prior year Impact of activity Rating by providers Rating by participants Prioritization, 1,2,3 or 4 Cross-network courses College Classroom Research Mentor Training 2high; hrs; Cat 1; 12; high value med high2 Cross-network coffee hours Cross-network committees Forum hosting and committee Cross-network operations group IT committee Expansion of the network Mentoring Site visits Regional networks Hosting in-person meetings Example – matrix to categorize, examine the data and cost, and prioritize activities

BU CIRTL What do you really like about this needs/resource process? What are your main concerns about the process?