Panelist Orientation: TAACCCT Grant Program July 2012 SGA/DFA PY 11-08 U.S. Department of Labor Employment and Training Administration

Welcome to Workforce3 One!

Practice In the Chat Room, please type the name of your organization, your location, and how many people are attending with you today.

How To Participate in a Poll

Access to Webinar Resources Recordings and transcripts are available within 2 business days after the event. Access to Webinar Resources

Presenters Melissa Abdullah, Grants Management Specialist , ETA Robin Fernkas, Director of the Division of Strategic Investments, ETA Steve Rietzke, Workforce Analyst in ETA Sharon Leu, Workforce Analyst in ETA.

Welcome to Paneling! The following topics will be covered during this presentation: Paneling Overview TAACCCT Funding Opportunity Description Evaluation Criteria How to Panel/Review Applications Panelist Rating Workbook

Orientation Materials Solicitation for Grant Applications (SGA) Electronic Copies of Assigned Proposals Today’s PowerPoint Presentation Program Tip Sheets Panelist Excel Workbook Conflict of Interest Form/Non-Disclosure Statement Security of Proposal Form Assigned Panel Information Deliberation Schedule POC: Melissa Abdullah 202-693-3346 (abdullah.melissa@dol.gov)

Role of Panelists Commit to thoroughly reading and scoring each proposal prior to the deliberation calls which includes: Review of the evaluation criteria. Review of each proposal based on the evaluation criteria and assigning a numerical rating for each criterion/sub-criterion. Documentation of your weakness statements for each criterion/sub-criterion to establish a clear relationship between the score awarded and the weaknesses identified. Finalize weakness statements as a result of deliberation call. Submit the Excel Workbook for each application by email to the deliberation specialist within 24 hours after the deliberation call.

Role of the Deliberation Specialist The Deliberation Specialist will: Ensure panelists receive a complete package of materials. Answer questions and address concerns in a timely and professional manner. Facilitate calls to ensure a fair deliberation process. Ensure panel members are not pressured to change their ratings. Provide timely feedback to panelists if written evaluations are incomplete.

Role of PEP, Inc. Performance Excellence Partners, Inc. (PEP) has been retained by DOL to identify and procure panelists. You will be working for PEP to perform the panel deliberations. PEP will provide information by email on invoice instructions and payment. Please direct questions to: Linda Metchikoff-Hooker at PEP at dolpanelists@pep8a.com 714-374-1140 (office), or 714-374-1120 (fax)

Division of Strategic Investments Robin Fernkas Division of Strategic Investments

TAACCCT Funding Opportunity PY 11-08 Overview – What is TAACCCT? Core Elements Award Information Eligible Institutions TAA for Workers Program Application Outline

Overview – What is TAACCCT? $2 billion for FY 2011-2014 ($500 million annually) Expand and improve ability to deliver education and training programs Suitable for TAA-eligible workers Lead to employment in high wage, high skill jobs Can be completed in two years or less Capacity building grants Help to meet the President’s college graduation goals.

Core Elements Evidence-Based Design Stacked and Latticed Credentials Online and Technology-Enabled Learning Transferability and Articulation Strategic Alignment Employers and Industry Public Workforce System Educational Institutions and Other Organizations

Award Information Up to $500 million in grant funds available $2.5 to $3.0 million to one individual applicant in each State, the District of Columbia and Puerto Rico $5.0 to $15 million to consortium applicants The period of performance is 48 months All programs must be developed and offered within the first 36 months of the period of performance. Enrollment should begin within 12-18 months. Final 12 months limited to gathering data for reporting outcome measures and third-party evaluation.

Eligible Institutions Institutions of higher education which offer programs that can be completed in not more than two years. Institutions that received funding as a single applicant or the lead in a consortium in the first round can not apply as a single applicant or the lead in a consortium in the second round. Can serve as member in consortium. Single applications - $2.5 to $3 million Consortia applications - $5 to $15 million

Consortia and Continuation DOL is particularly interested in consortium applications that include at least three eligible institutions, are state-wide and/or cross state lines, work with multiple employers within a sector, and/or reach out to and coordinate where possible with previously-funded TAACCCT projects. Applicants should note, however, that the Department does not intend to fund the continuation of projects funded under the TAACCCT 2011 SGA.

TAA for Workers Program Provides aid to US workers where foreign trade was an important cause of their actual or threatened job loss Includes supportive services such as tuition and fee costs, transportation, child care, dependent care and other services. TAA for Workers participants are the primary audience for TAACCCT-funded programs, although other adults may also be served. For more information, see www.doleta.gov/tradeact

Application Outline Part I – Cost Proposal Part II – Technical Proposal For individual institutions, limited to 30 doubled-spaced, single-sided, 8.5 inch x 11 inch pages with 12 point text font and one-inch margins. For consortia applications, limited to 45 doubled-spaced, single-sided, 8.5 inch x 11 inch pages with 12 point text font and one-inch margins. Part III – Attachments to the Technical Proposal

Division of Strategic Investments Steve Rietzke Sharon Leu Division of Strategic Investments

Evaluation Criteria Statement of Need (20 points) Description of the Project: Strategic Approach and Core Elements (40 points) Work Plan and Project Management (20 points) Outcomes (20 points) Non-Participant Data Submission (1 point Bonus) Total Possible Points - 101

1. Statement of Need (20 points) Scoring on this criterion will be based on three sub-criteria: Serving the Education and Training Needs of TAA-Eligible Workers (10 points) Evidence of Job Opportunities in the Targeted Industries and Occupations (5 points) Gap Analysis (5 points)

i. Serving the Education and Training Needs of TAA-Eligible Workers (10 points) Factor 1 – Impact of Foreign Trade Factor 2 – Partnerships with Applicable TAA Agencies Factor 3 – Education and Training Needs of TAA-Eligible Workers in Community(ies) to be Served

ii. Evidence of Job Opportunities in the Targeted Industries and Occupations (5 points) Factor 1 – Identification of Targeted Industries and Occupations Factor 2 – Evidence of Employer Demand Factor 3 – Understanding of Skills Required in the Targeted Industries and Occupations

iii. Gap Analysis (5 points) Factor 1 – Analysis of Gaps in Existing Educational and Career Training Programs and Systems Infrastructure Factor 2 – Analysis of Gaps in Institutional Ability to Implement the Core Elements Factor 3 – Impact of Gap Analysis

Scoring on this criterion will be based on five sub-criteria: 2. Description of the Project: Strategic Approach & Core Elements (40 pts) Scoring on this criterion will be based on five sub-criteria: Evidence-Based Design (8 points) Stacked and Latticed Credentials (8 points) Online and Technology-Enabled Learning (8 points) Transferability and Articulation (8 points) Strategic Alignment (8 points)

i. Evidence-Based Design (8 points) Factor 1 – Review of Evidence for Program Design Factor 2 – Use of Evidence in Program Design

ii. Stacked and Latticed Credentials (8 points) Factor 1 – Industry Engagement to Identify Credentials Factor 2 – Plans to Stack and Lattice Credentials Factor 3 – Prior Learning Assessment

iii. Online and Technology-Enabled Learning (8 points) Factor 1 – Incorporation of Technology into Program Design and Delivery Factor 2 – Expected Impact of Technology on Program Outcomes

iv. Transferability and Articulation (8 points) Factor 1 – Transferability of Credits and Credentials Factor 2 – Articulation of Credits and Credentials

v. Strategic Alignment (8 points) Factor 1 – Coordination with Employers and Industry Factor 2 – Coordination with the Public Workforce System Factor 3 – Outreach and Coordination with Educational Institutions and Other Organizations

3. Work Plan & Project Management (20 pts.) Project Work Plan (10 points) Project Management (10 points)

i. Project Work Plan (10 points) Factor 1 – Feasible and Realistic Activities and Timelines Factor 2 – Understanding of Costs Associated with Project Activities Factor 3 – Identification of Project Deliverables

ii. Project Management (10 points) Factor 1 – Effective Project Management and Staff Factor 2 – Effective Management Structures Factor 3 – Effective Systems and Processes Factor 4 – Well-Defined Roles and Responsibilities for Consortium Members (if applicable) Factor 5 – Sustainability Plan

4. Outcomes (20 points) Analysis of Outcome Projections (5 points) Process or System for Tracking and Reporting Outcome Measures (10 points) Using Data for Continuous Improvement (5 points)

i. Analysis of Outcome Projections (5 pts.) Factor 1 – Complete and Accurate Projections Provided Factor 2 – Appropriate Targets Projected Factor 3 – Appropriate Balance of Deliverables and Outcomes

ii. Process or System for Tracking and Reporting Outcome Measures (10 points) Factor 1 – Existing Tracking Procedures Factor 2 – Well-Defined Plan to Address Gaps in Tracking

iii. Using Data for Continuous Improvement (5 points) Factor 1 – Well-Defined Plan for Formal Data Reviews

5. Commitment to Submit Non-Program Participant Data (1 point Bonus) To be awarded the bonus point, the applicant “must provide an affirmative statement in response to this criterion committing to provide these data on an annual basis during the grant period of performance.”

Question & Answer Period

Office of Grants Management Melissa Abdullah Office of Grants Management

Paneling Basics Prior to the deliberation call: Read each proposal thoroughly without scoring Review each proposal a second time against the evaluation criteria Score each proposal against each criterion and sub-criterion Prepare weaknesses for each criterion and sub-criterion to account for any reduction of points from the maximum available point value when scoring. The page number and section of the proposal MUST be noted, if applicable, when referring back to specific information or details in the proposal

Paneling Basics For each criterion: Weaknesses must be included if any points are deducted The number and significance of weaknesses for each criterion should correspond directly to the score Provide full and complete sentences to support the weaknesses for each criterion

Using Criteria to Justify Scores Applications should be assumed to start at 100 points. The number of points allotted to each evaluation criterion should be determined by the associated sub-criteria and factors. As the proposal is read and weaknesses are documented, deduct a number of points appropriate to both the degree of the weakness and the point value associated with the relevant sub-criterion.

Documentation Tips When preparing weakness statements, please remember, the documentation is provided to assist the applicant in preparing future applications. All panelist comments will be compiled into one report; ensure statements are clear and concise to avoid any appearance of contradiction. Comments should be written with enough detail to stand alone. All comments should be objective and refer to specific evaluation criteria.

Words or Phrases to AVOID Such as… It appears… It seems… I believe… The applicant claims… The applicant should have… If the applicant would have… Questions (all questions and opinions must be captured under “Issues” or Comments”)

Examples of Good Weakness Statements #336 Good Deeds Community College WEAKNESSES: The applicant did not provide evidence that they will serve TAA-eligible workers (see application: page 2, Statement of Need). The applicant is missing a required employer partner (see application: page 12, Strategic Alignment).

Panelist Rating Workbook

Workbook Guidelines Only enter whole numbers in the Workbook. Save your Excel Workbook in the following format: Panel Letter, Panelist Last Name, Proposal #.xls (Example: Panel C, Smith,#10) Only change the values in blue cells. To adjust zoom, select “View>>Zoom…” from the toolbar at the top of the screen. Save your excel workbook often [in order to not lose info in the event of computer outage or glitch]!!

Deliberation Process Before the deliberation call [panelists discussion facilitated by Deliberation Specialist]: Read the assigned number of proposals; score each criterion and sub-criterion; and prepare written weaknesses for each criterion and sub-criterion. During the deliberation call: Actively participate in the discussion.

Sample E-mail Subject Line: Deliberation Process After the deliberation call: Adjust weakness statements and/or scores, if discussion or additional information deems an adjustment appropriate. Submit documentation in the Excel Workbook to the Deliberation Specialist by e-mail within 24 hours. Sample E-mail Subject Line: Smith, Rating Sheet, #1 Please note: the deliberation specialist will be reading the workbooks and will return to the panelist for revisions/clarifications as needed.

Deliberation Schedule Panelists will have three deliberation calls during the panel evaluation period. Deliberation Specialists will provide conference call number and pass code that will be used for all three deliberation calls. First deliberation call is Friday, July 27th; the first two proposals should be read and scored.

What’s Next?

Question & Answer Period

Share Your Ideas with Your Peers! You have the option to submit content for review by uploading the resource or providing a link to the resource. Share your demand-driven strategic plans, models, innovations, resources, and ideas!

Access to Webinar Resources Recordings and transcripts are available within 2 business days after the event. Access to Webinar Resources

Stay Informed, Get Connected!

THANKS! www.workforce3one.org For more information about the Workforce Investment System: Visit www.careeronestop.org Call 1-877-US2-JOBS THANKS! www.workforce3one.org