Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY July 16, 2012 (11:00 – 12:00 pm EST)

2 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Welcome to Workforce 3 One! Welcome to Workforce 3 One!

3 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY  To submit a question, type the question in the text field and press your Enter/Return key. ‒ Please enter the name to whom the question is directed.  Your name and your question will appear on your screen, indicating successful submission.  Questions are directly transmitted to presenters—no other participants will see your questions. Submitting Questions: Closed Chat Text Field Gary, where can I find today’s PPT?

4 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Practice In the Chat Room, please type the name of your organization, your location, and how many people are attending with you today.

5 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Access to Webinar Resources WEBINAR RESOURCES: Recordings and transcripts are available within 2 business days after the event.

6 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Panel Review Orientation for Disability Employment Initiative Grants SGA/DFA PY-11-11

7 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Featured Speakers  B. Jai Johnson, Grant Officer, ETA  Randee Chafkin, Program Manager, ETA  Eileen Banks, Grants Management Specialist, ETA

8 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Agenda  Best Practices for Paneling (B. Jai Johnson)  Funding Opportunity Description (Randee Chafkin)  Evaluation Criteria (Randee Chafkin)  Panel Materials (B. Jai Johnson)  Paneling Process (Eileen Banks)  Closing (B. Jai Johnson)

9 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Best Practices for Paneling

10 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY  The U. S. Department of Labor, Employment and Training Administration (ETA) has secured a diverse pool of professionals to panel this Solicitations for Grant Applications (SGAs).  Primary objective of the panel is to review and make a fair evaluation of proposals submitted in response to the SGAs.  Each panel team will include: panel members and an ETA deliberation specialist. Overview of the Paneling Process

11 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY  As the Grant Officer for this solicitation, I thank you all for your time and effort in this review process.  Reviewing grant applications is a serious responsibility. You need to be aware that we need your 100 % cooperation in the reviewing, reading, scoring and deliberation of these applications.  Again, thank you in advance for a great job.

12 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Paneling Process Orientation Materials  Today’s PowerPoint Presentation  Panelist Excel Workbook  Conflict of Interest Form  Security of Proposals Form  Assigned Panel Information  Deliberation Schedule  Solicitation for Grant Applications  Electronic Copies of Assigned Proposals POC: Eileen Banks or

13 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Role of Panelist  Paneling is a full-time commitment  Commit to thoroughly reading and scoring each proposal prior to the deliberation calls which includes: –Review of the evaluation criteria –Review of each proposal based on the evaluation criteria and assign a numerical rating for each criterion/sub-criterion –Documentation of your weakness statements for each criterion/sub- criterion to establish a clear relationship between the score awarded and the weaknesses identified –Finalizing weakness statements as a result of deliberation call –Submit the Excel Workbook for each application by to the deliberation specialist within 24 hours after the deliberation call

14 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Performance Excellence Partners, Inc.  DOL has retained Performance Excellence Partners, Inc. (PEP) has been retained by DOL to identify and procure panelists. You will be working for PEP to perform the panel deliberations.  PEP will provide information by on invoice instructions and payment.  Please direct questions to: –Linda Metchikoff-Hooker at PEP (office);

15 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY QUESTIONS?

16 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Funding Opportunity Description

17 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Eligibility Information and Funding Opportunity Descriptions  Eligible applicants are: –The state Workforce Investment Act (WIA) administering agency. The state workforce agency must select a minimum of four local State Workforce Investment Boards (LWIBs) that have the capacity to carry-out the State’s proposal under this SGA. Award Amount: –Approximately $20 million in grant funds from approximately $1.5 to $6 million each is available for cooperative agreements to state Workforce Investment Act administering entities. Period of Performance: –Three years (36 months)

18 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Eligibility Information and Funding Opportunity Descriptions continued  The expected award funding will be for approximately 6 to 10 grants  DOL will find applications that request funds exceeding the amount specified will be found non-responsive and DOL will not consider such applications for this grant award opportunity

19 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Proposal Outline  Part I – Cost Proposal:  Includes 4 items: 1) SF 424; 2) DUNS number; 3) SF 424 A Budget Information, including 4) Budget Narrative (Optional) = OMB Survey N Form Part II – Technical Proposal: –Limited to 30 doubled-spaced page limit, single-sided pages with 12 point text font and 1 inch margins. Do not review any materials beyond the 30 page limit. Do not read additional materials such as resumes or general letters.

20 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Proposal Outline Contents Cont’d  Part III – Required Attachments to Technical Proposal:  1) Two-page Abstract; 2) Project/Performance Site Location; 3) chart displaying WIA and Wagner-Peyser data to address evaluation criteria in section V.A.; 4) chart displaying state, population of state, and list of each local WIB; and 5)graphic display of work plan implementation schedule, expected milestones, and outcomes.  See the SGA for additional Required Attachments –The Required Attachments will be excluded from the 30 page- limit. –Required attachments must be affixed as separate, clearly identified appendices to the application. Do not review any materials beyond the 30 page limit.

21 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY QUESTIONS?

22 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Evaluation Criteria

23 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Evaluation Criteria Criterion Points 1.Program Delivery – Strategic Approach 30 a. Primary Focus, Rational & Objectives (5) b. Strategic Approach (25) 2. Partnership Commitment and Resources 15 a. Strategic Partners(10) b. Partner Resources (5)

24 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Evaluation Criteria Criterion Points 3. Demonstrated Experience 25 a. Services and Outcomes to Adults or Youth with Disabilities(5) b. Physical, Communication, and Programmatic Accessibility(5) c. Programmatic Experience and Initiative(10) d. Registration and Operation as an Employment Network(5)

25 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Evaluation Criteria Criterion Points 4.Project Management 10 a. Staff Capacity (5) b. Fiscal and Administrative Capacity (5) 5. Outcomes and Sustainability 20 a. Outcomes (15) b. Sustainability (5)

26 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY ) Base scoring on this criterion on the following subcategories: i.Primary Focus, Rationale and Objectives (5 points) ii.Strategic Approach(25 points) For this and all criteria, panelists must review and utilize the complete description in the SGA 1. Program Delivery –Strategic Approach (30 total points)

27 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY ) Base scoring on this criterion on the following subcategories: i. Strategic Partners (10 points) ii.Partner Resources (5 points) 2. Partnership Commitment and Resources (15 total points)

28 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Base scoring on this criterion on the following subcategories: i.Services and Outcomes of Adults and Youth with disabilities (5 points) ii.Physical, communication and programmatic accessibility (5 points) iii.Programmatic experience and initiative (10 points) iv.Registration and Operation as an Employment Network (5 points) 3. Demonstrated Experience (25 total points)

29 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Project Management (10 Points) Score this criterion based on two subcategories: i.Staff Capacity (5 points) ii.Fiscal and Administrative Capacity (5 points)

30 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Outcomes and Sustainability (10 Points) Panelists will score criterion based on these two categories: i. Outcomes (15 points) ii. Sustainability (5 points) 5. Outcomes and Sustainability (20 Points)

31 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY QUESTIONS?

32 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Panel Materials and Process

33 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Panel Materials  The following panel materials have been ed or sent via hard copy –Today’s PowerPoint Presentation –Panelist Excel Workbook –Conflict of Interest/Non-Disclosure Statement –Security of Proposals Form –Applicant Panel Sheet—proposal assignments to each panel –Deliberation schedule  Fax Number= (202)

34 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Role of the Deliberation Specialist  Ensure panelists receive a complete package of materials  Answer questions and concerns in a timely and professional manner  Facilitate calls to ensure a fair deliberation process  Ensure panel members are not pressured to change their ratings  Provide timely feedback to panelists if written evaluations are incomplete

35 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Deliberation Schedule  Wednesday, July 18th  Friday, July 20th  Monday, July 23rd  Please call and use Participant Passcode to join the conference. All conference call times are EASTERN time. This number is used for all calls (Deliberation Specialist Panel A –Charlotte Adams)  If you have any questions, please contact Eileen Banks at or

36 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Timeline and Overview of Deliberation con’t  The proposals to be reviewed by each panel can be found in the Applicant Panel Excel document and in the Panelist Excel Workbook under the tab labeled “Panel Assignments”  The applicant numbers are not in numerical order. Please start your review with the first proposal on the list.

37 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Expectations of YOU as a Panelist  Appeals determined to have merit may result in further legal scrutiny of panelist’s written evaluations by legal representatives of appealing applicants.  Note: it is vitally important that panelists only use the SGA’s Evaluation Criteria as the basis for evaluating the proposals.  Appeals involve the disclosure of panelists’ written evaluations to the Administrative Law Judge  Grant award selections are the ultimate responsibility of the Grant Officer  Grant award selections are subject to appeals by unsuccessful applicants.

38 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Scoring Basics Scoring Basics Prior to deliberation call:  Read the proposal thoroughly without scoring  Review the proposal a second time against the evaluation criteria  Score the proposal against each criterion  Prepare weaknesses for each criterion. Please note the page number and section when referring back to specific information or details in the proposal. (Placing the page # on your evaluation response sheet will enable quick referral back to the discussion area)

39 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Scoring Basics, continued For each criteria:  Weaknesses must be included if any points are deducted  The score given for each criterion should reflect the weaknesses assigned by the panelists

40 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Scoring Tips Words or Phrases to USE  “The applicant (clearly, effectively) demonstrates”  “The applicant did not demonstrate”  “The applicant describes in detail”  “The applicant did not describe in full detail”

41 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Scoring/Rating EXAMPLE of Clearly Supported Statement  The applicant clearly discussed the overall strategic approach that it will use and stated how this approach will: address the unique needs of the primary population; the workforce areas the applicant expects will participate; and why the applicant expects the project design will be successful. Application page 2

42 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLES of Correct “Weakness” Statements #336 Good Deeds Community College WEAKNESSES:  The applicant did not provide adequate data and analysis to demonstrate the WIA and Wagner-Peyser Services for adults and youth. (see proposal: page 2, Demonstrated Experience)  The applicant did not discuss partners involved at the state level. (see proposal, page 12, Partnership Commitment and Resources)  The applicant did not clearly demonstrate strategies for carrying on the project beyond the end of the DEI grant. (see proposal, page 19, Outcomes and Sustainability)

43 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY QUESTIONS?

44 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Excel Panelist Rating Workbook  Panelists enter scores, weaknesses, and issues in Excel workbooks, one for each proposal scored  Master copy called “Master Panelist Rating Workbook, SGA xls” –For each proposal scored, save a copy with the following naming scheme: Panel Number, Panelist Last Name, Proposal #.xls  Workbooks have eight tabs: 5) Demonstrated Experience 6) Project Management 7) Outcomes and Sustainability 8) Appendix: Evaluation Criteria 1) Score Summary 2) Panel Assignments 3) Program Delivery – Strategic Approach 4) Partnership Commitment and Resources

45 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Panelist Rating Workbook

46 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Panelist Rating Workbook

47 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Panelist Rating Workbook

48 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Panelist Rating Workbook

49 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY EXAMPLE Panelist Rating Workbook

50 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Workbook Guidelines  Only enter whole numbers in the Workbook  Save your Excel Workbook in the following format: –“Panelist Last Name, Rating Sheet, Proposal #.xls” (Example: Smith, Rating Sheet, #10)  Only change the values in blue cells  To adjust zoom, select “View>>Zoom…” from the toolbar at the top of the screen  Save your excel workbook often [in order to not lose info in the event of computer outage or glitch]!!

51 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Important Tips  For each proposal reviewed, Panelists will save the master as: –“Panel Number, Panelist Last Name, Proposal #.xls” (Example: Panel 2 Smith #10)  Panel will save their summary report as:  Only whole positive numbers will be accepted in the Workbook  Excel sheets are protected, you can only change the values in blue cells  Remember to save often!!  For the list of proposals to review, please refer to the tab of the Panelist Rating Workbook, which is called Panel Assignments.

52 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Deliverables and Close-Out  Excel Workbook for each proposal is due within 24 hours after each deliberation  Use the following format in the Subject line when ing the Excel Workbook: Panel #, Last Name, Rating Sheet, Proposal #.xls  Submit conflict of Interest/Non-Disclosure Statement  Panelists must keep the hard copy proposals for approximately 45 days after final deliberations. After 45 days, panelists may destroy the proposals.

53 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Summary- What do I do now? Paneling  START READING  Sign and date the Conflict of Interest/Non-disclosure Statement  Review each proposal according to the criteria  Score each proposal before the deliberation call. The first call is scheduled for Wednesday, July 18th  Participate fully in each deliberation call  Capture any non-scoring concerns in the designated “issues” section of the Excel Workbook  Submit an Excel workbook for each proposal  Return any signed forms to the grants office

54 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Information for Panelist  As we have mentioned several times, reviewing grant applications is a serious responsibility. Your review reports (submitted in Excel workbooks) must completely describe any weakness (es) in the application and support the score you have listed.  ETA will make every effort to protect your “draft” reports/documentation from litigation that may result from a non-selected applicant. However, you should be aware all the materials associated with this review may become documentation used in a non-selection appeal.

55 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY THANK YOU  For playing an important role in our panel review process  For contributing to our future workforce  And participating in the most important work of the federal government

56 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Please enter your questions into the Chat Room! Question and Answer Period

57 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Share Your Ideas with Your Peers! Share your demand-driven strategic plans, models, innovations, resources, and ideas! You have the option to submit content for review by uploading the resource or providing a link to the resource.

58 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Access to Webinar Resources WEBINAR RESOURCES: Recordings and transcripts are available within 2 business days after the event.

59 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY Stay Informed, Get Connected!

60 Panel Review Orientation for Disability Employment Initiative Grants SGA-DFA-PY For more information about the Workforce Investment System:  Visit  Call US2-JOBS