Grading system Two major categories –20% Class attendance, participation, assignment submissions –40% Quality and preparation of rough draft: Joshi –40% Your peers and Professors A95-100%Outstanding: must fund AB90-94Excellent: high priority B85-89Very Good: medium priority BC80-84Good: low priority C75-79OK: Some merit CD70-74BAD: Why did you waste my time? D65-69Very BAD: Do not fund!

So far… Melissa Roberts Chee Hui Lee Danilelle Dusold Eric Minner Ee Lim Tan Eric Winder Kate Tay Lindsey Tuominen Louis Paladino Matthew Barron Meagan Harless Tara Waybrant Wilfred Previant 6 have not given the rough draft which was originally due on March 6 th Today is March 26, 2008

Review Process Who decides the fate of your proposal? How?

Checklist Cover page Table of content Summary Project description References cited Biographical sketches Budget and budget justification Current and pending support Facilities and equipment Put summary, PD and references together in the same sequence in a single file, paginate it and convert it into a pdf file. me one pdf file with your last name as file name by April 9, 2008 Now, let us presume you have submitted the proposal via Fastlane through your SRO

Who looks at it and how deep? Administrative assistants Format compliance eligibility Complete set of documents Assign to area/manager as you requested Prepare list of individuals that are in conflict with you Program manager (also a Professor at some U) –Reads title and abstract –Confirms the program to which your proposal is assigned or shifts to other programs –Decides upon the panel manager/panel members –NSF has no separate panel manager. Program manager does the job as panel manager!

Panel manager Who is the panel manager? (Roll of dice) –Well-established, successful senior professor at some renowned University –Works for one-two years in a row –Is happy with his/her own career, well connected –Has vision for the future research in the topic –Reads ALL abstracts and first pages of ALL proposals (at least). Decides in consultation with program manager who could be panel members and ad hoc reviewers

Panel members Based on the topics of proposals submitted to a program, panel members are invited. Some are the same as last year’s panel Mostly they do not compete for funding in the same year or have to leave discussion if any direct conflict of interest exists. Primary and secondary panel members Each reads about proposals in depth and prepares reports for panel and submits electronically. Gets input from ad hoc e-reviewers

Ad hoc reviewers About 6-10 specialists are invited to review your proposal. They should have no direct conflict of interest. Done electronically via fastlane Use two major criteria –Intellectual merit –Broader impact They rate the proposal as excellent (e), very good (vg), good (g), fair (f), poor (p)

Next step… Not every ad hoc reviewer responds on time (~2 months) May have personal bias against or for you or too busy May be harsh or too mild (depending on where they are on ladder?) They may be jealous of your success too!! You hope.. About 3-9 reviews get in and put in excel spread sheet Panel member has to submit review first then they can see the rest All such reviews are seen by the primary reviewer who makes up his/her mind about fate of your proposal Defends or destroys your proposal in the panel meeting Secondary panel member keeps a check on primary member and takes notes of panel discussions Open process on hidden agendas!

After about 3 months after proposal submission Panel meets at Washington D.C. for 2 and half days Each proposal gets less than ~15 minutes of discussion. Someone has to passionately defend you! “Wow” science goes at the top Each proposal gets a rating (or other grades) –Outstanding, highly meritorious, meritorious, non-competitive If your name is on the outstanding side, your chances of success are high and very low if you are in the fourth pile Next day or so, panel members go back read proposals again and change ratings, if necessary

Horror stories Simple methodology omitted and proposal tanked. No publications from last grant, proposal rejected! Panel members had bad experiences with the PI The PI (postdoc of another well-funded researcher) almost lost a grant but saved at the last minute by his fax declaring change of job Supporting evidence submitted just before the panel meeting did save a dying proposal Catch-22 situation! First, asked for preliminary data and then asked why you need more money? No clear broader impact statements and proof Fan club reviewers so not funded

Program manager Decides the final ranking of proposals Two more visiting program managers help Has some freedom to move within the ranks Decides how much money could be given Calls or communicates with the PI Negotiates what needs to be done and for how much support $? Sends declination letters and reviews Answers your questions

What if you get a grant? Do Party but not forever! You are among the top 5-10% researchers in your field Hire people and deliver the goods promised on time Publish profusely in high quality journals Write more grants! Why? –funding does not last for ever –10% success rate –distribution of wealth principle

What if you do not get a grant? don't cry (OK, cry a little if you feel better) pick up the pieces of your failed proposal and restart your “grants writing” engine get reviewer’s comments, read and get angry then keep them in a drawer away from your view for a while.. come back and read reviews again talk to PM and your mentor/well wishers resubmit until you succeed

1. Lack of new or original ideas. 2. Diffuse, superficial, or unfocused research plan. 3. Lack of knowledge of published relevant work. 4. Lack of experience in the essential methodology. 5. Uncertainty concerning future directions. 6. Questionable reasoning in experimental approach. 7. Absence of an acceptable rationale. 8. Unrealistically large/small amount of work. 9. Lack of sufficient experimental detail. 10. Uncritical approach. Top ten reasons why funding is normally not awarded

How will your proposal be evaluated for this FW5850 class? You will submit a single pdf file of your proposal to me, your advisor and your peers by . It will contain only one page of summary, up to 15 pages of description and 3-4 pages of references. Please do this latest by April 9 th, 2008 (no extensions possible) Now, you will change your role. You will review (as an ad hoc reviewer) all the proposals from your group except your own. You will prepare reviews for each proposal in your group in the prescribed format (I will it to you and your advisors) Bring two hard copies of your reviews to class on April 21st, 2008 and put your name on only one and give it me and the other will be put in respective student's folder Your advisors and I may also review them and we will add our reviews to the folder! All reviews given to the PI will be anonymous

Evaluation Criteria Intellectual merit –How important this proposal is for advancement of knowledge? –Qualification of PI and quality of proposal? –Creative and original concepts? –How well conceived and organized is this activity? –Sufficient resources available for this research? Broader impacts –Advance discovery and understanding –Can promote teaching and research integration –Diversity (gender, ethnicity, disability, geographical), if any * –Infrastructure development* –Dissemination of information obtained –What is the benefit to society? ALL CRITERIA MAY NOT APPLY FOR EACH PROPOSAL! Focus more on the contents (summary, description) than the format! * Not important for your evaluation

Your review One page Please try to answer some of these questions –Is the title concise and attractive? –Did the summary give you sufficient idea about the activity to be undertaken? –Did the first page sell you the project? –Is the idea sufficiently novel? –Will it advance science significantly? –Was the background given adequate (and not excessive)? –Were references cited in the text? –Were you clear about what is known, what is proposed and what will be achieved by the end of project? –Were all necessary methods appropriately and briefly described? –Were there three-four objectives? Were they stand-alone? –Was proposal a pleasure to read? Were there any grammatical issues? –Was there a time-table? –Are the available facilities and equipment adequate for the project? –Is the reference style used uniform? –Were sufficient number of figures and tables used? –Did you like the project? Do you think it is a worth funding and well prepared project? Please do not just say “yes” or “no” but use these questions to build your review Finally, please rate the proposal as excellent (E), very good (VG), good (G), fair (F) and poor (P). Please also provide suggestions for improvement.

Panel meeting on April 21st and 23rd, 2008 You change your role again. Now, you are a panel member. There will be four panels = 4 peer groups On April 21st, we will meet in the atrium area. There will be four separate tables for panel meeting, one per group Each proposal will get minutes discussion (a bell will ring every 20 minutes) You will go to the other table when your own proposal is being discussed as shown in the next slide.

ENGINEERS Biomedical Chemistry Biology Me

Panel reviews Each panel member will first state their ratings and then discuss each proposal based on their own written reviews. Panel will prepare a one page report using three major categories: summary, strength, weakness (2-3 lines each) (see next slide). The panel will also give the final rating as Excellent (E), very good (VG), good (G), fair (F), poor (P) The panel report will be given to me at the end of your discussion and you will select a panel leader from the members within your own group to read the summary to whole class on April 23rd, the last day of this class.

Panel evaluation What is the main research topic of the proposal? (One-two sentences) What is the major strength of this proposal? What is the major weakness of this proposal (if any)? Summary statement (one or two lines)

Panel presentation (April 23rd 2008) You will select one leader from your group who will read all reports from your group after panel meeting to the whole class Each proposal will get ~two minutes Each report will have a specific final rating. Then you will enjoy your summer vacation!

Any questions?