Writing More Effective Proposals Russ Pimmel Abe Nisanci U of Alabama NSF. Share The Future IV March 17, 2003

Workshop Format “Working” workshop ½ to ¾ of time in team activities Limited time to complete activities Frequently feel you need more time Purpose is to get you started No “answers” or even the “formulas” Purpose: identify, consider, & discuss ideas

Workshop Goals Participants should be able to: List areas where good engineering education proposals can be improved Generate a list of suggestions for each area

Beyond a Good Idea Proposals must describe a good idea It must be explained and developed Workshop will assume a good idea Focus on areas for enhancing a proposal that contains a good idea

Warning on Generalizations NSF has several programs supporting undergraduate education Different requirements Different slants Proposal improvement ideas apply to all -- but in varying degrees Choose ideas based on Program solicitation Judgment

Scenario – Developing a Proposal Idea Prof. ____ has taught ENG ___ for several semesters He has idea for “greatly improving” the course by adding new stuff new stuff = laboratory, web experience, interactive set of material, workbook, new text He tried some preliminary material Based on this, Prof. ____ decided to prepare an NSF proposal

Proposal Skeleton Goal: Develop new stuff to enhance student learning at U of _____ Rationale: Observed shortcomings in educational experience of the students at U ____ & felt that new stuff would improve the situation Project Plan: “Details of new stuff “ Evaluation: Conduct course evaluations when using new stuff Dissemination: Describe new stuff using conference, journal papers, and web site

What’s Wrong? TASK: Prepare a list of ideas for improving this proposal What advice would you give Prof ___ PROCESS: Individually prepare a list Share ideas with neighbor Report neighbor’s best idea

What’s Wrong? -- NSF Project Directors’ Responses – Goals Indicate the development, evaluation, and assessment are the real goals Not “enhanced learning of students at U of ____”

What’s Wrong? -- NSF PD’s Responses – Rationale Describe experience at other schools Reference the educational literature Discuss effects on retention and broader participation Indicate why approach is new and innovative

What’s Wrong? -- NSF PD’s Responses – Evaluation Use external evaluator or assessment expert Include collaboration with other faculty at same or different school Include beta test at another site Include measures of student learning in evaluation process Tie evaluation to goals and objectives Include impact statement

What’s Wrong? – PD’s Responses – Dissemination Include collaboration with faculty members in other schools Include an outreach component K-12 or community colleges Include beta testing at other school Include faculty workshop

What’s Wrong? -- PD’s Responses – General Include letters of support Form a collaborative effort Include a plan with timeline, milestones, and responsibilities Make sure to select the appropriate NSF program

What’s Wrong – Four Concerns Goals focused on local problem Ignores broader impact Considers only applicant’s experience Ignores the experience of others Ignores the literature Limits evaluation to students’ impressions Ignores learning goals and outcomes Dissemination plan passive Needs to be proactive and aggressive

What’s Wrong – Four Concerns Goals focused on local problem Ignores broader impact Considers only applicant’s experience Ignores the experience of others Ignores the literature Limits evaluation to students’ impressions Ignores learning goals and outcomes Dissemination plan passive Needs to be proactive and aggressive

Improving Rationale -- Global vs Local Problem TASK: Generate a list of locations or sources that will provide a broader view of the problem leading to broader goals PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

Strategies Team Exercises Be positive, supportive, and cooperative Limit critical or negative comments Be brief and concise in discussions Avoid lengthy comments, stories Avoid arguments Stay focused Designate roles Coordinator, recorder, gatekeeper

NSF PD’s Responses -- Global vs Local Problem Education literature Journals and conference proceeding Education sessions at discipline meetings Lay scientific press NY Times science section Panel reports “What’s wrong with Education in _____’ Industry or advisory board input

NSF PD’s Responses -- Global vs Local – Part 2 NSF web site Education oriented web sites Teaching and learning centers at some universities Education pages at professional society sites Colleagues at other schools Web sites at other schools

Improving Goals & Objectives Statement TASK: Generate a list of improvements for the goals and objectives in Reading # 1 A list of suggestions that will broaden and clarify the goals and objectives PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

PD’s Responses – Improving Goals & Objectives Relate goals to student learning objectives Use more specific, goal-oriented verbs “Enhance “ and “acquaint” are vague Don’t describe measurable actions Be more specific Eliminate the “apple pie” goals

PD’s Responses – Improving Goals & Objectives – Part 2 Use broader goals Don’t just focus on effects on student's in PI’s course Make the goals to develop, evaluate, and disseminate material Be careful about the distinction between goals and objectives Goals – higher-level, broad-reaching Objectives – specific, measurable outcomes

What’s Wrong – Four Concerns Goals focused on local problem Ignores broader impact Considers only applicant’s experience Ignores the experience of others Ignores the literature Limits evaluation to students’ impressions Ignores learning goals and outcomes Dissemination plan passive Needs to be proactive and aggressive

Build on Experiences of Others TASK: Generate a list of locations or sources that describe prior work by others PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

NSF PD’s Responses -- Others’ Experiences Same as earlier list These sources Justify a broader need Summarize others’ experiences

Improving Rationale TASK: Generate a list of improvements for the rationale statement in Reading # 2 A list of suggestions that will provide a broader view PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

PD’s Responses – Improving Rationale Discuss shortcomings or problems in programs described by others Discuss the general need for the new material -- the need at other schools Indicate student interests (current & projected) – include references Indicate demand for graduates (current & projected) – include references Discuss how the new material will fit in the existing curriculum

What’s Wrong – Four Concerns Goals focused on local problem Ignores broader impact Considers only applicant’s experience Ignores the experience of others Ignores the literature Limits evaluation to students’ impressions Ignores learning goals and outcomes Dissemination plan passive Needs to be proactive and aggressive

Evaluate Goals, Implementation, Outcomes TASK: Generate a list of aspects that can be evaluated in a projects PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

NSF PD’s Responses – Evaluation Aspects Measure gains in student learning Pre and post tests Experimental and control groups Use formative and summative evaluations Formative to guide development Summative to verify & document success Include diverse audiences Universities & community colleges Majors and non majors

NSF PD’s Responses – Evaluation Aspects – Part 2 Evaluate at several levels Appropriateness of learning objectives What is being taught/learned Attitude of students How is it being taught Learning outcomes How successful was the instruction Examine effects on retention and diversity Consider beta testing

Evaluate Goals, Implementation, Outcomes TASK: Generate a list of improvements for the evaluation plan in Reading # 3 PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

PD’s Responses – Improving Evaluation Add more formative evaluation Monitor students’ attitude and learning during course Measure student learning Need learning objectives Include copy of evaluation tool or sample questions e. g., student survey form Develop specific criteria for evaluation by other faculty in subsequent courses

What’s Wrong – Four Concerns Goals focused on local problem Ignores broader impact Considers only applicant’s experience Ignores the experience of others Ignores the literature Limits evaluation to students’ impressions Ignores learning goals and outcomes Dissemination plan passive Needs to be proactive and aggressive

Use Active, Aggressive Dissemination TASK: Generate a list of approaches for disseminating results of project PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

NSF PD’s Responses – Dissemination Approaches Educational journals & meetings Don’t neglect regional meetings Faculty workshops Personal or course web sites Professional group or subspecialty web sites Professional group or subspecialty newsletters

NSF PD’s Responses – Dissemination – Part 2 Textbooks, manuals, instructor guides Agreements with other faculty members to critique or evaluate material Mailing to colleagues General or targeted

Include Active, Aggressive Dissemination TASK: Generate a list of improvements for the dissemination plan in Reading # 4 PROCESS: Individually prepare a list Work as a team to build a consensus list Report team’s ideas

PD’s Responses – Improving Dissemination Any or all items on previous list

Practical Aspects of Review Process Reviewers have ten or so proposals from several areas Reviewers have limited time for your proposal Reviewers may be experienced or inexperienced in review process Reviewers may be an expert or a novice in proposal area

Dealing With Practical Aspects of Review Process TASK: Generate a list of approaches that an applicant should consider in dealing with these practical aspects PROCESS: Individually prepare a list Share ideas with neighbor Report team’s ideas

NSF PD’s Responses – Practical Aspects of Review Use good style Be concise, specific, but complete Write simply but professionally Avoid jargon Use spell and grammar checkers

NSF PD’s Responses – Practical Aspects – Part 2 Follow guidelines Double space, use correct font size Use readable structure Use sections, headings, bullets Follow the order given in solicitation Use appendices sparingly

NSF PD’s Responses – Practical Aspects -- Part 3 Emphasize results when writing about prior funding Reinforce your ideas Summarize Highlight them (e. g., use bullets) Pay attention to the rationale, goals and objectives, evaluation, and dissemination Have expert and non-expert read draft version

Warning on Generalizations NSF has several educational programs Different requirements & slants Proposal improvement ideas apply to all -- but in varying degrees Read the solicitation carefully

Beyond a Good Idea To enhance a good proposal Describe broader impact in rationale & goals Consider and build on others’ experiences Cite the literature Evaluate learning goals, students’ impressions, outcomes, etc. Include proactive & aggressive dissemination

Use Judgment When writing proposals, you will wonder “Should I include ____?” “Should I do _____?” “How should I do _____?” The answer is “It depends.” There is no magic formula. Read the solicitation Use your judgment Don’t include a half-bake section because someone told you that it’s you needed

Final Comment If you have a good idea, thinking about it in terms of How you could broaden the objective How you could relate it to the literature How you could evaluate it How you could interest others in it will sharpen the idea

Questions

Russ Pimmel Abe Nisanci