1. “Knowing Revisited” And that’s how we can move toward really knowing something: Richard Feynman on the Scientific Method

2. Scientific Broadly Defined Science is an objective, logical, and systematic method of analyzing and explaining phenomena, devised to permit the accumulation of reliable knowledge. The product of science is knowledge

3. Scientific Broadly Defined Premises of science are –empiricism –objectivity –falsifiability –control What this means: –rationality and skepticism are how we learn about the universe and shape new principles –If the arguments and experiments are sound, if the theory can withstand skeptical scrutiny, if the work was undertaken within the framework of past research and provides a basis for further discovery, then it is science

4. The Process of Science The process of science begins with speculation, observation, and growing understanding of some idea or phenomenon. This understanding is used to shape research questions, which in turn are used to develop hypotheses that can be tested by proof or experimentation. The results are described in a paper, which is then submitted for independent review before (hopefully) being published; or the results are described in a thesis that us then submitted for examination. --Justin Zobel

5. Elements of Good Scientific Research? Problem clearly specified and hypotheses formulated based on observations Objectives clearly defined - scope, limitations are fully specified Process detailed - can be repeated and except when secrecy is imposed reveal the sources of data and means by which they were obtained Design thoroughly planned - make as objective as possible

6. Elements of Good Scientific Research? High ethical standards applied Limitations frankly revealed - there are very few perfect designs adequate analysis - data classified to clearly reveal findings, probability of error should be estimated Threats to validity (external, internal, construct) clearly identified and addressed findings presented unambiguously conclusions justified

7. Research Project Phases An individual research project (such as a Ph.D. dissertation) follows a specific life cycle: –Choose research question/problem; formulate hypotheses –Determine current state of knowledge –Apply appropriate methods to produce research results To verify the hypotheses To evaluate the proposed solutions –Write up research results Research is not complete until it is written up (and published) – Peer Review is critical!

8. A Research Project Checklist (I) Are ideas clear and consistent? Is the problem clearly specified? Is it worthy of investigation? Are there proposed solutions to be explored? Does the project have appropriate scope? What are the specific hypotheses / research questions What would disprove the hypothesis? What are the assumptions and are they “sensible”?

9. A Research Project Checklist (II) Has the research plan/design been critically assessed? Are you convinced that it is based on sound science? What forms of evidence are to be used? How are outcomes to be evaluated? What are the metrics? Why are the selected methods appropriate for verifying the hypothesis or evaluating the proposed solutions? What are the likely weaknesses of your solutions?

10. A Research Project Checklist (III) Is there a detailed written research plan? Have milestones, timelines, and deadlines been identified? Do the deadlines leave enough time to receive feedback on the drafts and to allow colleagues to contribute? Has the literature been adequately explored? Once the work is done – and your perspective has changed – does in need to be explored again?

12. Unstructured Environment especially in PhD need to develop a routine make research a priority regular meetings with your advisor join or set up a research group role of course work keep a journal - get in habit of writing down ideas

13. Research Is About Reading Foundation of good research: find and critically read related scientific articles Locate important research tools in your area –journals and conferences –workshops –important or seminal papers in your area –good research sites and groups –other faculty web sites –develop your own Web repository (why?) How to read papers –first look at abstract, keywords, references –next: for some papers look at intro. and conclusions –next: for some, read in detail and write notes

14. What to consider when critically reading papers (write a summary for every paper you read) what is the main problem? why is it important? what is the hypothesis? how precise are the claims? what are the main results (in your own words)? how does it relate to previous work? what are the most important references? what is the evidence and how was it gathered? what are the metrics and how are measurements taken? how carefully are the algorithms and experiments described? what would the reproduction of the results involve? why and how are the results significant? ideas for future work?

15. Exercise Select one recent paper relevant to you area of interest Critically read the paper (using criteria in previous slide) Submit the paper and your notes in two weeks You may be asked to summarize in class

16. Research Is About Writing writing is the most fundamental part of research writing not just an end, but also a tool writing logical reasoning take a writing course when reading papers, mark those that are especially well- written keeping a journal, revisited consider writing a survey paper write one or more research proposals Note: programming or math are not writing, by themselves; programming is also not research