1. A Template for Producing Research Papers in the AI Lab
Byron, Dan, Zan, and Jennifer
Acknowledgement: Hundreds of meetings with Dr. Chen
Disclaimer: Use this advice at your own risk. If we already knew how to do it, it wouldn’t be research, and we would already have tenure!
Disclaimer 2: This presentation is filled with student perceptions of what Dr. Chen said or meant, and spiced with our own commentary. These views may or may not accurately reflect Dr. Chen’s position when you turn in a paper or give a presentation.

2. Agenda Notes on the Title Abstract Introduction
Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Experimental Results
Discussion
Conclusions and Future Directions
References
Some additional suggestions
Reviewing papers

3. Notes on the Title
A good title should reflect the entire contents of the paper, avoid using cute titles.
Should be less than 8-9 words.
Complete sentences are not necessary, use colons if needed.
Acronyms
If needed, select meaningful acronyms that convey the meaning of the work/project (e.g., COPLINK, GeneScence).
Acronyms help in branding a system/project.
Do not forget acknowledging the funding agencies on title slides of presentations!

4. Abstract – What is in it?
One of the most important parts of the paper.
Concisely express the problem in one sentence or two.
Mention why the work is important (if the goal of the research was accomplished, what good thing would happen… )
Describe methodology
Highlight the most important results
Gives the reviewer a reason to continue reading the paper
Should consist of short sentences
Don’t invent new words!
* Slides do not generally include an abstract *

5. Abstract – Our Abstract
Publish, Publish, Publish (or Perish!)
Although many different methodologies can lead to an accepted publication, these principles should be particularly useful to you in the AI Lab environment.
We will present our view of how to get papers past Dr. Chen and out for publication.
The content is presented in the form of a template.

6. Introduction – What is in it?
The Introduction “hooks” the reader.
What is the motivation for the work?
What is the context?
Big picture, how did we address the problem?
What will the rest of the paper look like?
Introduction should be 3-5 paragraphs
A four page introduction is not a good introduction
The last paragraph always points to the structure of the paper

7. Introduction – Motivation/Context
PhD students want to learn to be productive researchers.
A number of research approaches are effectively employed by various researchers and research groups.
The model used here in the AI Lab has a strong track record of producing work publishable in top tier journals.

8. Introduction – The Big Picture
To be productive we need to be:
Doing the right research,
Doing the research right, and
Packaging the research appropriately for distribution.
Missing any of these elements substantially reduces the chance that your work will be published.

9. Doing the Right Research
Criteria for choosing a research project that might eventually be funded.
Does it advance Science? Choose work that makes a contribution to some scientific body of knowledge. Implementing an effective system is not enough: the methodology and techniques are important.
Does it have potential impact? Do work that will improve important real-world processes. Thus, we emphasize domain-specific applications and completing a line of work in a user study. Caveat: Making an impact with one paper is tough.
80%
20%

10. Doing the Research Right
Research should be publishable.
Strong methodology is vital. Are the experiments rigorous and valid?
Precise hypotheses
Ideally, hypotheses are based on previous literature or established theories
Appropriate statistical tests
Even if the contribution is small, good methodology can get a paper over the top.
Methodological flaws give reviewers an excuse to reject your work.

11. Right Research for Junior Faculty
The should be publishable, not necessarily suited for funding.
Extend your current work, don’t go down an entirely different path. Choose wisely in your Ph.D.
Use your methodology on other collections and different contexts.
Define an area, so people know you for your work.

12. Appropriate Packaging
Even solid work will be rejected if it is not appropriately packaged.
Main ideas of good packaging:
Be concise
Be professional
Target the journal or conference
Be persuasive

13. Appropriate Packaging
Good Slides
Force you to organize concisely and clearly,
Allow Dr. Chen to present the work to keep the $$$ coming, and
Reduce the time needed to write a good paper.
Good writing
Describes previous work in a digested form
Does not distract the reader
Makes a coherent argument
Employs good examples to illustrate difficult techniques or concepts
The transition from good slides to a good paper is 2-3 weeks.

14. Introduction – Looking Ahead
In future sections we will:
Review the main points: Lit Review
Present our Research Questions
Describe important environmental issues: Research Testbed
Discuss methodology: Research Design
Present our Results
Discuss the implications
Draw Conclusions and look to Future Work

15. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

16. Literature Review: What is it?
One of the most important parts of a paper
The lit review frames the work.
Connection between introduction and research questions
Introduction points out the motivation
Literature review provides more evidence of the limitations in previous studies
Following this logic flow, lit review leads to specific research questions
What to do? e.g., new algorithm, performance, etc.
Although, a paper is sometimes accepted largely because of a strong lit review that summarizes and organizes an area of inquiry

17. Literature Review: Key Idea
The literature review presents “digested material”
Taxonomies/Frameworks are good
A taxonomy of 2~4 dimensions
Know all studies in the field and focus on relevant ones
Tell what previous work means
Choose the right/relevant subset of all the papers you could cite
Don’t try to review everything, understand the audience of the paper

18. Literature Review: Completeness
Different level of completeness depending on journal/audience
In general, a more comprehensive review
For special issues, not too big, more focused
Different focus depending on your research question
To propose a new task,
To compare performance, or
Etc.

19. Literature Review: Tips
NOT*:
“use too much tutorial,”
“educating the reader,”
“lose the seminal works,”
“making sure we mentioned everything” (No laundry lists!), or
“too critical to others’ work” (Maybe he/she is the reviewer)
INSTEAD:
Enough coverage
Be Comprehensive
Critiques (what are missing leads to your research questions)
Show why our approach makes sense
Provide a benchmark for comparing our results
* Although, a paper is sometimes accepted largely because of a strong lit review that summarizes and organizes an area of inquiry

20. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

21. Research Questions
The intro said why. The lit review set up the argument.
Research Questions:
focus the work suggesting what we can measure,
follow logically from the lit review (address the critics in lit review and lead to your findings), and
should be answered by the experiment(s).

22. Research Questions Have 2-5 major research questions,
They should have clear scientific motivations:
Innovation to basic science, and
potential impacts.

23. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

24. Research Testbed What data sets will be used in the experiment(s)?
Testbed should be interesting, relevant, and significant.
We have:
Slides available from previous presentations
Published papers

25. Prepare Initial Slides
System Design
Describe how the architecture works and its components
The basic publication flow:
Write Grant Proposals
Topic Identification
Format and Submit
Prepare Initial Slides
Revise and Write
Revision Letters
Experimentation
Final Slides
Presentations
Conferences &
Funding Agencies
Write a Paper

26. System Design
Good diagrams help readers understand better and clarify our own thoughts.
Behavioral papers may have a methodology section instead of system design.
Algorithm papers discuss methodology and algorithm design (pseudo codes and diagrams are suggested) in this section.
Methodology needs to have a theoretical foundation.

27. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

28. Research Design: What’s in it?
Focus on the experiment (s).
Present hypotheses
Measurable
Address the research questions
Plan for statistical tests

29. Research Design: What’s in it?
Is to validate your research.
Use credible experiments to verify the hypotheses.
Methodology:
Quantitative measures: such as accuracy and speed.
Qualitative measures: explains the inside phenomena of the quantitative results.
Simulation: is often used in system design arena.

30. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

31. Research Findings Tables and figures are critical.
Need to be consistent and neat.
Highlight interesting numbers.
In caption, you may use 3-4 sentences to describe more details about a figure or a table.
Use a small paragraph in text to explain the essence about a figure or a table.
You may group your findings in chunks, each of which starts with a bold summarizing sentence.

32. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

33. Discussion The discussion section gives meaning to the results.
Why did you get the results you got?
If some of the results were surprising, why?
What did you observe outside the measured information presented in the research findings section?

34. Conclusions and Future Directions
Can have some duplication with the abstract.
State the contribution, but don’t overstate it. Don’t form questions in the reviewers mind.
Don’t mention trivial future directions.
Point to several promising directions.

35. References Where has similar work been published?
What kind of articles are accepted by the target journal?
Remember who did previous work.
Know where it was published.
Try to reference related papers that were published in the target journal.
Must have 5-10 key journals, key conferences in the field
Number the references
Have a consistent format

36. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

37. Professionalism Eliminate typos and grammar errors
Consistent formatting
Clear figures and tables
Captions make the meaning of the figure clear
The layout should be clear and clean
Every figure/table must be referenced in the text

38. Presentations Control your time; 40 slides not 75!
Present with energy and enthusiasm
Listen to questions; you can clarify before you answer
Don’t avoid questions, especially if they ask for specific information.
Rehearse; know what slides come next
Don’t read from your slides

39. Be concise. Consider the flow
Do you need this slide/sentence/word/paragraph?
Is the prose wordy?
Active sentences are better than passive.
Flow: Why is this point here?
Present info in a logical, top-down flow
Good: Therefore…..
Bad: You’ll see why later….

40. An example of good flow (1 of 2)
What is the problem?
Why do we care?
How has it been addressed before?
What is the research gap?
How are you going to address the research gap?
It should be clear from previous material or input here why you chose each part of your solution.

41. An example of good flow (2 of 2)
How will you measure the results?
What were your results?
Were they statistically significant?
What did you learn?
Why is that important?
What will you do next?

42. More things to think about
If you vary from the template you should have a good reason.
Dr. Chen resists incomplete slides.
Multiple revisions improve your slides. Get through a couple of revisions before you show them to Dr. Chen.
Let your colleagues help.

43. A challenge: Be ready with a good set of slides 3 days ahead.
Practice presenting them.
Tighten and revise.
Have a final version no less than 24 hours in advance.
Practice presenting the slides!

44. Agenda Abstract Introduction Background and Literature Review
Research Questions
Research Testbed, System Design
Research Design
Research Findings
Discussion
Conclusions and Future Directions
Additional Suggestions
Reviewing Papers

45. Reviewing Papers Be professional Be accurate Be specific Be critical
Pretend everyone will see your review
Be accurate
Be specific
Be critical
Of the methodology

46. Reviewing Papers Read similar papers Summarize the paper
Separate major and minor comments
Your review becomes your reputation
Don’t rewrite the article

47. Journal Reviews 1-3 pages (never less than ½) Selection Categories:
Include high level…
Summary
Strengths
Weaknesses
Selection Categories:
Accept as is
NEVER choose this one
Indicates laziness (Yours!)
Minor revisions
No methodology problems
Findings are interesting

48. Journal Reviews Selection categories cont…
Major revisions
Paper can be fixed through a new experiment
Needs significant clarification
Is incomplete
Rejection
Wrong methodology/implementation
Findings are trivial/uninteresting
Have I learned something new?
Wrong Journal
Suggest a different journal
Not substantial enough, recommend as a short note
Complete Journal Reviews in 2-3 months

49. Conference Review ½-1 page Accept only if light editing is necessary
No time for major overhauls
Is research and methodology interesting?
Complete conference reviews in 2-4 weeks