1. SENG 531: Labs TA: Brad Cossette brad.cossette@gmail.com cossette@cpsc.ucalgary.ca http://pages.cpsc.ucalgary.ca/~cossette/ Office Hours: Monday, Wednesday 3-4pm ICT 524

2. Labs This Week: Monday  Reading Academic Papers Wednesday  Presenting your Academic Paper

3. Background Most academic research in CS is disseminated through research papers The point of these papers is two-fold:  To disseminate new ideas or findings  To build the reputation of the researcher

4. Background There are three main venues for papers  Journals Most prestigious, well reviewed work Research is thorough  Conferences Can be as prestigious as a journal, though not often Research is good, maybe lacking some steps or evaluation  Workshops Conference-lite, much lower standards to get in Research is in very early stages, huge gaps may exist

5. Audience Papers are written  By CS researchers who are experts in a focused field  For CS researchers who have similar expertise This means they’re hard to read  Page limits & formats don’t help  Wording is formal, academic, downright stupid We chose CS over English for a reason!  Terminology is usually very precise

6. Papers: So What’s the Point? The authors are trying to convince you: 1. That they have a good idea 2. That they are solving a real problem 3. That their solution works To really understand a paper, you going to have to figure out if they succeeded on each of the 3 points above.

7. Papers: What you need to know You want to answer the following questions: 1. What’s the problem that they’re trying to solve? Do I think this is a real problem? 2. How do they solve the problem? 3. What problems or flaws exist with their approach or they paper?

8. Papers: A General Outline Most papers follow something like this: 1. Introduction & Motivation 2. The Solution or The Research 3. Evaluation of the Solution 4. Discussion of Evaluation 5. Related Work 6. Conclusion

9. Papers: Introduction & Motivation The Authors have to sell you that there’s a real problem worth solving Key Questions: 1. What’s the problem? 2. Why is this a problem? 3. What have we done so far to fix it? Why isn’t that good enough?

10. Papers: The Solution/Research The Authors are trying to convince you that they did significant work on this issue. This section depends heavily on the type of paper  Tool/Solution/Approach?  Understanding the problem in further detail?

11. Papers: The Solution/Research Key Questions: 1. What did they do? 2. Does their solution/approach make sense for the problem? If their solution doesn’t seem to fit the problem well, it may be a solution in search of a problem. 3. Did they provide you with enough information on what they did that you could also do it?  May be hard to answer if you’re not familiar with their field.

12. Papers: Evaluation Ideas are a dime a dozen, convince us that yours actually does something. Point of the Evaluation:  Convince you that their solution works  Convince you that it’s not a fluke  Discover interesting tidbits that will suggest further research

13. Evaluations: An Overview What are they doing?  Case Study, Field Study, Lab Experiment?  Evaluating a system/code or people? System  Open source?  Size?  Realism vs. Triviality? People  Undergrads vs. Grads vs. Professionals?

14. Evaluations: An Overview How are they doing it?  Is there enough information that you could replicate the experiment? What’s missing?  Does what they’re doing make sense? Is their experiment really testing their solution against the problem? Could other factors besides their solution explain the results?

15. Evaluations: An Overview What are the problems?  Authors are enthusiastic about their work, and will tend to overlook or minimize problems in their evaluation.  Problems can be good or bad: Negative results can tell you what parts of the solution aren’t working Unexpected results can warrant new research Your solution also might not make any improvement over existing approaches

16. Papers: Discussion of Results A lot of this feeds into the “Evaluations” section  Did their solution solve the original problem?  Authors should address any weaknesses in their approach  Results should indicate if this is still worth looking into  Related & Future work – don’t sweat this part too much.

17. Papers: Discussion of Results A lot of this feeds into the “Evaluations” section  Did their solution solve the original problem?  Authors should address any weaknesses in their approach  Results should indicate if this is still worth looking into

18. Papers: Related & Future Work Related Work  Mostly acknowledging previous contributions in the field  Shows the history of an idea Future Work  Often a research wish list  May show where the researchers are taking this now For your presentations, don’t worry too much about this section.

19. Papers: Conclusions Well, if you haven’t gotten the point by now here’s their last chance.

20. Presentations: How should you prepare? Your presentation has two parts:  Very good understanding of your paper  Some understanding of the other papers

21. Presentations: Your Paper & The 3 Questions 1. What’s the problem? Introduction & Motivation 2. How do they solve the problem? The Solution or Research Evaluation, Discussion of Results 3. Problems or flaws (Critique)? Motivation Evaluation Discussion of Results

22. Presentations: Their Paper & The 3 Questions Key sections to understand:  Abstract A very quick summary of the whole thing  Introduction & Motivation  High level understanding of Solution Evaluation Results  Conclusion

23. General Comments Reading academic papers is painful  Start as early as you can so can pace it out Your evaluation for your presentation is MOSTLY on content  Same idea as with your reflections: Do you understand it & can you explain it? Can you critique / analyze it?

24. General Comments Remember that your audience is not who your paper was written for  You need to show Rob you know what’s going on  It’d be nice to not sedate the rest of the class Some of the papers actually have interesting ideas Figure out what details are important

25. Labs Next Week: Monday  Demos Wednesday  Demos