SENG 531: Labs TA: Brad Cossette Office Hours: Monday, Wednesday 3-4pm ICT 524

Labs This Week: Monday  Reading Academic Papers Wednesday  Presenting your Academic Paper

Background Most academic research in CS is disseminated through research papers The point of these papers is two-fold:  To disseminate new ideas or findings  To build the reputation of the researcher

Background There are three main venues for papers  Journals Most prestigious, well reviewed work Research is thorough  Conferences Can be as prestigious as a journal, though not often Research is good, maybe lacking some steps or evaluation  Workshops Conference-lite, much lower standards to get in Research is in very early stages, huge gaps may exist

Audience Papers are written  By CS researchers who are experts in a focused field  For CS researchers who have similar expertise This means they’re hard to read  Page limits & formats don’t help  Wording is formal, academic, downright stupid We chose CS over English for a reason!  Terminology is usually very precise

Papers: So What’s the Point? The authors are trying to convince you: 1. That they have a good idea 2. That they are solving a real problem 3. That their solution works To really understand a paper, you going to have to figure out if they succeeded on each of the 3 points above.

Papers: What you need to know You want to answer the following questions: 1. What’s the problem that they’re trying to solve? Do I think this is a real problem? 2. How do they solve the problem? 3. What problems or flaws exist with their approach or they paper?

Papers: A General Outline Most papers follow something like this: 1. Introduction & Motivation 2. The Solution or The Research 3. Evaluation of the Solution 4. Discussion of Evaluation 5. Related Work 6. Conclusion

Papers: Introduction & Motivation The Authors have to sell you that there’s a real problem worth solving Key Questions: 1. What’s the problem? 2. Why is this a problem? 3. What have we done so far to fix it? Why isn’t that good enough?

Papers: The Solution/Research The Authors are trying to convince you that they did significant work on this issue. This section depends heavily on the type of paper  Tool/Solution/Approach?  Understanding the problem in further detail?

Papers: The Solution/Research Key Questions: 1. What did they do? 2. Does their solution/approach make sense for the problem? If their solution doesn’t seem to fit the problem well, it may be a solution in search of a problem. 3. Did they provide you with enough information on what they did that you could also do it?  May be hard to answer if you’re not familiar with their field.

Papers: Evaluation Ideas are a dime a dozen, convince us that yours actually does something. Point of the Evaluation:  Convince you that their solution works  Convince you that it’s not a fluke  Discover interesting tidbits that will suggest further research

Evaluations: An Overview What are they doing?  Case Study, Field Study, Lab Experiment?  Evaluating a system/code or people? System  Open source?  Size?  Realism vs. Triviality? People  Undergrads vs. Grads vs. Professionals?

Evaluations: An Overview How are they doing it?  Is there enough information that you could replicate the experiment? What’s missing?  Does what they’re doing make sense? Is their experiment really testing their solution against the problem? Could other factors besides their solution explain the results?

Evaluations: An Overview What are the problems?  Authors are enthusiastic about their work, and will tend to overlook or minimize problems in their evaluation.  Problems can be good or bad: Negative results can tell you what parts of the solution aren’t working Unexpected results can warrant new research Your solution also might not make any improvement over existing approaches

Papers: Discussion of Results A lot of this feeds into the “Evaluations” section  Did their solution solve the original problem?  Authors should address any weaknesses in their approach  Results should indicate if this is still worth looking into  Related & Future work – don’t sweat this part too much.

Papers: Discussion of Results A lot of this feeds into the “Evaluations” section  Did their solution solve the original problem?  Authors should address any weaknesses in their approach  Results should indicate if this is still worth looking into

Papers: Related & Future Work Related Work  Mostly acknowledging previous contributions in the field  Shows the history of an idea Future Work  Often a research wish list  May show where the researchers are taking this now For your presentations, don’t worry too much about this section.

Papers: Conclusions Well, if you haven’t gotten the point by now here’s their last chance.

Presentations: How should you prepare? Your presentation has two parts:  Very good understanding of your paper  Some understanding of the other papers

Presentations: Your Paper & The 3 Questions 1. What’s the problem? Introduction & Motivation 2. How do they solve the problem? The Solution or Research Evaluation, Discussion of Results 3. Problems or flaws (Critique)? Motivation Evaluation Discussion of Results

Presentations: Their Paper & The 3 Questions Key sections to understand:  Abstract A very quick summary of the whole thing  Introduction & Motivation  High level understanding of Solution Evaluation Results  Conclusion

General Comments Reading academic papers is painful  Start as early as you can so can pace it out Your evaluation for your presentation is MOSTLY on content  Same idea as with your reflections: Do you understand it & can you explain it? Can you critique / analyze it?

General Comments Remember that your audience is not who your paper was written for  You need to show Rob you know what’s going on  It’d be nice to not sedate the rest of the class Some of the papers actually have interesting ideas Figure out what details are important

Labs Next Week: Monday  Demos Wednesday  Demos