1. Software Usability Course notes for CSI 5122 - University of Ottawa
2019 Deck D:
Course Skills – Research and Presentations
Timothy C. Lethbridge
< >

2. Presenting AN ORAL SUMMARY OF AN ASSIGNED PAPER
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

3. Papers for oral summary of an assigned paper 1
A list of papers and reports for you all to read has been sent to you
Read all the material, not just those you will be presenting
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

4. Papers for oral summary of an assigned paper 2
When you discuss a paper or report you should orally tell the class:
The main point of the paper or report (1 minute)
The methodology the authors used (if appropriate 1 min)
The results they obtained (if appropriate 1-2 mins)
The conclusions or recommendations (1-2 mins)
Any threats to validity of the conclusions that you see (1 min)
See discussion later in this deck
What you liked/and-or disliked about the paper/report (1-2 mins)
Time limit: 7 minutes to quickly discuss the above
But 5 minutes would be OK if you can say enough interesting things
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

5. Papers for oral summary of an assigned paper 3
Do not ‘read’ a speech you have written
Talk from point-form notes you have prepared
No powerpoint slides as we want to be quick
After you have spoken, I mayask one or two questions
And I may jump in if you look like you are taking two long
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

6. Conducting literature Research FOR YOUR RESEARCH PAPER
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

7. General Research Sources - 1
Databases that the University of Ottawa Library maintains a subscription to are listed below
Must be using an on-campus computer or connected to campus network
See for details on how to access the library’s digital resources
Scopus
Very good general meta-search for scientific information
Example searches to try
Use colour user interface
Cognitive walkthrough
Usability engineering
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

8. General Research Sources - 2
IEEE Xplore
ACM Digital library and giude to literature
Springerlink
Access to Springer journals and Lecture Notes in Computer Science
Google Scholar
Science Direct
HCIBib
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

9. Deck D: Course Skills - Research and Presentations
Key Journals
ACM transactions on Computer-Human Interaction
Human-Computer Interaction
International Journal of Human Computer Interaction
International Journal of Human Computer Studies
Interacting With Computers
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

10. Deck D: Course Skills - Research and Presentations
Other periodicals
ACM interactions
Magazine style
ACM SIGCHI Bulletin
Columns, reports, etc.
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

11. Deck D: Course Skills - Research and Presentations
Key conferences
Up to date list of some upcoming conferences
CHI
HCI International
Proceedings published by Springer
Interact
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

12. Key websites for usability
The Nielsen- Norman Group: Jakob Nielsen and Don Norman
Some stuff is free, other material is not
Don Norman’s Essays
Usability First
World Usability Day
Usability Professionals Association
Resource list
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

13. A sample of books on usability
See here for a good list of books
Newer books have a white background in the listing
Some particularly noteworthy classic books
H. Sharp, Y. Rogers, & J. Preece Interaction Design: Beyond Human-Computer Interaction, John Wiley & Sons, Inc., latest edition
Jeffrey Rubin and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. 2nd Edition. New York: Wiley, 2008
B. Shneiderman and C Plaisant. Designing User Interfaces: Strategies for Effective Human-Computer Interaction, Fifth Edition, 2009
Donald A. Norman. The Psychology of Everyday Things., 1990
Randolph G. Bias & Deborah J. Mayhew (Eds.) Cost-Justifying Usability. Boston: Academic Press, Updated edition, 2005
Jakob Nielsen. Usability Engineering, Academic Press, 1993
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

14. Deck D: Course Skills - Research and Presentations
Patent searches
US patents
Canadian patents
Do a search on
User interface
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

15. Examples of topics for your research paper - 1
A survey of usability metrics
Discuss papers where different metrics were used
Measuring individual differences
There is a large body of research literature on this
You could do a small experiment, or just report on some narrow topic from the literature
Making software usable for the disabled
Pick a type of disability and a class of application
You can combine literature research with your own evaluation
A comparison of software usability research and usability or ergonomics studies in other engineering disciplines
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

16. Research paper topic ideas - 2
International standards for usability
See
Also see
A study of how usability improvements have helped organizations financially.
Research on the effects on usability of various design issues
modality, choice of colours, menu design alternatives, etc.
Reliability and validity of usability testing
If you do two similar studies, will you get the same results?
What are the threats to validity?
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

17. Research paper topic ideas - 3
Rapid (discount) approaches to usability
Perhaps you could compare this to more expensive approaches
Ethics in usability testing
What different approaches are taken in different places
What issues arise
Effects of alternatives in experimental design
Focus on what aspect of experiments
Threats to validity
Blocking
Statistical analysis
Compare some experiments in the literature
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

18. Research paper topic ideas - 4
Usability studies of a certain class of applications
help systems
web search engines
word processors
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

19. Deck D: Course Skills - Research and Presentations
Threats to Validity
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

20. Deck D: Course Skills - Research and Presentations
No research is perfect
You should always consider factors that could mean that certain research results are less applicable
Always discuss these
When doing a literature review
When leading a paper discussion in class
When writing your own experiment paper
When presenting results to class
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

21. Deck D: Course Skills - Research and Presentations
Threats to validity - 1
Conclusion validity
Factors leading you to incorrectly believe some conclusion
Seeing things that aren’t there due to bias
Working with only one group of participants
Participants learn from one step to the next
Many other bias sources
Certain statistical errors, like doing many T-tests
Factors leading you to not reach a conclusion you should reach
Not finding the needle in the haystack
Not enough data
Not enough participants
Too much noise in the data
Not asking the right questions
Not using the right statistics
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

22. Deck D: Course Skills - Research and Presentations
Threats to validity - 2
Internal validity - are the relationships causal
Could something other than what you think be causing the results you see
Construct validity - did we measure what we wanted to measure?
External validity - can we generalize the results?
Are the results just true in this specific situation
Good website on threats to validity
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

23. Writing a Good Research Paper
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

24. Writing a good research paper (for this course) - 1
Step 1: Have something to say
Learn the relevant background
Search and read/skim the literature
Follow the citations to read
Papers that cite what you are reading
Papers that what you are reading cites
For studies you do yourself sketch out
Method you followed
Hypotheses
Research questions answered
Analysis of results
Threats to validity
Key conclusions
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

25. Writing a good research paper - 2
Step 2: Develop a structure
E.g.
Abstract
Introduction (write at end)
Background
Method
Results and discussion
Conclusions and future work (write at end)
References
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

26. Writing a good research paper - 3
Step 3: Fill in details
When you get stuck, work on a different section
Step 4: Review many times
Does it ‘tell a story’?
Are there details that could be left out?
Are important elements missing?
Is each sentence/paragraph well written?
Have you used citations/references well?
Ask somebody else to proofread
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

27. Write according to a publication format
Here are some possibilities
SIGCHI format
IEEE Format
Word document to use
ACM Format
Word document to use:
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

28. Some references on writing good papers
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

29. How we will evaluate research papers - 1
25% Writing quality
Clarity, conciseness, ease of reading
Document structure
Introductions, body, conclusions
If your first language is not English, you should consider getting somebody to proofread your report for you.
10% Quality of visual material
Tables, lists, diagrams, figures, and appendices.
If you are presenting data, focus on highlighting the most important aspects of the data.
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

30. How we will evaluate research papers - 2
25% Incorporation of background material
Synthesis of ideas from various sources
Journals and conferences (peer reviewed)
A goal is to cite 5 of these
Use online databases (discussed earlier)
Books
Good quality, reliable web-based material
Should not be a majority
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

31. How we will evaluate research papers - 3
Reference any relevant ideas discussed in the lectures.
Do not repeat material I discussed in class
Brief paraphrasing or quoting of my material is OK
Properly formatted the bibliography at the end
Give proper citations in the text for all the ideas you include that are not your own.
Quotations must be clearly marked.
At all costs, avoid accidental plagiarism
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

32. How we will evaluate research papers - 4
40% Depth of understanding, scientific maturity and/or originality
You do not have to discover new principles,
But a paper should be interesting for me to read, and make me think.
Appropriateness of study design and execution
If you are reporting on a study or experiment
report what you did in detail,
problems you encountered
the rationale for choices
Validity of conclusions
You will lose marks if you did everything in a rush at the end.
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

33. Deck D: Course Skills - Research and Presentations
Submitting Files To ME
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

34. How to submit files to me
Send them using a Brightspace link with the following naming convention
5122<lastname><contentDescription>.<xxx>
Where .xxx is .docx for draft reports, .docx or .pdf for final reports and .pptx for presentations
If you are sending me a draft, please add the word ‘Draft’ before the dot.
There is no need to print anything out that you submit
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

35. Doing a Good Presentation
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

36. Class presentation marking scheme - 1
25% Quality of overheads (or other A/V elements).
Use large fonts (20 point or higher) and point form.
Use diagrams, tables etc. if possible.
Each slide should have about 7-15 lines of text, or a graphic.
Avoid more than about 7 ‘chunks’ in a list
Use subheadings as on this page
Each point should take no more than 2 lines.
25% Organization of material presented
Did you provide enough background (but not to much)?
Did you explain the experimental or study design (if appropriate)?
Did you show interesting data?
Did you show conclusions?
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI

37. Class presentation marking scheme - 2
25% Delivery
Pacing (1 slide every 1-5 minutes)
Speaking clearly
Avoiding reading what is written
Leaving time for questions
Handling of questions
25% Information content
Amount the class would have learned from your presentation.
You will not get good marks if you just repeat material from lectures
Deck D: Course Skills - Research and Presentations
Lethbridge - CSI