1. ITEC6310 Research Methods in Information Technology
Instructor: Prof. Z. Yang
Course Website:
Office: Tel 3049

2. What is Research? There are many accepted definitions for the term
“research”, for example:
Research is an active, diligent and systematic process of inquiry in order to discover, interpret or revise facts, events, behaviours, or theories, or to make practical applications with the help of such facts, laws or theories. The term ‘research’ is also used to describe the collection of information about a particular subject

3. What is Research? Other insightful definitions of “research” are:
Systematic, intensive, patient study and investigation in some field of knowledge, usually employing the techniques of hypothesis and experiment, whose purpose is to reveal new facts, theories, or principles
Means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge
Systematic study directed toward more complete scientific knowledge or understanding of the subject studied

4. Basic and Applied Research
Basic research
Conducted to evaluate theories or empirical positions
Goal is to acquire information about a phenomenon
Little emphasis on application to real-world problems

5. Basic and Applied Research (cont’d)
Evaluates real-world problem
Hypotheses may come from theory, but goal is to apply results to real-world problem
Substantial overlap between basic and applied research
Each type of research has implications for the other

6. The Research Process

7. Criteria for Identification of a Suitable
Research Problem
The research problem should address
an important question so that the answer will make a difference.
2. The research problem should advance the frontiers of knowledge by leading to new ways of thinking, suggesting possible applications, or paving the way for further research in the field.

8. State the Assumptions Assumptions are basic to the research problem.
All assumptions that have a material bearing on the problem should be openly and unreservedly set forth.
A statement of the assumptions is necessary for others to evaluate the conclusions of the study.
A statement of the assumptions reveals what the researcher may be taking for granted with respect to the problem.

9. Benefits of Conducting a Literature Review
1. It can offer new ideas, perspectives, and approaches that may not have occurred to you.
2. It can inform you about other researchers who conduct work in the same area.
It can show you how others have handled methodological and design issues in studies similar to your own.
It can reveal sources of data that you may not have known existed.
It can introduce you to measurement tools that other researchers have developed and used effectively.

10. Benefits of Conducting a Literature
Review (con’t)
6. It can reveal methods of dealing with problem situations that may be similar to difficulties you are facing.
7. It can help you interpret and make sense of your findings and, ultimately, help you tie your results to the work of those who have preceded you.
8. It will bolster your confidence that your topic is one worth studying, because you will find that others have invested considerable time, effort, and resources in studying it.
In short: the more you know about investigations and perspectives related to your topic, the more effectively you can tackle your own research problem.

11. Case study
You are reading a literature review written by a novice researcher in your field. You notice that the researcher seems to have relied heavily on the World Wide Web in preparing the literature review. Your hunch is supported when you glance at the reference page and find a long list of Internet addresses. What goes through your mind regarding the quality and adequacy of the literature review in this situation?

12. Organization of a Research Project
1. Title Page 2. Abstract 3. Statement of the Problem 4. Background and Literature Review 5. Research hypothesis and Null hypothesis 6. Definition of research variables and operationalization 7. Research design and strategy 8. Threats to internal and external validity 9. Sampling 10. Data collection 11. Data analysis and statistical tests 12. Conclusions 13. References

13. Preliminary Research Proposal
in one or two pages
states the research question
states the purpose of the research
sketched the initial model
discusses (explains) the initial model
identifies pertinent background literature

14. Choosing Variables for Your Study
Research Tradition
Your variables may be similar to those included in previous studies.
You may use previously used dependent variables while manipulating new independent variables.
Theory
A theory on which you are relying may suggest certain variables to be included.

15. Choosing Variables for Your Study
Availability of New Techniques
Sometimes a new technique is developed, allowing you to observe a variable that previously could not be observed.
Availability of Equipment
The variables you manipulate or observe may be limited by the equipment available to you.

16. The RELIABILITY of a Measure
A reliable measure produces similar results when repeated measurements are made under identical conditions.

17. The RELIABILITY of a Measure
Reliability can be established in several ways
Test-retest reliability: Administer the same test twice
Parallel-forms reliability: Alternate forms of the same test used
Split-half reliability: Parallel forms are included on one test and later separated for comparison

18. Example
Indicate for each of the types of reliability listed below whether it measures stability over time, equivalency of items, or both of these things:
Test-retest reliability
Parallel-forms reliability
Split-half reliability

19. ACCURACY of a Measure
An accurate measure produces results that agree with a known standard.
A measurement instrument can be inaccurate but reliable.
The reverse cannot be true

20. The VALIDITY of a Measure
A valid measure measures what you intend it to measure
Validity can be established in a variety of ways
Face validity: Assessment of adequacy of content. Least powerful method
Content validity: How adequately does a test sample behavior it is intended to measure?
Criterion-related validity: How adequately does a test score match some criterion score? Takes two forms
Concurrent validity: Does test score correlate highly with score from a measure with known validity?
Predictive validity: Does test predict behavior known to be associated with the behavior being measured?
Construct validity: Do the results of a test correlate with what is theoretically known about the construct being evaluated?

21. Example
You have developed a comprehensive test for research methods course that covers all aspects of the course. What type(s) of validity would you recommend establishing for this measure? Why is face validity not considered a true measure of validity? How is it possible for a test to be reliable but not valid?

22. Choosing Your Measures
Using Established Versus New Measures
You need not spend time assessing validity and reliability of established measures.
However, they may not meet your research needs.
New measures must be evaluated for validity and reliability, which takes time and effort.
However, they may better meet your research needs.

23. Scales of Measurement Nominal Scale Ordinal Scale
Lowest scale of measurement involving variables whose values differ by category (e.g., male/female)
Values of variables have different names, but no ordering of values is implied
Ordinal Scale
Higher scale of measurement than nominal scale
Different values of a variable can be ranked according to quantity (e.g., high, moderate, or low self-esteem)

24. Scales of Measurement Interval Scale Ratio Scale
Scale of measurement on which the spacing between values is known (e.g., rating a book on a scale ranging from 0 to 10)
No true zero point
Ratio Scale
Similar to interval scale, but with a true zero point (e.g., running time of a certain algorithm)

25. Factors Affecting Your Choice of a Scale of Measurement
Information Yielded
A nominal scale yields the least information.
An ordinal scale adds some crude information.
Interval and ratio scales yield the most information.
Statistical Tests Available
The statistical tests available for nominal and ordinal data (nonparametric) are less powerful than those available for interval and ratio data (parametric).
Use the scale that allows you to use the most powerful statistical test.

26. Examples
Identify the scale of measurement for each of the following variables:
ZIP code
Software cost
SAT score
Class rank
Letter grade
Correction rate