ITEC6310 Research Methods in Information Technology Instructor: Prof. Z. Yang Course Website: http://people.math.yorku.ca/~zyang/itec6310.htm Office: Tel 3049

What is Research? There are many accepted definitions for the term “research”, for example: Research is an active, diligent and systematic process of inquiry in order to discover, interpret or revise facts, events, behaviours, or theories, or to make practical applications with the help of such facts, laws or theories. The term ‘research’ is also used to describe the collection of information about a particular subject

What is Research? Other insightful definitions of “research” are: Systematic, intensive, patient study and investigation in some field of knowledge, usually employing the techniques of hypothesis and experiment, whose purpose is to reveal new facts, theories, or principles Means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge Systematic study directed toward more complete scientific knowledge or understanding of the subject studied

Basic and Applied Research Basic research Conducted to evaluate theories or empirical positions Goal is to acquire information about a phenomenon Little emphasis on application to real-world problems

Basic and Applied Research (cont’d) Evaluates real-world problem Hypotheses may come from theory, but goal is to apply results to real-world problem Substantial overlap between basic and applied research Each type of research has implications for the other

The Research Process

Criteria for Identification of a Suitable Research Problem The research problem should address an important question so that the answer will make a difference. 2. The research problem should advance the frontiers of knowledge by leading to new ways of thinking, suggesting possible applications, or paving the way for further research in the field.

State the Assumptions Assumptions are basic to the research problem. All assumptions that have a material bearing on the problem should be openly and unreservedly set forth. A statement of the assumptions is necessary for others to evaluate the conclusions of the study. A statement of the assumptions reveals what the researcher may be taking for granted with respect to the problem.

Benefits of Conducting a Literature Review 1. It can offer new ideas, perspectives, and approaches that may not have occurred to you. 2. It can inform you about other researchers who conduct work in the same area. It can show you how others have handled methodological and design issues in studies similar to your own. It can reveal sources of data that you may not have known existed. It can introduce you to measurement tools that other researchers have developed and used effectively.

Benefits of Conducting a Literature Review (con’t) 6. It can reveal methods of dealing with problem situations that may be similar to difficulties you are facing. 7. It can help you interpret and make sense of your findings and, ultimately, help you tie your results to the work of those who have preceded you. 8. It will bolster your confidence that your topic is one worth studying, because you will find that others have invested considerable time, effort, and resources in studying it. In short: the more you know about investigations and perspectives related to your topic, the more effectively you can tackle your own research problem.

Case study You are reading a literature review written by a novice researcher in your field. You notice that the researcher seems to have relied heavily on the World Wide Web in preparing the literature review. Your hunch is supported when you glance at the reference page and find a long list of Internet addresses. What goes through your mind regarding the quality and adequacy of the literature review in this situation?

Organization of a Research Project 1. Title Page 2. Abstract 3. Statement of the Problem 4. Background and Literature Review 5. Research hypothesis and Null hypothesis 6. Definition of research variables and operationalization 7. Research design and strategy 8. Threats to internal and external validity 9. Sampling 10. Data collection 11. Data analysis and statistical tests 12. Conclusions 13. References

Preliminary Research Proposal in one or two pages states the research question states the purpose of the research sketched the initial model discusses (explains) the initial model identifies pertinent background literature

Choosing Variables for Your Study Research Tradition Your variables may be similar to those included in previous studies. You may use previously used dependent variables while manipulating new independent variables. Theory A theory on which you are relying may suggest certain variables to be included.

Choosing Variables for Your Study Availability of New Techniques Sometimes a new technique is developed, allowing you to observe a variable that previously could not be observed. Availability of Equipment The variables you manipulate or observe may be limited by the equipment available to you.

The RELIABILITY of a Measure A reliable measure produces similar results when repeated measurements are made under identical conditions.

The RELIABILITY of a Measure Reliability can be established in several ways Test-retest reliability: Administer the same test twice Parallel-forms reliability: Alternate forms of the same test used Split-half reliability: Parallel forms are included on one test and later separated for comparison

Example Indicate for each of the types of reliability listed below whether it measures stability over time, equivalency of items, or both of these things: Test-retest reliability Parallel-forms reliability Split-half reliability

ACCURACY of a Measure An accurate measure produces results that agree with a known standard. A measurement instrument can be inaccurate but reliable. The reverse cannot be true

The VALIDITY of a Measure A valid measure measures what you intend it to measure Validity can be established in a variety of ways Face validity: Assessment of adequacy of content. Least powerful method Content validity: How adequately does a test sample behavior it is intended to measure? Criterion-related validity: How adequately does a test score match some criterion score? Takes two forms Concurrent validity: Does test score correlate highly with score from a measure with known validity? Predictive validity: Does test predict behavior known to be associated with the behavior being measured? Construct validity: Do the results of a test correlate with what is theoretically known about the construct being evaluated?

Example You have developed a comprehensive test for research methods course that covers all aspects of the course. What type(s) of validity would you recommend establishing for this measure? Why is face validity not considered a true measure of validity? How is it possible for a test to be reliable but not valid?

Choosing Your Measures Using Established Versus New Measures You need not spend time assessing validity and reliability of established measures. However, they may not meet your research needs. New measures must be evaluated for validity and reliability, which takes time and effort. However, they may better meet your research needs.

Scales of Measurement Nominal Scale Ordinal Scale Lowest scale of measurement involving variables whose values differ by category (e.g., male/female) Values of variables have different names, but no ordering of values is implied Ordinal Scale Higher scale of measurement than nominal scale Different values of a variable can be ranked according to quantity (e.g., high, moderate, or low self-esteem)

Scales of Measurement Interval Scale Ratio Scale Scale of measurement on which the spacing between values is known (e.g., rating a book on a scale ranging from 0 to 10) No true zero point Ratio Scale Similar to interval scale, but with a true zero point (e.g., running time of a certain algorithm)

Factors Affecting Your Choice of a Scale of Measurement Information Yielded A nominal scale yields the least information. An ordinal scale adds some crude information. Interval and ratio scales yield the most information. Statistical Tests Available The statistical tests available for nominal and ordinal data (nonparametric) are less powerful than those available for interval and ratio data (parametric). Use the scale that allows you to use the most powerful statistical test.

Examples Identify the scale of measurement for each of the following variables: ZIP code Software cost SAT score Class rank Letter grade Correction rate