A Measure of Common Language Effect Size Differences in Information Systems Success

Four Paradigms As Described by Hirschheim & Kline (1989) Functionalism (1) Social Relativism(2) Radical Structuralism(3) Neohumanism(4)

Functionalism Order Social Relativism Objectivism Subjectivism Neohumanism Radical Structuralism Conflict

Question? System design implies assumptions. Hirschheim & Kline (1989) Do the assumptions of the developer impact the success of the system?

Methodological Outline 1. Test for significant differences of success between competing paradigms. 2. Compute the Common Language Effect Size difference of the levels of success between competing paradigms. 3. Describe the implications of the Common Language Effect Size differences.

1. Test for significant differences of success between competing paradigms. A survey instrument will be developed to assess the attitude of the system developer at the time he begins system design. Each time the developer begins a new system, he will be required to complete the survey (longitudinal).

1. Test for significant differences of success between competing paradigms. After the completion of the system, end-users will receive a survey to measure satisfaction. Satisfaction will be measured by easy of use and usefulness constructs.

1. Test for significant differences of success between competing paradigms. Based on the developer’s response to the survey, his design approach will be classified in one of the four Hirschheim and Kline categories.

Compilation of Data

Testing After a large sample of survey responses have been accumulated, (N>60 for each subject and each category as well as satisfaction surveys from end-users) statistical testing may begin.

Testing Manova may be an appropriate statistical test assuming that the surveys are designed with discrete measurement variables and continuous inferred outputs.

2. Compute the Common Language Effect Size difference of the levels of success between competing paradigms. After a significant p value is obtain, a Common Language Effect Size (CL) should be compute to report the magnitude of difference of success among paradigms. CL is a transformation, like Z-testing, to quantify a common unit of measurement for a variable.

3. Describe the implications of the Common Language Effect Size differences. Explain the implication of a small, medium, or large effect size. Report the power attained by the statistical test (No less than .8) If power falls below .8, break study into segments to avoid bonferroini aggregation of alpha.