Software/Tools For Running Web Experiments By Robert Slater Texas A&M -Corpus Christi

Considerations Skills Skills Technology Technology Data Privacy Data Privacy Features Features

Skills What is your proficiency? What is your proficiency? –HTML, ASP, VB, JAVA, NONE Hire external party Hire external party –Do they understand what you are doing? –How important is their friendship? Learn new skills? Learn new skills? –learning curve.

Technology What type of computer systems will access the software? What type of computer systems will access the software? –Windows XP, ME, 2000, 98, 95, 3.1, DOS, MAC Will the code you have work on the different operating systems or web browsers? Will the code you have work on the different operating systems or web browsers? Do you have a web-server you can use to host your study? Do you have a web-server you can use to host your study? Would a third party notify you in advance of server maintenance? Would a third party notify you in advance of server maintenance?

Data Privacy Where will the data be stored and who will have access to it? Where will the data be stored and who will have access to it? If you use a third party hosting site can you keep your data confidential? If you use a third party hosting site can you keep your data confidential? Can you control backup copies of your data? Can you control backup copies of your data?

Features What kind of features do you need in your study? What kind of features do you need in your study? I needed: I needed: –Randomization into treatment groups –Piping with mathematical manipulation –Multiple Question formats –Data Export –Customization –User sign in options (specific/anonymous) –Java script capability within survey pages

My Solution Adapted an open source survey software to meet experimental needs ( Adapted an open source survey software to meet experimental needs ( ASP based open source software ASP based open source software Multiple database support – Multiple database support – –Supports data export to excel Professional looking finished product Professional looking finished product No programming experience needed No programming experience needed Quick learning curve Quick learning curve

SelectSurveyASP

SelectSurveyASP

Finished Product

Survey Options

Export Options

Limitations and Solutions Software not designed for multiple treatments: Software not designed for multiple treatments: –Create multiple questionnaires based on treatment groups Created one master survey and copied it to create separate treatments Created one master survey and copied it to create separate treatments Software would not randomize subjects: Software would not randomize subjects: –Use separate web page and basic java script to randomize subjects into treatment groups Software allowed hyperlinks to surveys Software allowed hyperlinks to surveys

Multiple Treatments

Limitations and Solutions (cont) Software allowed for piping but not mathematical manipulation as needed in my study. Software allowed for piping but not mathematical manipulation as needed in my study. I needed to be able to: I needed to be able to: –Take users estimate of Earnings Per Share –Give them managements estimate that was calculated as a percentage of users response.

Piping Take data input by participants and use in the study. Take data input by participants and use in the study. –Personalize questions –To develop rankings using data input by users.

Taking User Input

Using the Input

Capture Input for Ranking

Ranking The Captured Input

Piping Solution Program already allowed for piped responses. Program already allowed for piped responses. Open source software allowed me to manipulate the ASP source code to perform mathematical function on piped response. Open source software allowed me to manipulate the ASP source code to perform mathematical function on piped response.

Users Input EPS

Design Screen

Pipe Instructions

Actual Question Text

User View Piped Response

Conclusions Quick learning curve. Little expertise necessary. Quick learning curve. Little expertise necessary. Customizable software allowed for unique functionality. Customizable software allowed for unique functionality. Locally housed database on web server allowed for data privacy and further customized data gathering. (Timed responses) Locally housed database on web server allowed for data privacy and further customized data gathering. (Timed responses) Customized programming problems led to incompatibility with some operating systems. Customized programming problems led to incompatibility with some operating systems. –Participants were dropped from the study before completion.

Other Commercial Software ZoomerangSuperSurveyGreenfield Online InfopollPerseusPollCat InquisiteCool SurveysSurvey System Apian SoftwareHosted SurveySurveyView StatPacSurveyGoldSurveySite InstantSurveyEZSurveyMercator SurveyCrafterPollProLiveSurveys InSiteObjectPlanetSurveySaid QuickSense Active WebsurveySumQuest CustomerSatStatSurveySurveyMonkey *(list of survey companies found on

Recent Academic Studies on Web Experiments Bryant, B., Hunton, J., and Stone, D., 2004, Web Experiments: Prospects and Possibilities for Behavioral Accounting Research, Behavioral Research In Accounting, 16: 1-24 Bryant, B., Hunton, J., and Stone, D., 2004, Web Experiments: Prospects and Possibilities for Behavioral Accounting Research, Behavioral Research In Accounting, 16: 1-24 Birnbaum, M., 2004, Human Research and Data Collection via The Internet, Annual Review of Psychology, 55: Birnbaum, M., 2004, Human Research and Data Collection via The Internet, Annual Review of Psychology, 55: