ERPsim : A platform for multi-method experimental research in IS Prof. Pierre-Majorique Léger, Ph.D. Director ERPsim Lab, HEC Montréal Videoconference, University of Arkansas, Sept 17th 2010

Summary of the presentation: ERPsim research platform Usage data Psychophysiological data Performance data Psychome tric data Virtual time Direct observation Synchronization of IT usage, psychophysiological and psychometric data Research platform enabling realistic business IT environments in laboratory settings 2

About ERPsim Lab ERPsim Research platform Commercial platform Academic platform The mission of ERPsim Lab is to conduct research, development and transfer in the field of serious games in information technology. More than 100 universities worldwide are using ERPsim to teach ERP concepts to more than 3000 students annually Many large organisations such as Deloitte, SAP, Coca Cola and Croyola, are using ERPsim to train their end-users on ERP systems Several researchers have started using ERPsim to run experimental protocol in IS The academic version of ERPsim is available via the SAP University Alliance sdn.sap.com/ua 3

Simulated flight environment ERPsim: A flight simulator for ERP systems Simulated plane 4

ERPsim: A flight simulator for ERP systems Simulated business environment Real ERP system 5

What does ERPsim do ? ERP system (SAP)Participants 1)Business analytics 2)Business decisions 1)Automation of administrative tasks 2)Simulation of a market 3)Simulation of passing of time 6

© Léger et al ERPsim est une technologie propriétaire développée par des chercheurs de HEC Montréal, de l’École Polytechnique de Montréal et du Western Michigan Univ. ERPsim est détenu par Polyvalor et Valorisation Gestion et commercialisé par Baton Simulations. Plus d’info à erpsim.hec.ca Version : Sept 2009 Scope of the simulation 7

Sales Automated by ERPsim Planning & Procurement Automated by ERPsim Production Automated by ERPsim ERPsim : Automation of administrative tasks Léger, P.-M. (2006): Using a Simulation Game Approach to Teach Enterprise Resource Planning Concepts,” Journal of Information Systems Education 17: Léger, P.-M., Robert, J., Babin, G., Pellerin, R., Wagner, B. (2007): ERPsim, ERPsim Lab (erpsim.hec.ca), HEC Montreal, Montreal, Qc. 8

ERPsim academic platform

Academic platform: Pedagogical objectives Shows how the ERP system supports business strategies To develop a hands-on understanding of the concepts underlying enterprise systems To experience the tangible benefits of enterprise integration firsthand To develop technical skills at taking decision using an enterprise software 10

Paradigm shift in IT education Paradigm shift in the teaching From instructors to coaches From specialized technical knowledge to holistic business knowledge Paradigm shift in the learning From “point-and-click” to discovery of how SAP can help achieve business goals From technical questions to business strategy issues Charland, P., Leger, P-M, Feldstein, H., Robert, J., Babin, G., Lyle, D. « Business simulation training: Guideline for new training approaches in IT training », Proceeding of the ICT 2010 Conference, June 30 – July 2 nd, Singapour 11

Moving toward a competency approach Kang and Santhanam,

Research results on attitude and learning (Cronan et al, 2009a 2009b) The results suggest that the ERP simulation game was instrumental in ERP business process learning. Participant attitudes toward SAP and its use by employees were significantly higher following the simulation game experience. After three months on the job, new hires indicate that the learning levels as well as attitudes toward ERP/SAP remain high. Cronan, T. P.; Douglas, D. E.; Schmidt, P.; and Alnuaime, O. (2009a), “ERP Simulation Game; Learning and Attitudes toward SAP Samples Company First Time Hires, ITRI-WP , Information Technology Research Institute, University of Arkansas. Cronan, T. P. ; Douglas, D. E.; Schmidt, P.; and Alnuaime, O. (2009b), “Evaluating the Impact of an ERP Simulation Game on Student Knowledge, Skills, and Attitudes‘ ITRI-WP , Information Technology Research Institute, University of Arkansas. 13

Research results on self- perceived vs objective measure The results indicate significant correlation between the objective measures and the self- assessed measures of knowledge. Assessment of the validity of using perceived (self-assessed) measures as well as provide effective objective measures of learning in this ERP simulation context. Cronan, T. P.; Léger, P.-M.; Charland, P.; Robert, J.; Babin, G. (2010) “Comparing Objective Measures and Perceptions of Cognitive Learning in an ERP Simulation Game: A Research Note, ITRI-WPXXX-XXXX, Information Technology Research Institute, University of Arkansas. 14

Research results : ERP and Business Knowledge Correlation between ERP knowledge and performance Correlation between Business knowledge and performance Overall sample0,05 NS–0,01 NS Only top 10 %0,410 *0,496 ** Only lower 10 %–0,258 NS–0,216 NS Correlation between Performance and ERP and Business knowledge (n = 120) (*) significant at 10%; (**) significant at 5%, (NS) Non significant The correlation in the 2 last lines are non-parametric correlations. Léger, P.-M., Charland, P., Robert, J., Cronan, P. et Babin, G., Cronan, P., (2010) « Assessing the dimensionality of an objective measurement of ERP knowledge », SAP Curriculum Congress, March 20th

Research results : the role of GPA and complex knowledge Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE. BetaSig. (Constant)0.200NS GPA-0.074NS Knowledge Easy-0.259NS Knowledge Complex0.442* Experience0.538*** R2R2 30,4 %*** Wald chi *** * p<0.05, ** p<0.01, *** p<0.001 Regression (panels, n=480) 16

A platform for multi- method experimental research in IS

Usage data Psychophysiological data Performance data Psychometric data Virtual time Direct observation 18

Example of performance data in simulation game Net profits Virtual time 19

Example of usage data in SAP 20

Research results : Role transformation (1/2) Cluster analysis (20 subjects, 9 quarters) Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE. 21

Research results : Role transformation (2/2) Léger, P.-M. et al. (unpublished). PLEASE DO NOT CITE. 22

Psychophysiological data ProComp Infinity from Thought Technology Ltd. 23

Psychophysiological data: Our start-up kit ! EEG Sensor EMG Sensor EKG Sensor EDA Sensor Temperature Sensor ProComp Infiniti 2 x $ / unit $ / unit $ / unit $ / unit 24

Linking ERPsim and psychophysiological data Performance data (virtual time, ERP database) Usage data (real time - ERP database) Psychophysiological data (real time – laptop link to encoder) Psychometric data and observation data Virtual time line

Linking ERPsim and psychophysiological data (manual synchronisation) 26

Experimental platform Expected beta release date : Winter

Demonstration 28

Research results : Cognitive absorption (1/3) Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no 10-04(septembre 2010), HEC Montréal, 10 pages. 29

Research results : Cognitive absorption (2/3) Survey Day 1: Only SAM Survey Day 15: Only SAM Survey Day 30 : SAM + CA Q Discussion Q Discussion Q Discussion Game 1 Q Discussion Game 2 Q Discussion Q Discussion Q Discussion Game 3 Q Discussion Q Discussion Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no (septembre 2010), HEC Montréal, 10 pages. 30

Cognitive absorption Curiosity Control Temporal dimension Focus immersion Heightened enjoyment Heightened enjoyment The ability to register the passage of time while engaged in interaction The experience of total engagement where other attentional demands are ignored The pleasurable aspects of the interaction The user’s perception of being in charge of the interaction The extent to which the experience arouses an individual’s sensory and cognitive curiosity (Agarwal and Karahanna, 2000) 31

SAM Valence Arousal Dominance (Lang, 1995) 32

Research results : Cognitive absorption (3/3) Léger,P.-M., Davis, F.D., Perret, J. et Dunaway, M.M., « Psychophysiological Measures of Cognitive Absorption Patterns », Working paper, ERPsim Lab, no (septembre 2010), HEC Montréal, 10 pages. ValenceArousalDominance Temporal dissociation Heighted Enjoyment Curiosity Focused immersion ControlProfit Electrodermal response Corr.sigCorr.sigCorr.sigCorr.sigCorr.SigCorr.sigCorr.sigCorr.sigCorr.sig % EDA.248** * Moving average 1 min.267*** * 2 min.238** * ns 5 min.228** ns 10 min.171* ns Moving standard deviation 1 min ** ** 2 min **.314* * 5 min *** ** 10 min *.506*** *** Correlation between EDA, psychometric measures and profits SAM CA construct 33

Upcoming experiment at U. Arkansas Oct – Nov 2010 Novice, Easy simulation Challenges Skills Novice, Complex simulation Expert, Easy simulation Expert, Complex simulation Cognitive absorption + SAM every 10 minutes Skin conductance EKG EEG 34

Upcoming experiment at U. Arkansas Oct – Nov 2010 Debriefing Complex simulation Relaxation10 minutesCA survey 110 minutesCA survey 2 Easy simulation Relaxation10 minutesCA survey 110 minutesCA survey 2 Montage Consent formTrainingEDA, EMG and EEG 35

ERPsim : A platform for multi-method experimental research in IS Prof. Pierre-Majorique Léger, Ph.D. Director ERPsim Lab, HEC Montréal Videoconference, University of Arkansas, Sept 17th 2010