O BTAINING KEY PERFORMANCE INDICATORS BY USING DATA MINING TECHNIQUES Lucentia Research Group Department of Software and Computing Systems Roberto Tardío & Jesús Peral UNIVERSITY OF ALICANTE

1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion O UTLINE

1. I NTRODUCTION Dashboards and Scorecards (Kaplan et al., 1996)  decision makers to quickly assess the status of an organization. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Dashboards  the preferred tool across organizations to monitor business performance. Key Performance Indicators (KPIs) (Parmenter, 2015) play a crucial role, since they facilitate quick and precise information by comparing current performance against a target required to fulfill business objectives.

1. I NTRODUCTION KPIs are not always well known sometimes it is difficult to find an adequate KPI to associate with each business objective (Angoss, 2011). 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Organizations use existing lists of KPIs An organization performs an innovative activity KPIs may be redundant (Rodríguez et al., 2009), misdirecting the effort and resources of the organization. people responsible for (wrong) KPIs develop a resistance to change once they have found how to maximize their value (Parmenter, 2015). there is a tendency to focus on results themselves (Parmenter, 2015; Angoss, 2011 ) (e.g. Sales) rather than on the actual indicators that can be worked on (e.g. Successful deliveries/Total deliveries) and lead to the results obtained.

1. I NTRODUCTION There is a need for techniques and methods that improve the KPI elicitation process, providing decision makers with information about relationships between KPIs and their characteristics. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion the implications of the data for the company are unknown, and, thus, eliciting their relationships with internal KPIs can make these data actionable, adding value to them. Big Data

1. I NTRODUCTION Big Data  huge volume, complex and heterogeneous sources 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Visualization. What You See Is What You Get. Only when the analytical results are friendly displayed, it may be effectively utilized by users KPIs elicitation

1. I NTRODUCTION Our approach combines these two aspects: to drive data mining techniques. obtaining specific KPIs for business objectives in a semi- automatic way. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion The main benefit of our approach organizations do not need to rely on existing KPI lists. In order to show the applicability of our approach we apply our proposal to the novel field of MOOC (Massive Open Online Course) courses in order to identify additional KPIs to the ones being currently used.

2. B ACKGROUND (Kaplan et al., 1996)  Balanced Scorecard, a tool that consists on a balanced list of KPIs associated with objectives covering different business perspectives. (Kaplan et al., 2004)  Strategy Map, describes the way that the organization intends to achieve its objectives, by capturing the relationships between them in an informal way. (Horkoff et al., 2014)  Business strategy models, combine KPIs, objectives, and their relationships all together in a single formal view. Are the KPIs adequate?? 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

2. B ACKGROUND (Parmenter, 2015)  the design and implementation of KPIs within Dashboards. The author differentiates between Key Result Indicators (KRIs) and KPIs (Rodríguez et al., 2009)  the QRPMS method to select KPIs and elicit relationships between them. The method starts from a pre-existing set of candidate KPIs, and performs a series of analysis steps. using data mining techniques 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

2. B ACKGROUND Big Data datasets that we can not manage with current methodologies or data mining software tools principally due to their huge size and complexity. Big Data mining is the capability of extracting useful information from these large datasets or streams of data. New mining techniques are necessary due to the volume, variability, and velocity, of such data. The Big Data challenge is becoming one of the most exciting opportunities for the years to come. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

2. B ACKGROUND There are a number of works focused on monitoring performance by means of KPIs However, most of the works that tackle the problem of KPI selection require a pre-existing set of KPIs. Obtaining this set of KPIs can be a tough task in already established organizations (Angoss, 2011), becomes a challenge when the business activity is developed in an innovative environment. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

3. M ETHODOLOGY STAGES 1 & 2 First of all, we start by focusing on modeling the business strategy and known KPIs (if any) to guide the process. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Business strategy model includes: The relationships between the different business objectives to be achieved (optionally) The processes that support them (the objectives). The dependencies are modeled in a semiautomatic mode.

1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

3. M ETHODOLOGY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion The aim of this step is to relate business objectives with entities and measures that are related to their performance. a set of candidate KPIs for each objective is defined. Analysis to merge the information multidimensional model for analysis STAGES 3 & 4 Decision makers  provide the required information to fulfill the objectives. By interviewing these decision makers we can create new user requirement views  in order to implement the DW.

3. M ETHODOLOGY STAGES 5 & 6 The multidimensional model allows the mapping from the indicators to DW elements  DW schema is generated automatically. 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion The following step is to analyze the candidate KPIs through data mining techniques to ensure that they reflect the relationships identified during business strategy modeling. Finally, we define or update the analysis views for different roles, materialized in dashboards that will allow decision makers to access and monitor the new KPIs.

4. C ASE S TUDY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion The effect of the globalization along with the proliferation of open online courses has radically changed the traditional sectors of education. New technologies symbolise a big opportunity it is also required to face significant challenges to take full advantages of them. Massive Open Online Course (MOOC) an online course with the objective of interacting and promoting participation and open access via the web. slides, video lectures (off-line and on-line), user forums… gain popularity: number of students has increased exponentially during the last years.

4. C ASE S TUDY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion We present the process followed to elicit and model the critical information from the MOOC named UniMOOC (Platform Courses for Entrepreneurs of the University of Alicante). UniMOOC is a MOOC currently has over unique 20,000 students registered and focuses on entrepreneurship. the course includes several units and modules as well as links to social networks for students to interchange opinions.

4. C ASE S TUDY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion STAGES 1 & 2

4. C ASE S TUDY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion STAGES 3 & 4 Interviews with the organizers of this course. A first set of indicators were obtained in a generic way: increment in number of students, dropout ratio, recovery ratio of students, % Of active students, % Of students who fail the course, etc. An initial version of the multidimensional model for analysis. two analysis cubes: Enrollment and Activity. Enrollment, allows us to analyze if the characteristics of the students, such as country, interests and expectations present certain patterns.

1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

4. C ASE S TUDY 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion STAGES 5 & 6 We have started by applying the classical data mining techniques to the database of the course. Due to the big amount of data of this course  these techniques are not very suitable because they are difficult to interpret: they produce a lot of rules in association rules and decision trees. they also produce many hidden neural connections in the artificial neural networks. The best way to analyse these data is by using visualization methods. the visualization techniques allow to see how the graphical grow dynamically.

1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion

5. D ISCUSSION 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Dashboards are the preferred tool across organizations to monitor business performance. Different data visualization techniques Key Performance Indicators (KPIs) play a crucial role in facilitating quick and precise information by comparing current performance against a target required to fulfill business objectives.

5. D ISCUSSION 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Very often it is difficult to find an adequate KPI to associate with each business objective. The main objective is to obtain specific KPIs for business objectives in a semi-automatic way. This approach is illustrated with a case study, a MOOC course, which is a very novel area and therefore very suitable for their purpose.

5. F UTURE WORK 1. Introduction 2. Background 3. Methodology 4. Case study 5. Discussion Automatic extraction of KPIs from Business strategy model. Student interviews and feedbacks. Data Mining techniques (supervised, unsupervised, hybrid) to check the correlation. Big Data environments extract KPIs from data. visualization methods.

QUERIES?