Multidimensional evaluation of DEA software DEA-software is keyfactor of acceptance in practice Question of implementation in pratice rising number of software tools requires a detailed and stakeholder-oriented overview Point of stakeholders view, especially practitioner Objective is not to show mistakes Objective is to give some suggestion to support e.g. usability, avoiding pitfalls, … Kersten Hentrich, Heinz Ahn July 8, 2009

Overview 1. Motivation 2. Requirements engineering 2.1 Stakeholder 2.2 Requirements engineering process 2.3 Structure of criteria catalogue 3. Evaluation and developing 3.1 State-of-the-art of DEA software 3.2 Approach of webbased developing 4. Conclusion

Bandwith of stakeholder Computer expert Practitioner User Developer Bandwith of stakeholder Scientist

2.2 Requirements engineering Requirements Engineering (RE) means not to hit one´s mark but the mark. feasible as process part of RE management systematic and iterative approach assumption: DEA basic model focus: application layer begin of searching detecting analyzing and rating attraction of requirements engineering to match interests to reach more stakeholder documentation document validation system specification

2.3 Criteria catalogue I Require-ments using requirements engineering process for searching requirements neither stakeholder-oriented nor structured weighting requirements and transforming to criteria minimum to must (binary) basis to normal level (binary) desireable to optional (different scales possible) weigthing require-ments criteria building a structure to sort criteria Strucuture of criteria software and economical aspects (non-functional) DEA theory (functional)

2.3 Criteria catalogue II Model selection Cost-efficieny Software Non-functional criteria Functional criteria Model selection Cost-efficieny Software Concretizing data Licence Analyzing Interaction Results of model- oriented outcome Services Report of references Content of folders consists of more than 60 criteria.

Overview 1. Motivation 2. Requirements engineering 2.1 Stakeholder 2.2 Requirements engineering process 2.3 Structure of criteria catalogue 3. Evaluation and developing 3.1 State-of-the-art of DEA software 3.2 Approach of webbased developing 4. Conclusion

3.1 State-of-the-art of DEA software filtered by: compatibility to testing platform, application runs „Out of the box“ CCR-I configurable, graphical User Interface, ability to load 50 DMUs, not webbased, 6 software-items, which include different types of licence for free or inserted in book or high pice. Commercial Software DEA-Solver-Pro Version 6.0 Frontier Analyst Version 4 KonSi DEA Analysis xIDEA 2.0 PIM DEASoft-V2 GAMS/DEA DEAFrontier LINDO Evaluated Software Frontier Analyst Version 4 PIM DEASoft-V2 DEAFrontier DEA-Solver-LV(V5) EMS Version 1.3 Win4DEAP 1.1.2 Non-commercial Software DEAP Version 2.1 Win4DEAP 1.1.2 MDeap 0.2.1 EMS Version 1.3 DEA-Solver-LV(V5) myDEA Qhull * iDEAs ** Dea Solver Online Version 1.2 ** Some highlights and shortcomings subsequently

+ - Win4DEAP 1.1.2 Practictioner Scientist Computer Expert For free Unlimited number of DMUs + Limited reporting Limited visualizing No guided interface No variation - No graphical interface for outcome, mixture of two languages No development since more than 5 years

+ - EMS 1.3 Practictioner Scientist Computer Expert For free Unlimited number of DMUs + Limited reporting Limited visualizing Limited declaration No guided interface No variation - No graphical interface for outcome, mixture of two languages No development since more than 5 years Only for academic use

+ - DEA-Solver-LV(V5) Practictioner Scientist Computer Expert literature and tutorial guided interface API , VBA and makros + Limited reporting No write-protected reports Limited variation - 50 DMUs knowledge of pre-constructed models

+ - DEAFrontier Practictioner Scientist Computer Expert literature and tutorial guided interface API , VBA and makros Technical support Comment function at outcome data + Limited reporting No write-protected reports Limited variation - Requirement: MS-Excel Max 199 DMUs at fullversion

+ - PIM DEASoft-V2 Practictioner Scientist Computer Expert Income- & outcome data locked Optionalexport of outcome-data Detailed upgrade possibilities Statistical features + Limited guided interface limited reporting limited visualizing No variation - Version of manual and software not identical

Frontier Analyst Version 4 Practictioner Scientist Computer Expert tutorial, support via phone, email guided Interface Many import and export formats Enterprise Licence Meaningful Visualiz- ing, Reporting & Visualizing + - No write-protected reports No variation

3.2 Approach of webbased developing structured criteria coded as webbased template for evaluation, development and information system Developer classifies features in template Customer evaluates via template (or extended via wiki, forum, …) searches via query for an adequate tool looks up new requirements Primary effect: matching interests Secondary effect: feedback of usability and pitfall

4. Conclusion especially non-functional requirements represent a strategic potential a large number of models does not mean better usability Results of stakeholder-oriented evaluation nearly all software tools are OR-oriented limited features and usability for practitioners only „myDEA“ available as Source-Code for computer experts Outlook: DEA software in progress which fulfills all requirements, open source based for computer experts

Thank you for your attention