Empirical Validation of UML Statechart Diagram Metrics: A Family of Three Experiments José A. Cruz-Lemus Department of Computer Science University of Castilla.

Slides:

Advertisements

Similar presentations

Cross Cultural Research

Advertisements

StatisticalDesign&ModelsValidation. Introduction.

Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 12 Measures of Association.

Causality Visualization Using Animated Growing Polygons Niklas Elmqvist Philippas Tsigas IEEE 2003 Symposium.

Culture and psychological knowledge: A Recap

Critiquing Research Articles For important and highly relevant articles: 1. Introduce the study, say how it exemplifies the point you are discussing 2.

1 User Centered Design and Evaluation. 2 Overview Why involve users at all? What is a user-centered approach? Evaluation strategies Examples from “Snap-Together.

1 A Framework for Measurement Valérie Paulus, Miguel Lopez, Gregory Seront, Simon Alexandre.

Empirically Assessing End User Software Engineering Techniques Gregg Rothermel Department of Computer Science and Engineering University of Nebraska --

Validation of the Method Adoption Model for Functional Size Measurement of Web Applications Silvia Abrahão Valencia University of Technology, Spain

BASIC STEPS OF CARRYING OUT RESEARCH  Select a research topic.  Formulate a research question/problem/statement of purpose.  A Research Problem is a.

Chapter 2 Research Process Part 2: Sept. 1, 2010.

Topics - Reading a Research Article Brief Overview: Purpose and Process of Empirical Research Standard Format of Research Articles Evaluating/Critiquing.

RESEARCH METHODS Lecture 19

Introduction to Communication Research

An Introduction to UML & Modeling Resources Using UML Rahul Shah EE249 Discussion Sept. 25, 2001.

Chapter 2 Research Process Part 1: Aug 29, Research Methods Importance of scientific method Research Process – develop ideas, refine ideas, test.

Research Methods in MIS

Today Questions about your own project? Review from last week What does Deets have to say about EBP? Who is known as the first nurse researcher? Did she.

Correlation Nabaz N. Jabbar Near East University 25 Oct 2011.

Factor Analysis Psy 524 Ainsworth.

I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted.

RESEARCH FRAMEWORK Yulia Sofiatin Department of Epidemiology and Biostatistics 2012 YS 2011.

© 2013 Cengage Learning. Outline  Types of Cross-Cultural Research  Method validation studies  Indigenous cultural studies  Cross-cultural comparisons.

Research in Psychology. Research Basics  All psychological research MUST follow the scientific method  Improves accuracy and validity of findings 

Chapter 3 An Overview of Quantitative Research

*Graduate School of Engineering Science, Osaka University

Evaluation of software engineering. Software engineering research : Research in SE aims to achieve two main goals: 1) To increase the knowledge about.

Cross-Cultural Research Methods. Methodological concerns with Cross-cultural comparisons  Equivalence  Response Bias  Interpreting and Analyzing Data.

Education 793 Class Notes Welcome! 3 September 2003.

1 Issues in Assessment in Higher Education: Science Higher Education Forum on Scientific Competencies Medellin-Colombia Nov 2-4, 2005 Dr Hans Wagemaker.

Introduction Algorithms and Conventions The design and analysis of algorithms is the core subject matter of Computer Science. Given a problem, we want.

The Scientific Method in Psychology.  Descriptive Studies: naturalistic observations; case studies. Individuals observed in their environment.  Correlational.

INTRO TO EXPERIMENTAL RESEARCH, continued Lawrence R. Gordon Psychology Research Methods I.

Correlational Research Chapter Fifteen Bring Schraw et al.

Experimentation in Computer Science (Part 1). Outline  Empirical Strategies  Measurement  Experiment Process.

EDU 8603 Day 6. What do the following numbers mean?

Reliability vs. Validity.  Reliability  the consistency of your measurement, or the degree to which an instrument measures the same way each time it.

What is Computer Science?  Three paradigms (CACM 1/89) Theory (math): definitions, theorems, proofs, interpretations Abstraction (science): hypothesize,

Slide 1 The introductory statement in the question indicates: The data set to use (2001WorldFactBook) The task to accomplish (association between variables)

Research Methods. Conducting Research in the Social Sciences The purposes of the social sciences are: – To describe and explain the behaviour of individuals.

Copyright © Allyn & Bacon 2008 Intelligent Consumer Chapter 14 This multimedia product and its contents are protected under copyright law. The following.

Validity: Introduction. Reliability and Validity Reliability Low High Validity Low High.

Anatomy of a Research Article Five (or six) major sections Abstract Introduction (without a heading!) Method (and procedures) Results Discussion and conclusions.

Experimentation in Computer Science (Part 2). Experimentation in Software Engineering --- Outline  Empirical Strategies  Measurement  Experiment Process.

Evaluating VR Systems. Scenario You determine that while looking around virtual worlds is natural and well supported in VR, moving about them is a difficult.

West Virginia University Sherif Yacoub, Hany H. Ammar, and Ali Mili A UML Model for Analyzing Software Quality Sherif Yacoub, Hany H. Ammar, and Ali Mili.

Chapter Eight: Quantitative Methods

Revision of Course: Lecture 1 – 15 Lecture 31 Research Tools and Techniques.

Lecture №4 METHODS OF RESEARCH. Method (Greek. methodos) - way of knowledge, the study of natural phenomena and social life. It is also a set of methods.

Basic Research Terms and Methods Goals of psychological research Measurement and description of behavior Understanding and prediction of behavior Application.

Logic of Hypothesis Testing

Requirements for the Course

Research Methods in I/O Psychology

Writing a sound proposal

BSc Computing and Information Systems Module: M2X8630 Research and Development Methods Introduction to Research Methods.

Selecting the Best Measure for Your Study

Oleh: Beni Setiawan, Wahyu Budi Sabtiawan

Title: Validating a theoretical framework for describing computer programming processes 29 November 2017.

Empirical Project.

Chapter Eight: Quantitative Methods

SCIENCE AND ENGINEERING PRACTICES

RESEARCH METHODS Lecture 19

Quantitative vs Qualitative Research

Research Methods & Statistics

Managerial Decision Making and Evaluating Research

Chapter 8 VALIDITY AND RELIABILITY

CS 594: Empirical Methods in HCC Experimental Research in HCI (Part 1)

Causal Comparative Research Design

Presentation transcript:

Empirical Validation of UML Statechart Diagram Metrics: A Family of Three Experiments José A. Cruz-Lemus Department of Computer Science University of Castilla – La Mancha (Spain)

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Outline Introduction Family of Experiments Definition Planning Operation Analysis and Interpretation Validity Conclusions

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Introduction Guarantee quality characteristics in OO systems since the initial lifecycle stages. MDD & MDA: conceptual models are backbone of OO systems development. Scarce references to behavioural diagrams.

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Introduction SizeNEntryANExitANANSSNCSNGNE Structural Complexity NTCC (Genero et al., 2002) Theoretical validation Briand et al. Poels and Dedene Empirical validation Metrics for UML Statechart Diagrams (SD)

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Outline Introduction Family of Experiments Family of Experiments Definition Definition Planning Planning Operation Operation Analysis and Interpretation Analysis and Interpretation Validity Validity Conclusions

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Chronology

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Definition Analyze Structural complexity and size metrics for UML SD For the purpose of purpose ofEvaluating With respect to respect to The capability of being used as indicators of the understandability of UML SD From the point of view of of view ofResearchers In the context of context of Undergraduate students of Computer Science and SE teachers of the Computer Science Department at the University of Castilla - La Mancha

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Planning E1R1R2 Context 8 teachers 10 fifth-year students 24 third-year students 49 third-year students Subjects high / averagelow experience Variables structural complexitysize Indep.: UML SD structural complexity and size understandability Dep.: UML SD understandability Instrumentation 20 UML SD Independent: metrics metrics Dependent: understandability time understandability time correctness (correct vs. answered) correctness (correct vs. answered) completeness (correct vs. asked) completeness (correct vs. asked)

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Planning E1R1R2 Hypotheses H 0,1 No correlation UML SD structural complexity and size - understandability time H 0,2 No correlation UML SD structural complexity and size - correctness H 0,3 No correlation UML SD structural complexity and size - completeness Design Within-subjects. Diagrams in different orders.

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Operation

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Operation

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Analysis and Interpretation

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Analysis and Interpretation

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Analysis and Interpretation NEntryANExitANANSSNCSNENGNTCC UTime Correct Complt UTime Correct Complt Utime Correct Complt α = 0.05E1R1R2

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Family of Experiments Validity Conclusion No pupulation sample (small and voluntary participation) ‘Pencil & Paper’ Construct Theoretical validation Internal Differences within subjects, knowledge of the universe of discourse, precision in time checking, learning and fatigue effects, motivation, plagiarism... External Materials and task used, not real projects data. Subjects. Expericenced enough.

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Outline Introduction Family of Experiments Definition Planning Operation Analysis and Interpretation Validity Conclusions

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Conclusions Metrics for structural complexity and size of UML Statechart Diagrams. Family of experiments. Descriptive statistics and Spearman’s correlation coefficients.  Experience  Results NA, NSS, NG, NT highly correlated to the understandability time.

Workshop on Software Metrics Validation - Ghent (Belgium) - July 6th Conclusions Future work Further validation. Practitioners and profesionals. Real projects data. Prediction model based on the metrics....

Empirical Validation of UML Statechart Diagram Metrics: A Family of Three Experiments José A. Cruz-Lemus Department of Computer Science University of Castilla – La Mancha (Spain)