Cognitive Metrics for the Management of Model-based Software Development Reuven Gallant, Yehuda Badihi, Hagai Sugarman Jerusalem College of Technology-Machon Lev JCT

Why are managers afraid of model-based development? It sounds good notation method/process tool

Why are managers afraid of model-based development?

They like pictures Why sw engineers do/don’t like models (1) They don’t like tools –Or are lacking motivation to use them They don’t have patience for semantics  NST:  (US)  (US)  : (s, s')  s – s'  +  s' – s  They like pictures

Is a picture worth 1000 words? “To set the alarm, the key must be inserted into the lock, turned anti clockwise, and held for at least half a second. “To reset the alarm, the key must be inserted into the lock and turned clockwise. The alarm will be deactivated immediately.”

Automobile Alarm

Why sw engineers do/don’t like models (2)

Key Characteristics of Engineering Models Abstraction Understandability Accuracy Predictiveness Inexpensive Brian Selic, “The Pragmatics of Model-Driven Development,” IEEE Software, Sept./Oct

Where measurements can help us manage model-based development Tool Evaluation People Evaluation Training Evaluation Model Evaluation

Metrics must be simple and easy to automate Fenton N. and Neil M. (2000). Software Metrics: a Roadmap. Future of Software Engineering. Anthony Finkelstein Ed., ACM,

Bottom Up Tool Evaluation

Gestalt Approach to Tool Evaluation R. Gallant, M. Winokur, and J. Kudish, "Tool and Method Reciprocity: A Case Study in Requirements Management," Proceedings of The Seventh Israel Conference on Computer Systems and Software Engineerinתg 1996.

“… attempt to produce the most aesthetic, and therefore comprehensible, layouts. ” Harel’s Aesthetics “ An Algorithm for Blob Hierarchy Layout", Proc. Working Conf. on Advanced Visual Interfaces (AVI ’ 2000), Palermo, Italy, ACM Press, New York, May 1999, pp

Automobile Alarm-unaesthetic?

Statechart Complexity Metrics (1) “The aim of the experiment was to investigate the relationship between the complexity of UML statechart diagrams and their understandability (one maintainability subcharacteristic). Defining and Validating Metrics for UML Statechart Diagrams Marcela Genero, David Miranda and Mario Piattini, QAOOSE2002

Statechart Complexity Metrics (2)

Automobile Alarm-too many of something?

Statechart Complexity Metrics: economy vs. explicity

How Do We Measure Understandability? (2) Cognitive Dimensions (CDs) Viscosity Visibility Premature commitment Hidden dependencies Role expressiveness Error-proneness Abstraction Secondary notation Closeness of mapping Consistency Diffuseness Hard mental operations Provisionality Progressive evaluation Proposed by Greene in 1989, who has developed it since then with Petre, Blackwell, et al.

Automobile Alarm-CD deficiency?

Patterns to the rescue! resetting setOrNeutral setOrSetting set setting neutral [ settingCondition] [neutralCondition] tm(setTime)[IS_IN(setting)]/setAction [ resettingCondition]/resetAction [neutralCondition]/neutralAction /setup()

Cognitive “Measures” Automatization –Automatized vs. controlled tasks Types of knowledge –Declarative, Conceptual, Procedural Expertise –From novice to expert Learning hierarchy –Associations and chains; Discriminations Concepts; Rules ; Higher order rules Knowledge structuring –Surface features task specific features abstract/hierarchical features Mental workload –single resource pool, multiple-resource

Automatization(1) Please enter OMTracer Command>> go idle OMTracer (0:00:00.200) Display[0] Sent to itself Event tm(200) at ROOT.active OMTracer (0:00:00.200) Display[0] Received from itself Event tm(200) at ROOT.active OMTracer (0:00:00.200) main() Invoked Display[0]->Take Event Timeout OMTracer (0:00:00.200) Display[0] Invoked isDone() OMTracer (0:00:00.200) Display[0]->isDone() Returned OMTracer (0:00:00.200) Display[0] Exited State ROOT.active OMTracer (0:00:00.200) Display[0] Invoked print(n = 1) OMTracer (0:00:00.200) Display[0]->print(n = 1) Returned OMTracer (0:00:00.200) Display[0] Entered State ROOT.active OMTracer (0:00:00.200) Display[0] set tm(200) at ROOT.active OMTracer (0:00:00.200) Display[0]->Take Event Timeout Returned

Automatization(2)

Automatization(3) off filling rinsing evStart/ itsTank->GEN(evTankFill) evFull Statechart of Dishwasher Statechart of Tank

Any questions?

Why engineers won’t use tools

Enter Professor Harel

More than one way to skin a cat!(1) Please enter OMTracer Command>> go idle OMTracer (0:00:00.200) Display[0] Sent to itself Event tm(200) at ROOT.active OMTracer (0:00:00.200) Display[0] Received from itself Event tm(200) at ROOT.active OMTracer (0:00:00.200) main() Invoked Display[0]->Take Event Timeout OMTracer (0:00:00.200) Display[0] Invoked isDone() OMTracer (0:00:00.200) Display[0]->isDone() Returned OMTracer (0:00:00.200) Display[0] Exited State ROOT.active OMTracer (0:00:00.200) Display[0] Invoked print(n = 1) OMTracer (0:00:00.200) Display[0]->print(n = 1) Returned OMTracer (0:00:00.200) Display[0] Entered State ROOT.active OMTracer (0:00:00.200) Display[0] set tm(200) at ROOT.active OMTracer (0:00:00.200) Display[0]->Take Event Timeout Returned

Automatization Shiffrin et al. [9] [10] distinguish between controlled and automatic processes. Whereas “automatized” processes require little or no cognitive resources, “controlled” processes are resource intensive. Abilities are important for mastering repetitive tasks, but not for performing them once mastered. In contrast, abilities continue to be important in the performance of controlled processes.

Level of Detail Diagram Type Application Type

How did the SHADOW get its name????? 2 Theories

Theory I Sikorsky Helicopter Advanced Demonstrator of Operator Workload

Theory II Jimmy Durante’s radio program and nose!

The SHADOW was top top secret!!!! Not members of the U.S. House of Representatives… Not even Senators… –Got to see the SHADOW

The SHADOW was top top secret!!!! But the King got to fly in it!

4 Helicopters were sent to fetch the King from Boston

It was raining, and the King decided that discretion was the better part of valour Coach More Women's handbags

How Do We Measure Understandability? (2) Cognitive Dimensions (CDs) Viscosity: resistance to change Visibility Greene, T. R. G. (1989). Cognitive Dimensions of Notations. In People and Computers V. A. Sutcliffe and L. Macaulay, (Ed.). Cambridge: Cambridge University Press, For non-specialists (no expertise in cognitive science or HFE required) Broad-brush Quick to apply, no lengthy analysis Check list approach: – abstraction, hidden dependencies, premature commitment, secondary notation, viscosity, visibility, closeness of mapping, diffuseness, error-proneness, hard mental operations, progressive evaluation, provisionality, role-expressiveness

Bottom Up Tool Evaluation