1. 25 juin 2010 Interactive Genetic Algorithms for Creative Enhancement in UI design Dimitri Masson Alexandre Demeure Gaelle Calvary 1

2. 2 Problem statement User Interface (UI) design is a creative process General question: How to support designers’ exploration of the design space? Instant messenger … … Design space of possible UIs

3. My research proposal Presentation of generated design solutions for the application under study Motivation –Compliance with designers’ practice: samples browsing –Sources of inspiration 3

4. Outline State of the art Approach Prototype Early evaluation Conclusion and future work 4

5. State of the art Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process –Design process –Tools –Model Based UI 5

6. State of the art Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process –Design process –Tools –Model Based UI 6

7. State of the art Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process –Design process –Tools –Model Based UI 7

8. State of the art: Creative process A Darwinian process 8 Variation & recombination Selective retention

9. State of the art: Creative process A Darwinian process 9 Variation & recombination Selective retention

10. State of the art: Creative process A Darwinian process 10 Variation & recombination Selective retention

11. State of the art Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process and places of creativity –Design process –Tools –Model Based UI 11

12. State of the art: IGA Interactive Genetic Algorithms: Suitable for large problem space for which –no analytical method can be applied –but candidate solutions can be evaluated 12 Variation & recombination Selective retention Performed by the system Performed by the designer

13. 13 State of the art: IGA Init a population of candidate solutions Evaluate candidate solutions Apply mutation and crossbreedings Select promising candidate solutions Design space 0.1 0.2 0.22

14. State of the art Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process –Design process –Tools –Model Based UI 14

15. Three contributions: [Monmarché 1999 & Oliver 2002] [Quiroz 2007] [Plessi 2008] –Input: one UI sample –Dimensions of the design space: fonts, colors, layout –Output: definitive UIs for the end-user not for the designer 15 State of the art: IGA & HCI

16. The exploration of the design space is limited Evolution is limited to predefined elements The UI remains basically the same Absence of high level description of the UI. 16

17. State of the art: IGA & HCI Absence of high level description of the UI. As a result: Evolution is limited to predefined elements The UI remains basically the same –No widget substitution 17 dd/mm/yy

18. State of the art: IGA & HCI Absence of high level description of the UI. As a result: Evolution is limited to predefined elements The UI remains basically the same –No widget substitution –No UI enrichment 18 dd/mm/yy Select a date:

19. State of the art: coverage Creativity –Creative person –Creative product –Creative process Creative evolutionary systems –Interactive Genetic Algorithms (IGA) –IGA & HCI UI design process –Design process –Tools –Model Based UI 19

20. State of the art: Model based UI High level description of the UI: concept & task model 20 Transformation 1 Transformation 2 Transformation N Transformations …

21. Outline State of the art Approach Prototype Early evaluation Conclusion and future works 21

22. My approach Three key points: #1 Combination of IGA and Model Based UI at C&T. #2 Evolution of transformations from C&T  UI #3 Support from a knowledge database 22 Concept and task (C&T)Transformations (C&T  UI) IGA Evolve Transformations (C&T  UI) uses

23. 23 My approach: #2 transformation Transformation = one tree –Edges: Select C&T elements –Nodes: Translate C&T elements into UI elements Root of transformation Select choices Translate into radiobuttons Select abstract tasks Translate into windows Select descendants abstract tasks Translate into frames with a red background Root of the task

24. 24 My approach: #2 Genetic operations Mutation –Edge: changing selector –Node: Changing translation –Add/remove nodes Select choices Translate into radiobuttons Select abstract tasks Translate into windows Select descendants abstract tasks Translate into frames Translate into comboboxes Select children abstract tasks 24

25. 25 My approach: #2 Genetic operations Crossbreeding

26. My approach: #3 Knowledge database Widget representations for each task Attributes for each widget and manipulation function 26 Choice of date Entry text Format Background color Foreground color Calendar Width Height 23/11/08

27. Outline State of the art Approach Prototype Early evaluation Conclusion and future works 27

28. VIDEO 28

29. Prototype: Implementation TCL COMET: a widget toolkit [Demeure 2008] –Task level interactors supporting multiple presentations –Database storing design knowledge 29

30. Outline State of the art Approach Prototype Early evaluation Conclusion and future works 30

31. Early evaluation The purpose is to –assess the effectiveness of the approach by collecting as many ideas and practices as possible –identify new areas for improvement The purpose is not to –assess the creativity improvement –evaluate the usability of the prototype 31

32. Early evaluation: Protocol Protocol defined with N. Mandran, sociologist Face-to-face semi-structured interviews –Questionnaire about design activity Personal activity, Design practices, tools, etc. –Sketch design of an Instant Messenger e.g.: Sketches done by E1, Appendix C –Manipulation of Magellan Thinking aloud technique 11 participants: Designers, HCI experts, Students 2h/interview + 4h/post-analysis (> 2 weeks) 32

33. Early evaluation: Results I conducted an experiment to test 3 hypothesis 1.Designers find benefits from the IGA approach. –“Not perfect but interesting”. –The process enable to reject bad UIs. 2.Designers find the evolution of widgets valuable. –Presentation of elements in the actual UI. 3.Designers find the UI enrichment valuable. –Few even notice the enrichment process. As well as eliciting unforeseen needs about such approach. –Need for layout support. –Need for User Interaction. 33

34. Outline State of the art Approach Prototype Early evaluation Conclusion and future work 34

35. Conclusion: My contributions Increment on previous approaches: IGA & MBUI –Transformation as individual representation –Widget substitution –UI enrichment Early inquiry of designers’ needs –Previous work focus more on the IGA than on the HCI Magellan [Masson 2010]: ACM EICS, Berlin, June 20-23 35

36. Conclusion: Difficulties Large scope of the state of the art Automatic generation of UI is complex Conducting experiment is complex –Very long to set up –Require experience IGA requires a good modeling of individuals and fine tuning of the process 36

37. Conclusion: Future work Direct edition of the samples presented –Integration of user sketches Support several level of fidelity Partial UI evaluation and evolution New evaluation 37

38. Questions 38

39. Research direction Presentation of generated design options. 39

40. A faire nb de ligne de code. 40

41. Early evaluation: findings Purpose 1: Assess the effectiveness of the approach by collecting as many ideas and practices as possible H1: Designers find benefits from the IGA approach –“Not perfect but interesting”. –The process enable to reject bad UIs H2 H3 –identify new areas for improvement 41