Components in Product Lines - The Next Steps Rob van Ommering Philips Research EuroMicro 2005, Porto, Portugal, September 1 st, 2005

2 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Introducing my domain… MB 64 kB 1 kB 1965

3 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 (1) Complexity

4 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Image Sound 100 Hz quality Dolby AC3 Data Processing User Interface Txt EPG menus animation 3D Connectivity P Storage Device TiVo VCR DVD TVCR HD Broadcasting Standard DTV Region Eu US AP Video Output Device PTV FTV LCTV Price UTV MTV (2) Diversity

5 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 (3) Lead Time Was: Yearly cycle of product introduction –Christmas –World championship Is: Decreasing to 6 (or even 3) months –Otherwise loose shelf space in shop

6 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Summary… Software Grows Exponentially (Moore’s Law) Software Grows Exponentially (Moore’s Law) Market demands... Shorter lead time… More product variation... Shorter lead time… More product variation... Need more people... Need more time... Need more people... Need more time...

7 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents of my talk Introduction Product families, populations, lines Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

8 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Philips TV product family cineoflatprojection wide screen designer

9 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Related product families VCR Audio STB DVD

10 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Convergence TV VCR DVD HD Audio STB TVCR TV-DVD Tivo Home Theater Digital TV = = = = =

11 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Convergence yesterday :-) GPS + GSM GSM + DigCam PDA + GPS CD-RW, DVD, Card, TV PDA + GSM + DigCam

12 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Convergence today :-) Includes PDA and FM radio, no GPS yet… USB powered! GPS Phone MP3 Camera Power supplies

13 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 A product population is: - a set of products with many commonalities, - but also with many differences, - developed by different sub-organizations, - each with its own time-line / lifecycle. Single Product Product Family Product Population Unrelated Products Decomposition Dedicated components Composition COTS

14 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 A product line is:

15 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents Introduction Product families and populations Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

16 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Module A Koala module is: - a non-reusable unit of code - with variation points Mindset: a Koala module can (at least in in principle): - be written in any language (C, Koala, HTML, makefile, …) - have any form (source, object, library, …)

17 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Variation Points Koala has a two-level parameterization mechanism: Level I Level II Name: r Type: ISomething interface ISomething { int p; int q = 7; int f(x); int g(x) = 1 + f(x); } This is Koala’s Interface Definition Language (IDL)

18 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Traditionally… We draw: interface IDiversity { int p; int q = 7; } interface IFunctionality { void f(void); int g(int x); } But this is only a convention (not built into Koala)

19 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Modules also provide stuff… Same two-level mechanism: Name: p Type: ISomething Level I Level II interface ISomething { int p; // defined in module int q = 7; int f(x); // defined in module int g(x) = 1 + f(x); }

20 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Again traditionally… We draw: interface IDiversity { int p; int q = 7; } interface IFunctionality { void f(void); int g(int x); } But again, this is only a convention (not built into Koala)

21 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Building Systems We can now build systems: m provides m requires m m’ …in both dimensions… m’’

22 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Component A Koala component is: - a reusable … - … unit of encapsulation … - … still with variation points - providing stuff m m’m’’ C Note how: - modules are internal - some interfaces are internal - some interfaces are external

23 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Component Definition m C component C { provides IXxx p; requires IYyy r; contains module m; connects p = m; m = r; } A component is described in a Component Definition Language (CDL): p r

24 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Compound Component C C2C2 C1C1 C3C3 Components can instantiate other components:

25 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents Introduction Product families and populations Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

26 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Setting Diversity C1 42 C1 7 x P1P1 P2P2 P3P3 Creating different products… …by setting diversity parameters differently

27 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Selecting Components Creating different products… …by selecting different subcomponents P1P1 C1 C2 P2P2 C1 C3 P3P3 C1 C2 C3 x

28 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Parameterization C1 C2 C3 Partial evaluation is used to create resource efficient configurations. Spreadsheet of diversity calculations

29 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Reflection C1 A component can observe whether interfaces are actually connected. This allows components to adapt themselves to their environment automatically.

30 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Reflection A component can observe whether interfaces are actually connected. Special notation for this common design pattern

31 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Example: my old digital camera… Dedicated “Dedicated”

32 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 The one I wanted to buy…

33 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 The one I bought… All general purpose components!

34 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 One snag though… Can do 110 Volts and 220 Volts But be careful!

35 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Self-configuring components Components specify how many resources they require This can be summed and provided to the component that delivers the resources at the product level C1 C3 C1 +

36 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents Introduction Product families and populations Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

37 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Multi-threading C1 Step 1: use message pumps created on virtual pump engines required through a diversity interface Step 2: bind these to pump engines (a real dispatcher loop) Problem: many (>100) activities but few (<10) threads C2 C3 CFireBrigade Same thread, No synchronisation required Different thread, Synchronisation required

38 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Threading Analysis a a b b b c * * Attribute interfaces with a symbolic thread label

39 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Threading calculus a a a a a b c d * * * * * a * a a * a * X X X X X O

40 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Unification p a a p p b b p q b c q Thread labels are unified Prepared for call on different threads, but called on the same thread q must be same as p C is a new thread, and cannot be equal to q

41 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005

42 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents Introduction Product families and populations Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

43 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Configuration Management Distinguish between: versions temporary variants permanent variants of components. We use our CM system for: versions temporary variants But we use the component model for: permanent variants C2C2 Internal diversity C2C2 C3C3 Structural diversity

44 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Roadmapping

45 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Branching in practice… There are a large number (>10) of long-living (> 1y) branches. Advantage: Safeguard products from ‘improvements’ in other products. Disadvantage: Multiple maintenance.

46 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 How much effort ‘leaks away’? Measure #LoC change between releases. Sum this over releases of branches. Sum this over releases of main line. Divide this (simplified formula):

47 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Show over time in a single picture: code size == LoC created  LoC modified on main line  LoC modified in branches

48 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 The Turn-Over Factor Defined as sum of created and modified LoC (read: effort) divided by resulting LoC (read code size). Turn-over factor

49 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Turn-over factor for 11 subsystems

50 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 First Impression Effort in branches is 10-20%. Effort in rewrites is 50%! A subsystem is either stable or non-stable. Subsystems are stable (only) when no developers have been assigned. Non-stable subsystems are completely rewritten in 3-4 years. To do: Zoom in Look at other software (in Philips, or Open Source)

51 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Archive Loader File Filter Line Filter Previous Mapper Sorted Diff Outline Viewer Detail Viewer New Tool Set Archive Loader File Filter Line Filter Previous Mapper Sorted Diff Outline Viewer Detail Viewer Zips Cache Provides more reliable results. And is much faster (658 Linux releases in 1 hour)

52 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Refined Measurement deleted created modified clones

53 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Comparing Various Subsystems

54 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Some Open-Source Packages Apache Linux Mono Subversion

55 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Zooming in…

56 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 New Impression Many people confirm rewrite every 3-4 year. More difficult to measure in growing systems. Open Source also shows rewrites on main line. Stable subsystems are dead code! Spin-off: Evolution browser helps to understand architecture

57 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Contents Introduction Product families and populations Koala 101 Configuration and reflection Calculating system properties Evolution and branching Current and future challenges

58 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Maintaining a Software Product Line… Architecture Products Subsystems Domain Engineering Application Engineering

59 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 … Is Difficult… Architecture Fewer Products => more inclined to develop in branches More components => more overhead

60 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Conway’s Law -1 The structure of the organization should mirror the architecture of the software. Bangalore Singapore Eindhoven Brugge Wien Knoxville Sunnyvale Briarcliff Hamburg Southampton Software development shifts east…

61 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Another Shift: CE  PS  3 rd party MBytes OS Applications A/V platform OS Applications A/V platform Middleware It is no longer efficient to write all the software yourself…

62 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Kaola Can Aspect Orientation help to (at least) reduce the code? Does AO work for more than logging and tracing? INXS Source: Pierre van de Laar

63 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Can I check more things statically? Multi-threading checking is a success, but transfer to our product division is cumbersome: Gain expected but cost is high Can I derive models from source code? ‘Our’ Robocop component model, in some sense a successor of Koala, defines a component as a set of models

64 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Can I do more things dynamically? No problem w.r.t. binding technology: Koala is prepared for that (need another back-end) Were already expecting to go this way in 2000! But you also give up certain things: Performance Memory use Source code browsing Source code analysis … How can we combine the good things?

65 © 2005 Philips Research, Rob van Ommering, EuroMicro 2005, Porto, Portugal, Sep 1 st, 2005 Thank You! Limited availability…