1. From Molecules To Landscapes: Rule-based FSPMs in the Language XL Winfried Kurth Brandenburg University of Technology at Cottbus, Chair for Graphics Systems

2. 1.Strengths and weaknesses of traditional approaches in plant modelling 2.Relational Growth Grammars (RGGs) as a generic tool on a formal level 3. The language XL 4. Future perspectives Cottbus, 10. 3. 2008

3. 1. Strengths and weaknesses of traditional approaches in plant modelling Challenges: connection of structure and function in a coherent model framework bridging the gap between different scales

4. bio-/geosphere region ecosystem population individual organ tissue cell organell / genome molecule GEOINFORMATICS BIOINFORMATICS / SYSTEMS BIOLOGY Cottbus, 10. 3. 2008

5. bio-/geosphere region ecosystem population individual organ tissue cell organell / genome molecule GEOINFORMATICS BIOINFORMATICS / SYSTEMS BIOLOGY Cottbus, 10. 3. 2008 ECOLOGICAL INFORMATICS  Transfer of the Systems Biology viewpoint to higher scale levels

6. Tools for modelling and simulation: (a) classical PBM (process-based models) -pools of substrates in compartments -fluxes between pools Example: STELLA flowcharts

7. mathematical formalisms: - qualitative: Petri nets - quantitative: systems of differential equations tools: - numerics software - graphical modelling environments (e.g., STELLA) Cottbus, 10. 3. 2008

8. PBM – drawbacks: spatial structure often poorly represented

9. Cottbus, 10. 3. 2008 PBM – drawbacks: spatial structure often poorly represented no representation of the objects with which the user really works

10. PBM – drawbacks: spatial structure often poorly represented no representation of the objects with which the user really works parameters partially difficult to measure and to interpret Cottbus, 10. 3. 2008

11. (b) structural models -entities: organs / modules (biologically senseful and visualizable entities) -effects of interaction occur emergently -parameters: relatively few, measurable barley model (Buck-Sorlin et al. 2005) Cottbus, 10. 3. 2008

12. (b) structural models -entities: organs / modules (biologically senseful and visualizable entities) -effects of interaction occur emergently -parameters: relatively few, measurable Cottbus, 10. 3. 2008 most important approach from computer science for this type of models (until recently): Lindenmayer systems (L-systems)

13. Examples of L-system based plant models: Prusinkiewicz & Lindenmayer 1990 K. 1998, 1999

14. Applications: virtual plant structures as a basis for simulations, e.g., light interception in a tree stand (Knyazikhin, Ibrom, K. 1997) water flow in a tree (Früh & K. 1999)

15. structure has impact on function – example of xylem sap flow (Früh & K. 1999) spruce (L-system model) spruce (3D measurement) Thuja (3D measurement) Cottbus, 10. 3. 2008

16. structural models – drawbacks: no (or very sparse) taking into acount of the functional aspects of organisms no metabolism, no linkage with lower scale levels Cottbus, 10. 3. 2008

17. structural models – drawbacks: no (or very sparse) taking into acount of the functional aspects of organisms no metabolism, no linkage with lower scale levels Cottbus, 10. 3. 2008 combination of model types

18. (c) Functional-structural plant models, FSPM Idea: distribution of the processes to the modules

19. (c) Functional-structural plant models, FSPM Idea: distribution of the processes to the modules Tool: object-oriented programming example of ALMIS (Eschenbach 2000)

20. (c) Functional-structural plant models, FSPM Idea: distribution of the processes to the modules Tool: object-oriented programming example of ALMIS (Eschenbach 2000)

21. FSPM example LIGNUM (Perttunen et al. 1996, 1998, Dzierzon & K. 2002; Sievänen et al. 2006)

22. Drawbacks of ad hoc FSPMs from the last years:  isolated solutions, often strongly specialized Cottbus, 10. 3. 2008

23. Drawbacks of ad hoc FSPMs from the last years:  isolated solutions, often strongly specialized  large, complex source code, containing technical details mixed with fundamental features of the model Cottbus, 10. 3. 2008

24. Drawbacks of ad hoc FSPMs from the last years:  isolated solutions, often strongly specialized  large, complex source code, containing technical details mixed with fundamental features of the model  low compatibility of the models with each other Cottbus, 10. 3. 2008

25. Drawbacks of ad hoc FSPMs from the last years:  isolated solutions, often strongly specialized  large, complex source code, containing technical details mixed with fundamental features of the model  low compatibility of the models with each other  complexity of the tool (for the user) has to be reduced a further challenge: Cottbus, 10. 3. 2008

26. traditional, commonly-used programming languages obviously not optimal for the purpose Cottbus, 10. 3. 2008

27. formal basis for tools in ecological informatics?  grammars Cottbus, 10. 3. 2008

28. Eric Mjolsness, Univ. of California 2006: "Future multiscale models must be able to integrate all the major different types of dynamical systems models,... These goals are achieved by the modelling framework of... grammars." Cottbus, 10. 3. 2008 formal basis for tools in ecological informatics?  grammars

29. 2. Relational Growth Grammars (RGG) as a generic tool on a formal level point from where to start: L systems (parallel string rewriting) Cottbus, 10. 3. 2008

30. Limitations of L-systems: in L-systems, only 2 possible relations between objects: "direct successor" und "branching" Cottbus, 10. 3. 2008

31. Limitations of L-systems: in L-systems, only 2 possible relations between objects: "direct successor" und "branching" multiscaled models are not supported Cottbus, 10. 3. 2008

32. Limitations of L-systems: in L-systems, only 2 possible relations between objects: "direct successor" und "branching" multiscaled models are not supported object-oriented programming is not supported (only symbols, resp. "modules" = parameterized symbols, no objects, no classes) Cottbus, 10. 3. 2008

33. Limitations of L-systems: in L-systems, only 2 possible relations between objects: "direct successor" und "branching" multiscaled models are not supported object-oriented programming is not supported (only symbols, resp. "modules" = parameterized symbols, no objects, no classes) structures must be serialized to strings Cottbus, 10. 3. 2008

34.  transgression to graph grammars Limitations of L-systems: in L-systems, only 2 possible relations between objects: "direct successor" und "branching" multiscaled models are not supported object-oriented programming is not supported (only symbols, resp. "modules" = parameterized symbols, no objects, no classes) structures must be serialized to strings

35. "Relational Growth Grammars" (RGG) as generic tool on a formal level = graph grammars with parallel application Cottbus, 10. 3. 2008

36. "Relational Growth Grammars" (RGG) as generic tool on a formal level = graph grammars with parallel application graph model: -with node attributes and types (type hierarchy for inheritance) -with edge labels (finitely many) -no multiple edges with the same label Cottbus, 10. 3. 2008

37. Nodes:  correspond to the symbols in L-systems Cottbus, 10. 3. 2008

38. Nodes:  correspond to the symbols in L-systems  simultaneously objects sensu OOP Cottbus, 10. 3. 2008

39. Nodes:  correspond to the symbols in L-systems  simultaneously objects sensu OOP  e.g., plant organs, geometric transformations Cottbus, 10. 3. 2008

40. Edges: their labels can represent different sorts of relations: is successor of contains bears as a lateral shoot reacts with encodes (genetically) is mating with (...)  also possible: representation of multiscaled structures Cottbus, 10. 3. 2008

41. multiscaled structures (different geometric levels of resolution in one model) relation of refinement (AMAPmod software description, CIRAD Montpellier, 1998)

42. multiscaled structures (different geometric levels of resolution in one model) relation of refinement (AMAPmod software description, CIRAD Montpellier, 1998) in computer graphics: "Level of Detail"

43. ● embedding model: Single-Pushout approach from algebraic graph grammar theory, extended by so-called connection transformations ● right-hand sides of rules are dynamically generated RGG replacement mechanism left-hand side of rule  right-hand side of rule Cottbus, 10. 3. 2008

44. an RGG rule and its application in graphical form: rule: application: Cottbus, 10. 3. 2008

45. an RGG rule and its application in graphical form: rule: application: rule in text form: i -b-> j -a-> k -a-> i = => j Cottbus, 10. 3. 2008

46. original graph: ABC Cottbus, 10. 3. 2008 implicit use of "connection edges" for the desired embedding: a:A ==>> a C (SPO rule) B ==> D E (rules of L-system type) C ==> A

47. ABC DEA connection edges (auxiliary edges) Cottbus, 10. 3. 2008 implicit use of "connection edges" for the desired embedding: a:A ==>> a C (SPO rule) B ==> D E (rules of L-system type) C ==> A

48. ABC DEA a: Cottbus, 10. 3. 2008 implicit use of "connection edges" for the desired embedding: a:A ==>> a C (SPO rule) B ==> D E (rules of L-system type) C ==> A

49. implicit use of "connection edges" for the desired embedding: a:A ==>> a C (SPO rule) B ==> D E (rules of L-system type) C ==> A AADE a: C Cottbus, 10. 3. 2008

50. Advantages:  L-systems as a special case Strings correspond to special graphs Cottbus, 10. 3. 2008

51. example: crossing over Two homologous chromosome strands, encoded as strings with "successor" relation RGG rule: application: alignment of homologous alleles  pure SPO rules as a special case Cottbus, 10. 3. 2008

52.  further special case: rules which change only parameters Cottbus, 10. 3. 2008 Example from barley model: Cell [lue1:LUE1] ::> lue1[concentration] :+= 0.5 * par.DELTA_T;

53. Realization of RGGs in a programming language: XL (eXtended L-system language) (Kniemeyer 2007) extension of Java nodes are Java objects additional constructions Cottbus, 10. 3. 2008 3. The language XL

54. Queries in the generated graphs possibility to link structure and function Cottbus, 10. 3. 2008

55. Queries in the generated graphs possibility to link structure and function Example: look for all leaves which are successor of node c and sum up their areas Cottbus, 10. 3. 2008

56. Queries in the generated graphs possibility to link structure and function Example: look for all leaves which are successor of node c and sum up their areas transitive closure aggregation operator Cottbus, 10. 3. 2008

57. Queries in the generated graphs possibility to link structure and function Example: look for all leaves which are successor of node c and sum up their areas transitive closure aggregation operator result can be transferred to procedural calculation (e.g., PBM) Cottbus, 10. 3. 2008

58. Query in a plant / herbivore model: p:Plant, (* a:Animal, (distance(a,p) < p[radius]) *) Cottbus, 10. 3. 2008

59. Query in a plant / herbivore model: p:Plant, (* a:Animal, (distance(a,p) < p[radius]) *) looks for all animals in the radius of p Cottbus, 10. 3. 2008

60. dynamics of networks (e.g., neighbourship relation in a tree stand)

61. Cottbus, 10. 3. 2008 dynamics of networks (e.g., neighbourship relation in a tree stand)

62. Cottbus, 10. 3. 2008 dynamics of networks (e.g., neighbourship relation in a tree stand)

63. Cottbus, 10. 3. 2008 dynamics of networks (e.g., neighbourship relation in a tree stand)

64. dynamics of networks (e.g., neighbourship relation in a tree stand) generally: dynamical systems with a dynamical structure Cottbus, 10. 3. 2008

65. Interactive 3D platform GroIMP (Growth-grammar related Interactive Modelling Platform) with XL compiler Cottbus, 10. 3. 2008

66. Interactive 3D platform GroIMP (Growth-grammar related Interactive Modelling Platform) with XL compiler GroIMP is an Open Source project ( http://www.grogra.de ) Cottbus, 10. 3. 2008

67. XL compiler (time series of) structure(s) (attributed graphs) model specification Cottbus, 10. 3. 2008

68. XL compiler description parser (time series of) structure(s) (attributed graphs) model specification descriptions of real organisms Cottbus, 10. 3. 2008

69. XL compiler description parser (time series of) structure(s) (attributed graphs) renderer visual representation model specification descriptions of real organisms Cottbus, 10. 3. 2008

70. XL compiler description parser (time series of) structure(s) (attributed graphs) XL consolerenderer visual representation statistical analysis software request for analysis (query) model specification descriptions of real organisms Cottbus, 10. 3. 2008

71. XL compiler description parser (time series of) structure(s) (attributed graphs) XL consolerenderer interfaces special simulation software visual representation statistical analysis software request for analysis (query) model specification descriptions of real organisms Cottbus, 10. 3. 2008

72. GroIMP provides complete XL libraries e.g., node classes for the emulation of Xfrog graphs (cf. Deussen, Lintermann) Diploma thesis Henke 2006, Henke et al. (submitted)

73. Cottbus, 10. 3. 2008 GroIMP provides complete XL libraries e.g., node classes for the emulation of Xfrog graphs (cf. Deussen, Lintermann) Diploma thesis Henke 2006, Henke et al. (submitted)

74. GroIMP provides complete XL libraries e.g., node classes for the emulation of Xfrog graphs (cf. Deussen, Lintermann) Diploma thesis Henke 2006, Henke et al. (submitted) Cottbus, 10. 3. 2008

75. (Henke 2006) Cottbus, 10. 3. 2008 GroIMP provides complete XL libraries extension of the Xfrog node classes by an „arrange“ node

76. (Henke 2006) Cottbus, 10. 3. 2008 GroIMP provides complete XL libraries extension of the Xfrog node classes by an „arrange“ node Input: „fields“ (given by image files) for spatial parameters heights densities tree parameters

77. GroIMP provides complete XL libraries extension of the Xfrog node classes by an „arrange“ node Input: „fields“ (given by image files) for spatial parameters (Henke 2006) heights densities tree parameters Cottbus, 10. 3. 2008

78. 4. Future perspectives  Improvement of rendering / radiation simulation currently under construction: - Metropolis raytracing - GPU-based raytracer - Radiosity calculation Cottbus, 10. 3. 2008

79. (Veach 1998) Weakness of bidirectional raytracing in case of indirect illumination: Improvement: Metropolis algorithm (mutation of an already detected ray path) Cottbus, 10. 3. 2008

80. e.g., molecule models (GroIMP filter for pdb files)  Extension of the possibilities for 3D modelling Cottbus, 10. 3. 2008

81.  exchange platform "OpenAlea" (Pradal 2007) modular platform to improve the interoperability of plant models and analysis tools Making XL available for this international project Cottbus, 10. 3. 2008

82.  3D data acquisition from real organisms Cottbus, 10. 3. 2008

83.  3D data acquisition from real organisms and linked to that:  solution of the inference problem for RGGs Aim: automatize the pipeline real object  3D model  RGG  abstract model  visualization of the abstract model Cottbus, 10. 3. 2008

84. ongoing project: development of a low-cost system for the acquisition of 3D data with structured light (Jiang & Bunke 1997) Cottbus, 10. 3. 2008

85. aerial photo + elevation model (ArcGIS)

86. aerial photo + elevation model (ArcGIS) view into simulated stand (Knauft 2001)

87. shadow calculation at forest soil level at different times of the day (Knauft 2001)

88. "ViWa" (Virtueller Wald = virtual forest) – landscape in the Solling region (Germany) (Knauft & K. 2001) elevation model + forest inventory data + structural model of stand + structural model of tree + rendering: Cottbus, 10. 3. 2008

89. Modelling of park landscapes (Rogge & Moschner 2007, for Branitzer Park Foundation, Cottbus) Cottbus, 10. 3. 2008

90. Thank you for your attention! Cottbus, 10. 3. 2008