1. ADBIS 2007; Varna, Bulgaria; 03.10.2007 Towards Self-Optimization of Message Transformation Processes Matthias Böhm 1,2,3 *, Dirk Habich 2, Uwe Wloka 3, Jürgen Bittner 1, and Wolfgang Lehner 2 1 SQL GmbH Dresden, Germany 2 Dresden University of Technology, Database Technology Group 3 University of Applied Sciences Dresden, Database Group

2. 2 / 28 Outline Introduction Integration Platform TransConnect ® Process Optimization Techniques Summary and Conclusion

3. 3 / 28 Outline Introduction Integration Platform TransConnect ® Process Optimization Techniques Summary and Conclusion

4. 4 / 28 Introduction starting point –integration of heterogenous information systems –horizontal service integration by message-based communication using the Message Transformation Model (MTM) motivation / problem description –suboptimal modeled processes –dynamic workload characteristics –total costs of ownership contribution towards self-optimization –first rule-based optimization techniques –first workload-based optimization techniques –prototypical implementation within TransConnect

5. 5 / 28 Introduction Message Transformation Model (MTM)

6. 6 / 28 Introduction Message Tansformation Model (MTM) –Message Model –Process Model (reconsidered) Interaction-oriented activ. Control-flow-oriented activ. Data-flow-oriented activ. Base model "Directed Graph" Hierarchical message structure

7. 7 / 28 Introduction Message Tansformation Model (MTM) –Example Process

8. 8 / 28 Outline Introduction Integration Platform TransConnect ® Process Optimization Techniques Summary and Conclusion

9. 9 / 28 Integration Platform TransConnect TransConnect –message based application integration –inbound adapters –outbound adapters –process engine TransConnect 1.3.6 overall architecture

10. 10 / 28 Integration Platform TransConnect TransConnect 1.3.6 Server architecture

11. 11 / 28 Integration Platform TransConnect Component ProcessParser

12. 12 / 28 Integration Platform TransConnect External Layer: WSBPEL 2.0 process

13. 13 / 28 Integration Platform TransConnect Conceptual Layer: MTM process type

14. 14 / 28 Integration Platform TransConnect Internal Layer: JAVA process plan public class es_process1 extends ProcessPlan { private InternalMessage msg1 = null; private InternalMessage msg2 = null; @Override protected InternalMessage executeNode(InternalMessage input) throws MTMException { try { Invoke node1 = new Invoke("sap_mq","DEQUEUE",AService.OTYPE_RECEIVE); node1.setIDs(getPTID(), getPID(), getNID()); msg1 = node1.execute( msg3 ); } catch( MTMSignalException mse ) { /*signal handling*/ } /*...*/ }

15. 15 / 28 Integration Platform TransConnect Component SystemMonitor –interval monitoring / continuous monitoring –determination of suboptimal process plans –recompilation of process plans –Self-Optimization according to IBM MAPE concept (Monitor, Analyse, Plan, Execute) adaptive optimization strategies

16. 16 / 28 Integration Platform TransConnect Component SystemMonitor –Inbound monitor events: performance measurement -- average process type execution time -- (not normalized!) SELECT AVG(EndTime - StartTime) FROM ProcessingPerformance WHERE NID = -1 AND -- node type process PID IN ( SELECT PID FROM Process WHERE PTID = (SELECT PTID FROM ProcessType WHERE Name=‘es_process1‘))

17. 17 / 28 Outline Introduction Integration Platform TransConnect ® Process Optimization Techniques Summary and Conclusion

18. 18 / 28 Process Optimization Techniques influencing-factors –optimization aim: throughput / execution time –execution knowledge: statistics / ad-hoc –optimization techniques: rule-based / workload-based technique classification –Rule-based process optimization Control flow optimization Data flow optimization –Workload-based process optimization Message indexing Control flow optimization Data flow optimization

19. 19 / 28 Process Optimization Techniques Rule-based process optimization –Control flow optimization Redundant control flow elimination Unreachable subgraph elimination Preventing local subprocess invocation

20. 20 / 28 Process Optimization Techniques Rule-based process optimization –Data flow optimization Double Variable Assignments Unnecessary Variable Assignments Unnecessary Variable Declarations Two sibling Tanslation operators Unnecessary Switch -paths Two sibling validations Basically these techniques are adopted from imperative programming language compilers

21. 21 / 28 Process Optimization Techniques Workload-based process optimization –Message indexing

22. 22 / 28 Process Optimization Techniques Workload-based process optimization –Message indexing –Control flow optimization query scrambling techniques (external systems delay, network delay elimination, and disk I/O delay) parallel flow management

23. 23 / 28 Process Optimization Techniques Workload-based process optimization –Data flow optimization Switch operator optimization

24. 24 / 28 Process Optimization Techniques Evaluation Experiment - "Complex Integration Process" rule-based and workload-based process plan rewriting

25. 25 / 28 Process Optimization Techniques Evaluation Experiment - "Complex Integration Process" –average inbound message size: 7KB

26. 26 / 28 Outline Introduction Integration Platform TransConnect ® Process Optimization Techniques Summary and Conclusion

27. 27 / 28 Summary and Conclusion Summary –optimization techniques were illustrated –implementation and evaluation prove the high optimization potential –lots of further research items along Conclusion –research of optimization techniques will be displaced from the grounding systems to the integration process Future work –DIPBench (Data-Intensive Integration Process Benchmark) –GCIP (Model-Driven Generation and Optimization of Complex Integration Processes) –MIX (Message Indexing for Document-Oriented Integration Processes) –Adaptive Enterprise Integration Platform

28. ADBIS 2007; Varna, Bulgaria; 03.10.2007 Towards Self-Optimization of Message Transformation Processes Matthias Böhm 1,2,3 *, Dirk Habich 2, Uwe Wloka 3, Jürgen Bittner 1, and Wolfgang Lehner 2 1 SQL GmbH Dresden, Germany 2 Dresden University of Technology, Database Technology Group 3 University of Applied Sciences Dresden, Database Group