1. Software Engineering for Business Information Systems (sebis) Department of Informatics Technische Universität München, Germany wwwmatthes.in.tum.de Data-Parallel Transcoding for the 3D-Internet Master‘s Thesis Final Presentation Al-waleed Shihadeh, 10 th November, 2014, Garching

2. Agenda  Business Vision  Problems  Thesis Contribution  Implementations  System Evaluation  Outlook © sebis Al-waleed Shihadeh 2014 2

3. Business Vision © sebis Al-waleed Shihadeh 2014 3 Use the Digital Mockup (DMU) as key entry point and reference to support communication, collaboration and data exchange Develop A web-based collaborative engineering environment that :  Integrates different engineering domains.  Allows engineers to have different views for the same 3D model

4. Problems 3D models are very large  more than 1GB  Extract shapes into separate files Extract shapes into separate files  Loading the 3D model shapes in background.  Create web resources for each of the shapes in the 3D model. Browsers issues  Transcode the extracted shapes to binary format. Transcode the extracted shapes to binary format  Compress the size of the shapes.  Enhance the performance of the 3D viewer. Problem  Transcoding large 3D models takes very long time  time consuming task  Ariane5, 84 MB, 879 shapes, more than 40 hours. © sebis Al-waleed Shihadeh 2014 4

5. Thesis Contribution  Data-parallel transcoding approach for X3D data  Distribute the transcoding process.  Decrease transcoding time  Hadoop framework and MapReduce programming model  Evaluation and performance analysis of the proposed approach  Local evaluation : Airbus machines  Remote evaluation : Amazon Elastic Compute Cloud (EC2) © sebis Al-waleed Shihadeh 2014 5

6. Implementations © sebis Al-waleed Shihadeh 2014 6

7. Phase 1 : Pre-parsing © sebis Al-waleed Shihadeh 2014 7 Objectives :  Remove unwanted nodes.  Replace USE nodes in the X3D file.  Create product breakdown tree.

8. Phase 2 : Partitioning © sebis Al-waleed Shihadeh 2014 8 Objectives  First Extract All Shapes:  Extract All parts.  Extract The root part. XQuery and BaseX

9. Phase 2 : MapReduce © sebis Al-waleed Shihadeh 2014 9 Tasks  Setup the environment  Install required libraries: Java, openssh, pydoop, etc..  Install and setup Hadoop cluster. Cluster TypeNodesMemory (GB)OSCPU Single node cluster18Ubuntu- 64 bit 2.93 GHz Dual node cluster28 and 2Ubuntu- 64 bit 2.93 GHz, 2.5 GHz Multiple node clusters48Ubuntu- 64 bit8 vCPUs, 2.5 GHz  Design and implement MapReduce jobs  Using Custom JAR technology.  Using Hadoop streaming technology.

10. Phase 4 : Deployment © sebis Al-waleed Shihadeh 2014 10 At this stage…  Each of the phases is performed manually.  MapReduce job results : transcoded files & shapes web resources. There is a need for a web application that has the following features:  Allows end users to transcode their 3D models automatically  Create and deploy web pages for viewing the transcoded models  Enable users to specify the transcoding commands  Provide basic functionality for the implemented 3D viewer  3D interaction features such as rotating the 3D model, zoom in and zoom out, hide and show the parts of the 3D model, etc.

11. Phase 4 : Deployment © sebis Al-waleed Shihadeh 2014 11

12. Evaluation © sebis Al-waleed Shihadeh 2014 12  Local Evaluation  The goal is to select the approach with the best performance for further evaluation  Performed on the local machines at AirBus  Using the Hadoop single and dual node clusters  EC2 Evaluation  Evaluate only the best approach  Evaluate the effects of both environment and system parameters  System parameters (Shapes Number, Files number, Split size, shapes type)  Environment parameters (CPU power, Memory size, Node number)

13. Local Evaluation © sebis Al-waleed Shihadeh 2014 13

14. EC2 Evaluation: System Parameters © sebis Al-waleed Shihadeh 2014 14

15. EC2 Evaluation: Environment Parameters © sebis Al-waleed Shihadeh 2014 15

16. Summary & Outlook © sebis Al-waleed Shihadeh 2014 16 Local solution vs. EC2 solution.  Local solution  Very expensive  Requires log time to be prepared.  EC2 solution  Access to powerful machines.  Easy and quick to construct.  Security risk Enhance the 3D Viewer  Add more 3D functionality like picking and selecting specify parts.  Enhance the loading of the 3D model. Move the 3D model information to a database  Parts and shapes names and descriptions.  Engineers comments.

17. Technische Universität München Department of Informatics Chair of Software Engineering for Business Information Systems Boltzmannstraße 3 85748 Garching bei München Tel+49.89.289. Fax+49.89.289.17136 wwwmatthes.in.tum.de Al-waleed Shihadeh 17100 a.shihadeh@tum.de Questions?

18. Backup Slides © sebis Al-waleed Shihadeh 2014 18

19. Extract Shapes © sebis Al-waleed Shihadeh 2014 19

20. Transcode Geometries © sebis Al-waleed Shihadeh 2014 20

21. Local Evaluation © sebis Al-waleed Shihadeh 2014 21

22. HDFS Hadoop Custom JAR © sebis Al-waleed Shihadeh 2014 22

23. Demo © sebis Al-waleed Shihadeh 2014 23