1. www.epikh.eu The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) Riccardo Rotondo (riccardo.rotondo@garr.it)riccardo.rotondo@garr.it Consortium GARR Joint CHAIN/EPIKH School for Application Porting to Science Gateways Beijing, 11.04.2012 GILDA Virtual Machine

2. VM Requirements Virtualization –VMWare (Player(free), Workstation, etc …) –OVF image file available for other platforms Hosting Machine –X86_64 CPU Arch. (Overrides 4Gb of RAM limit) –1,9 Gb Disk space for compressed VMX –4 GB RAM Virtul Machine –1,5 GB RAM –20 GB Virtual disk (about 3GB at first boot) –NAT/Bridged Network (default and suggested) Beijing, Asia 4, 11.04.2012 2

3. VM Services Liferay 6.0.6 on Glassfish 3.1 Liferay plugins sdk MySql v5 VMWare tools (Enables directory sharing) NTP for time synchronization mini_httpd (for Robot Proxy simulation) JSAGA 0.9.15 GridEngine 1.4.5 Beijing, Asia 4, 11.04.2012 3

4. VM Users&Credentials (MySQL) Glassfish/Liferay ’ database backend Beijing, Asia 4, 11.04.2012 4

5. VM Users&Credentials (Glassfish) Glassfish administrator user credentials Administration page Web Address Beijing, Asia 4, 11.04.2012 5

6. VM Users&Credentials (Liferay) Liferay home page Administrator user credentials Beijing, Asia 4, 11.04.2012 6

7. VM Users&Credentials (ROOT) VM ROOT user credentials Beijing, Asia 4, 11.04.2012 7

8. VM Users&Credentials (liferayadmin) VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users Beijing, Asia 4, 11.04.2012 8

9. Glassfish http://gildavm:8130 User: liferayadmin Password: liferayadmin Beijing, Asia 4, 11.04.2012 9

10. Glassfish (Control Panel) Configure the Application server: Resources,Pool DB Connections, Log, other settings … Beijing, Asia 4, 11.04.2012 10

11. Liferay (Sign In) http://gildavm:8162 User: test@liferay.com Password: testtest@liferay.com Beijing, Asia 4, 11.04.2012 11

12. Liferay (Application Space) Beijing, Asia 4, 11.04.2012 12

13. Liferay (MyJobs) Beijing, Asia 4, 11.04.2012 13

14. Liferay (ServerLog) Beijing, Asia 4, 11.04.2012 14

15. About Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information Beijing, Asia 4, 11.04.2012 15

16. Grid Engine Takes care of any Grid Transaction –JAVA APIs + Users Tracking Database Acces to the UsersTrackingDB – mysql –u tracking_user –p usertracking userstracking GridEngine tables +-------------------------+ | Tables_in_userstracking | +-------------------------+ | ActiveGridInteractions | | GridInteractions | | GridOperations | | all_ces | +-------------------------+ GridOperation (for tests) +-------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+--------------+------+-----+---------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | portal | varchar(120) | NO | | NULL | | | description | varchar(200) | NO | | NULL | | +-------------+--------------+------+-----+---------+----------------+ GridEngine Library: –/opt/glassfish3/glassfish/domains/liferay/lib/jsaga-job-management- 1.4.5.jar Beijing, Asia 4, 11.04.2012 16

17. Development Environment (Working Directory) 1 st Define a VM shared folder link or (better) use a ssh OS integration app. 2 nd Define links to existing liferay-plugin-sdk and liferay libraries –ln -s /opt/liferay-plugins-sdk-6.0.6/ \ /mnt/hgfs/ /liferay-plugins-sdk –ln -s /opt/glassfish3/glassfish/domains/liferay/lib/ /mnt/hgfs/ /liferay-lib –WARNING: –On Windows and Mac machines it is not possible to work using symbolic links  Copy the whole content of directories: /opt/liferay-plugins-sdk-6.0.6/ /opt/glassfish3/glassfish/domains/liferay/lib/ –On Mac you may use Mac OSX Fuse+MacFusion to access directories via SSH 3 rd Define a VM readable hostname (optional, only with NAT) –Insert into VM hosting machine /etc/hosts file the entry  As root add: ‘ gildavm'  For Windows machines add the same entru inside: %SystemRoot%\system32\drivers\etc\hosts Beijing, Asia 4, 11.04.2012 17

18. Development Environment (Working Directory) Liferay Inc offers an SDK to develop plugins like portlet: –/opt/liferay-plugins-sdk-6.0.6/ There is an sh script able to create a liferay plugin ready to be edited and deployed. Beijing, Asia 4, 11.04.2012 18

19. Development Environment (IDE) High level development tools can be integrated with the VM exploiting the shared directory; ( NetBeans or Eclipse) See point 7 th of hostname-portlet Wiki content https://gilda.ct.infn.it/wikimain/-/wiki/Main/hostname-portlet See point 7 th of hostname-portlet Wiki content https://gilda.ct.infn.it/wikimain/-/wiki/Main/hostname-portlet Beijing, Asia 4, 11.04.2012 19

20. Compile your portlets As liferayadmin user connect to the VM and ‘cd’ to the liferay-plugin-sdk/portlets/ ; then type: –ant deploy During the compilation, have a look on the produced server log with: –tail -f /opt/glassfish3/glassfish/domains/liferay/logs/server.log WARNING: Compilation may fail even the ant deploy command tells that the deployment is successful; pay attention to the produced log Beijing, Asia 4, 11.04.2012 20

21. Brief intro to hostname portlet Many Grid applications have the same requirements to run over a distributed environment: –Specify the Executable –Define Std-Out/Err files –Speficy files for the Input and Output Sandoboxes –Provide a BASH shell script that pilots the application execution on the remote host –Configure distributed infrastructure services All these requirements have been collected into the hostname portlet: (Historically the hostname Job its’ the ‘hello-world’ test application for Grids) Beijing, Asia 4, 11.04.2012 21

22. hostname portlet (MrBayes) First Grid application (proof of concept) First Grid application (proof of concept) Beijing, Asia 4, 11.04.2012 22

23. hostname portlet (GATE) Prototyping the first experience Beijing, Asia 4, 11.04.2012 23

24. hostname portlet (Template) Portlet Template The interface takes a input file or a text input. It just executes the hostname on the remote machine displaying information like: Working directory, Home directory and produces a files as well. It also simulates the creation of an output file. The interface takes a input file or a text input. It just executes the hostname on the remote machine displaying information like: Working directory, Home directory and produces a files as well. It also simulates the creation of an output file. Beijing, Asia 4, 11.04.2012 24

25. hostname portlet (preferences) Portlet Template Preferences Information System Resource Manager ROBOT prroxy server Virtual Organization Proxy role Application Id (GridEngine) Requirements Pilot Script Portlet Template Preferences Information System Resource Manager ROBOT prroxy server Virtual Organization Proxy role Application Id (GridEngine) Requirements Pilot Script WARNING When the portlet first installs do not forget to place here the application pilot script WARNING When the portlet first installs do not forget to place here the application pilot script Beijing, Asia 4, 11.04.2012 25

26. hostname portlet (about) Portlet Template About Licence Information Versioning Info … Portlet Template About Licence Information Versioning Info … Beijing, Asia 4, 11.04.2012 26

27. Multi Infrastructure Available only from Grid Engine 1.4.3 Same behavior of standard hostname-portlet New preference pane showing more infrastructures and allowing to add/remove/edit infrastructure settings Allows job submissions over different infrastructure. The selection mechanism actually done via a random choice Beijing, Asia 4, 11.04.2012 27

28. Portlet workflow JSP page doView ProcessAction ACTION/VIEW Architecture Hostname code provides two Enum types: Actions and Views For each Action a different View can be selected and easily managed by switch statements The two enumerated types used to manage the portlet behavior Between portlet methods: ‘doView and ProcessAction’. doView JSP page ProcessAction JSP page default Beijing, Asia 4, 11.04.2012 28

29. Testing VM Connect the VM as liferayadmin user Extract from GILDA software repository: –svn checkout http://svn.ct.infn.it/svn/liferay/trunk/gilda/mi-hostname-portlet ! WARNING if there exists already a hostname portlet; remove the directory and extract again the code. This ensures you will have the latest version.http://svn.ct.infn.it/svn/liferay/trunk/gilda/mi-hostname-portlet Change directory to ‘(mi-)hostname-portlet’ and compile: –ant deploy Open the Server Log and wait until appears the message: Connect to the Liferay Page, install and test the portlet Do little changes on the code first and do several tests [#|2011-11- 09T12:26:46.094+0100|INFO|glassfish3.1|javax.enterprise.system.tools.deployment.org.glassfish.deployment.co mmon|_ThreadID=138;_ThreadName=Thread-1;|[AutoDeploy] Successfully autodeployed : /opt/glassfish3/glassfish/domains/liferay/autodeploy/hostname-portlet.war.|#] Beijing, Asia 4, 11.04.2012 29

30. Advanced topics: GILDA VM patches From the GILDA VM 1.4 on it is possible to keep updated the VM. As soon as new patches are available the user will be prompted: –By a message into the VM main screen message –By the welcome message at each remote connection (SSH) New changes checked via: –A scheduled cron job starting each day at 9:00 am –At each machine boot –User may force the check with the script: /etc/GILDA/vmpatcher.sh View content/Apply patches: – vmupdate -i / vmupdate (!Please do a backup before to apply any patch) Beijing, Asia 4, 11.04.2012 30

31. Advanced topics: Liferay domain How to start/stop/restart Liferay –Liferay and Glassish automatically start upon boot. These services (together) can be managed through:  /etc/init.d/liferay –Glassfish may be managed by root user though the utility (*)  /opt/glassfish3/glassfish/bin/asadmin  Start/Stop Glassfish+Liferay: asadmin start-domain (liferay) Asadmin stop-domain (liferay)  Show installed web applications asadmin --port -u list-applications  Undeploy portlets sh asadmin --port --user undeploy  Create domain asadmin create-domain --portbase  Getting help asadmin –help (*) Disable the liferay service if you like to manage Glassfish directly Beijing, Asia 4, 11.04.2012 31

32. Advanced topics: Robot proxy server simulator Liferay GILDA VM simulates internally a Robot Proxy server Production portlets work since /etc/hosts/ includes the defautl proxy server: myproxy.ct.infn.it The GILDA VM uses a mini_httpd server to emulate the Robot Proxy server start/stop/status for mini_httpd services –/etc/init.d/mini_httpd You may use your own personal certificate proxy coping it to the file: –/etc/GILDA/eToken/proxy.txt –Copy a generated proxy from UI (use of a personal certificate)  scp :/tmp/x509up_u501 /etc/GILDA/eToken/proxy.txt Multi Infrastructure job submission needs a dedicated file into: /opt/mini_httpd-1.19/eTokenServer/eToken/ /index.html Beijing, Asia 4, 11.04.2012 32

33. Advanced topics: Network Checker The VM provides a service able to restart the network as soon as network connection changes More useful when operating the VM in Bridged mode Called on startup by /etc/inittab Script: /etc/GILDA/NetworkChecker.sh Once network settings change boot and ssh login messages will change accordingly Beijing, Asia 4, 11.04.2012 33

34. References GILDA VM –https://gilda.ct.infn.it/serviceshttps://gilda.ct.infn.it/services GILDA VM Instructions –https://gilda.ct.infn.it/wikimain/-/wiki/Main/GILDA+Liferay+Virtual+Machinehttps://gilda.ct.infn.it/wikimain/-/wiki/Main/GILDA+Liferay+Virtual+Machine Wiki hostname portlet template –https://gilda.ct.infn.it/wikimain/-/wiki/Main/hostname-portlethttps://gilda.ct.infn.it/wikimain/-/wiki/Main/hostname-portlet Beijing, Asia 4, 11.04.2012 34

35. Questions? Beijing, Asia 4, 11.04.2012 35