Grid Initiatives for e-Science virtual communities in Europe and Latin America Riccardo Rotondo Consortium GARR Joint School for Application Porting to Science Gateways Bucaramanga, GILDA Virtual Machine

VM Requirements Virtualization –VMWare (Player(free), Workstation, etc …) –OVF image file available for other platforms Hosting Machine –X86_64 CPU Arch. (Overrides 4Gb of RAM limit) –1,9 Gb Disk space for compressed VMX –4 GB RAM Virtul Machine –1,5 GB RAM –20 GB Virtual disk (about 3GB at first boot) –NAT/Bridged Network (default and suggested) Bucaramanga,

VM Services Liferay on Glassfish 3.1 Liferay plugins sdk MySql v5 VMWare tools (Enables directory sharing) NTP for time synchronization mini_httpd (for Robot Proxy simulation) JSAGA GridEngine Bucaramanga,

VM Users&Credentials (MySQL) Bucaramanga, Glassfish/Liferay ’ database backend

VM Users&Credentials (Glassfish) Bucaramanga, Glassfish Administration page Web Address Administrator user credentials Glassfish Administration page Web Address Administrator user credentials

VM Users&Credentials (Liferay) Bucaramanga, Liferay home page Administrator user credentials

VM Users&Credentials (liferayadmin) Bucaramanga, VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users

Glassfish Bucaramanga, User: liferayadmin Password: liferayadmin

Glassfish (Control Panel) Bucaramanga, Configure the Application server: Resources,Pool DB Connections, Log, other settings …

Liferay (Sign In) Bucaramanga, User: Password:

Liferay (Application Space) Bucaramanga,

Liferay (MyJobs) Bucaramanga,

Liferay (ServerLog) Bucaramanga,

About Bucaramanga, Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information

Grid Engine Takes care of any Grid Transaction –JAVA APIs + Users Tracking Database Acces to the UsersTrackingDB – mysql –u tracking_user –D userstracking -p –(password: usertracking) GridEngine tables | Tables_in_userstracking | | ActiveGridInteractions | | GridInteractions | | GridOperations | | all_ces | GridOperation (for tests) | Field | Type | Null | Key | Default | Extra | | id | int(11) | NO | PRI | NULL | auto_increment | | portal | varchar(120) | NO | | NULL | | | description | varchar(200) | NO | | NULL | | GridEngine Library: –/opt/glassfish3/glassfish/domains/liferay/lib/jsaga-job-management jar Bucaramanga,

Development Environment (Working Directory) 1 st Define a VM shared folder link or (better) use a ssh OS integration app. 2 nd Define links to existing liferay-plugin-sdk and liferay libraries –ln -s /opt/liferay-plugins-sdk-6.0.6/ \ /mnt/hgfs/ /liferay-plugins-sdk –ln -s /opt/glassfish3/glassfish/domains/liferay/lib/ /mnt/hgfs/ /liferay-lib –WARNING: –On Windows and Mac machines it is not possible to work using symbolic links  Copy the whole content of directories: /opt/liferay-plugins-sdk-6.0.6/ /opt/glassfish3/glassfish/domains/liferay/lib/ –On Mac you may use Mac OSX Fuse+MacFusion to access directories via SSH 3 rd Define a VM readable hostname (optional, only with NAT) –Insert into VM hosting machine /etc/hosts file the entry  As root add: ‘ gildavm'  For Windows machines add the same entru inside: %SystemRoot%\system32\drivers\etc\hosts Bucaramanga,

Development Environment (Working Directory) Liferay Inc offers an SDK to develop plugins like portlet: –/opt/liferay-plugins-sdk-6.0.6/ There is an sh script able to create a liferay plugin ready to be edited and deployed. Bucaramanga,

Development Environment (IDE) High level development tools can be integrated with the VM exploiting the shared directory; ( NetBeans or Eclipse) Bucaramanga, See point 7 th of hostname-portlet Wiki content See point 7 th of hostname-portlet Wiki content

Compile your portlets As liferayadmin user connect to the VM and ‘cd’ to the liferay-plugin-sdk/portlets/ ; then type: –ant deploy During the compilation, have a look on the produced server log with: –tail -f /opt/glassfish3/glassfish/domains/liferay/logs/server.log WARNING: Compilation may fail even the ant deploy command tells that the deployment is successful; pay attention to the produced log Bucaramanga,

Brief intro to hostname portlet Many Grid applications have the same requirements to run over a distributed environment: –Specify the Executable –Define Std-Out/Err files –Speficy files for the Input and Output Sandoboxes –Provide a BASH shell script that pilots the application execution on the remote host –Configure distributed infrastructure services All these requirements have been collected into the hostname portlet: (Historically the hostname Job its’ the ‘hello-world’ test application for Grids) Bucaramanga,

hostname portlet (MrBayes) Bucaramanga, First Grid application (proof of concept) First Grid application (proof of concept)

hostname portlet (GATE) Bucaramanga, Prototyping the first experience

hostname portlet (Template) Bucaramanga, Portlet Template The interface takes a input file or a text input. It just executes the hostname on the remote machine displaying information like: Working directory, Home directory and produces a files as well. It also simulates the creation of an output file. The interface takes a input file or a text input. It just executes the hostname on the remote machine displaying information like: Working directory, Home directory and produces a files as well. It also simulates the creation of an output file.

hostname portlet (preferences) Bucaramanga, Portlet Template Preferences Information System Resource Manager ROBOT prroxy server Virtual Organization Proxy role Application Id (GridEngine) Requirements Pilot Script Portlet Template Preferences Information System Resource Manager ROBOT prroxy server Virtual Organization Proxy role Application Id (GridEngine) Requirements Pilot Script WARNING When the portlet first installs do not forget to place here the application pilot script WARNING When the portlet first installs do not forget to place here the application pilot script

hostname portlet (about) Bucaramanga, Portlet Template About Licence Information Versioning Info … Portlet Template About Licence Information Versioning Info …

Multi Infrastructure Available only from Grid Engine Same behavior of standard hostname-portlet New preference pane showing more infrastructures and allowing to add/remove/edit infrastructure settings Allows job submissions over different infrastructure. The selection mechanism actually done via a random choice Bucaramanga,

Portlet workflow Bucaramanga, JSP page doView ProcessAction ACTION/VIEW Architecture Hostname code provides two Enum types: Actions and Views For each Action a different View can be selected and easily managed by switch statements The two enumerated types used to manage the portlet behavior Between portlet methods: ‘doView and ProcessAction’. doView JSP page ProcessAction JSP page default

Testing VM Connect the VM as liferayadmin user Extract from GILDA software repository: –svn checkout svn://svn.code.sf.net/p/ctsciencegtwys/liferay/trunk/gilda/mi-hostname-portletsvn://svn.code.sf.net/p/ctsciencegtwys/liferay/trunk/gilda/mi-hostname-portlet –! WARNING if there exists already a hostname portlet; remove the directory and extract again the code. This ensures you will have the latest version. Change directory to ‘(mi-)hostname-portlet’ and compile: –ant deploy Open the Server Log and wait until appears the message: Connect to the Liferay Page, install and test the portlet Do little changes on the code first and do several tests Bucaramanga, [#| T12:26: |INFO|glassfish3.1|javax.enterprise.system.tools.deployment.org.glassfish.deployment.co mmon|_ThreadID=138;_ThreadName=Thread-1;|[AutoDeploy] Successfully autodeployed : /opt/glassfish3/glassfish/domains/liferay/autodeploy/hostname-portlet.war.|#]

Advanced topics: GILDA VM patches From the GILDA VM 1.4 on it is possible to keep updated the VM. As soon as new patches are available the user will be prompted: –By a message into the VM main screen message –By the welcome message at each remote connection (SSH) New changes checked via: –A scheduled cron job starting each day at 9:00 am –At each machine boot –User may force the check with the script: /etc/GILDA/vmpatcher.sh View content/Apply patches: – vmupdate -i / vmupdate (!Please do a backup before to apply any patch) Bucaramanga,

Advanced topics: Liferay domain How to start/stop/restart Liferay –Liferay and Glassish automatically start upon boot. These services (together) can be managed through:  /etc/init.d/liferay –Glassfish may be managed by root user though the utility (*)  /opt/glassfish3/glassfish/bin/asadmin  Start/Stop Glassfish+Liferay: asadmin start-domain (liferay) Asadmin stop-domain (liferay)  Show installed web applications asadmin --port -u list-applications  Undeploy portlets sh asadmin --port --user undeploy  Create domain asadmin create-domain --portbase  Getting help asadmin –help (*) Disable the liferay service if you like to manage Glassfish directly Bucaramanga,

Advanced topics: Robot proxy server simulator Liferay GILDA VM simulates internally a Robot Proxy server Production portlets work since /etc/hosts/ includes the defautl proxy server: myproxy.ct.infn.it The GILDA VM uses a mini_httpd server to emulate the Robot Proxy server start/stop/status for mini_httpd services –/etc/init.d/mini_httpd You may use your own personal certificate proxy coping it to the file: –/etc/GILDA/eToken/proxy.txt –Copy a generated proxy from UI (use of a personal certificate)  scp :/tmp/x509up_u501 /etc/GILDA/eToken/proxy.txt Multi Infrastructure job submission needs a dedicated file into: /opt/mini_httpd-1.19/eTokenServer/eToken/ /index.html Bucaramanga,

Advanced topics: Network Checker The VM provides a service able to restart the network as soon as network connection changes More useful when operating the VM in Bridged mode Called on startup by /etc/inittab Script: /etc/GILDA/NetworkChecker.sh Once network settings change boot and ssh login messages will change accordingly Bucaramanga,

References GILDA VM – GILDA VM Instructions – Wiki hostname portlet template – Bucaramanga,

Questions? Bucaramanga,