The Developing Environment Mr. Riccardo Rotondo Consortium GARR, Rome, Italy

Outline GILDA Virtual Machine Portlet Develop Hostname Portlet OGF35, Delft (NL), 18 Jun 20122

VM Requirements Virtualization o VMWare (Player(free), Workstation, etc …) o OVF image file available for other platforms Hosting Machine o X86_64 CPU Arch. (Overrides 4Gb of RAM limit) o 1,9 Gb Disk space for compressed VMX o 4 GB RAM Virtul Machine o 1,5 GB RAM o 20 GB Virtual disk (about 3GB at first boot) o NAT/Bridged Network (default and suggested) OGF35, Delft (NL), 18 Jun 20123

VM Services Liferay on Glassfish 3.1 Liferay plugins sdk MySql v5 VMWare tools (Enables directory sharing) NTP for time synchronization JSAGA GridEngine OGF35, Delft (NL), 18 Jun 20124

VM Users&Credentials (MySQL) OGF35, Delft (NL), 18 Jun Glassfish/Liferay ’ database backend

VM Users&Credentials (Glassfish) OGF35, Delft (NL), 18 Jun Glassfish administrator user credentials Administration page Web Address

VM Users&Credentials (Liferay) OGF35, Delft (NL), 18 Jun Liferay home page Administrator user credentials

VM Users&Credentials (ROOT) OGF35, Delft (NL), 18 Jun VM ROOT user credentials

VM Users&Credentials (liferayadmin) OGF35, Delft (NL), 18 Jun VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in ‘sudoers’ users

Glassfish 10 User: liferayadmin Password: liferayadmin OGF35, Delft (NL), 18 Jun 2012

Glassfish (Control Panel) OGF35, Delft (NL), 18 Jun Configure the Application server: Resources,Pool DB Connections, Log, other settings …

Liferay (Sign In) OGF35, Delft (NL), 18 Jun User: Password:

Liferay (Application Space) OGF35, Delft (NL), 18 Jun

Liferay (MyJobs) 14OGF35, Delft (NL), 18 Jun 2012

Liferay (ServerLog) 15OGF35, Delft (NL), 18 Jun 2012

About 16 Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information OGF35, Delft (NL), 18 Jun 2012

Grid Engine Takes care of any Grid Transaction o JAVA APIs + Users Tracking Database Acces to the UsersTrackingDB o mysql –u tracking_user –D userstracking -p o (password: usertracking) GridEngine tables | Tables_in_userstracking | | ActiveGridInteractions | | GridInteractions | | GridOperations | | all_ces | GridOperation (for tests) | Field | Type | Null | Key | Default | Extra | | id | int(11) | NO | PRI | NULL | auto_increment | | portal | varchar(120) | NO | | NULL | | | description | varchar(200) | NO | | NULL | | GridEngine Library: o /opt/glassfish3/glassfish/domains/liferay/lib/jsaga-job-management jar OGF35, Delft (NL), 18 Jun

What is Porltet ? Defined in the Java Specification Request (JSR 168/286) A porlet is a web component able to set a frame portion of a web page. Differently from Servlet it needs a Porltlet Container to allow the user to interact with it. Standard Porltet can be deployed in any Porlet Container in case they will use standard libray. 18OGF35, Delft (NL), 18 Jun 2012

Porlet Life Cycle OGF35, Delft (NL), 18 Jun End of Services? NOYES

Porltlet Interface Method Class GenericPortlets { init (PortletConfig); processAction (ActionRequest, ActionResponse); render (RenderRequest, RenderResponse); destroy(); doView (Request, Response); doEdit (Reuqest, Response); doHelp (Request, Response); } OGF35, Delft (NL), 18 Jun

Portlet Mode A porltet works in three different states. View Mode: generate a markup ( a portion of a web page); Edit Mode: allow portlet customisation and setting preferences. Help Mode: explains portlet functionalities. OGF35, Delft (NL), 18 Jun

doEdit() Action Request OGF35, Delft (NL), 18 Jun Action Request processAction() Render Request view.jsp doView() edit.jsp Action Request processAction() Render Request edit.jsp view.jsp

Render Request OGF35, Delft (NL), 18 Jun Render Request view.jsp doView() view2.jsp

Portlet files (root) build.xml docroot/ WEB_INF/ css/ images/ js/ jsp_page_1.jsp … jsp_page_n.jsp OGF35, Delft (NL), 18 Jun 2012 Used by ‘ ant ’ to compile and deploy the portlet Define here JSP’s styles JSP image files Portlet JSP pages 24

Portlet files WEB_INF/ glassfish-web.xml web.xml lib/ Job/ liferay-display.xml liferay-plugin-package.properties liferay-portlet.xml portlet.xml src/ /code.java tld/ OGF35, Delft (NL), 18 Jun 2012 Used to define database and other resources settings Jar Library directory (local scope) Job files (Custom dir) Java source files Portlet configuration files Portlet configuration files XML structure file 25

Parameters exchange (JSP->Java) Portlet modes: o VIEW – Userd for normal portlet operations -> calls processAction() o EDIT – Userd to configure portlet parameters -> calls doEdit() o HELP – Userd to show help information -> calls doHelp() OGF35, Delft (NL), 18 Jun 2012 <form action=action="<portlet:actionURL portletMode="view"> <portlet:param name="param_name_1" value="param value 1"/> … <portlet:param name="param_name_n" value="param value n"/> … <form action=action="<portlet:actionURL portletMode="view"> <portlet:param name="param_name_1" value="param value 1"/> … <portlet:param name="param_name_n" value="param value n"/> … processAction(ActionRequest request,… doView/doHelp/doEdit(RenderRequest request,… // To obtain the parameter just set … String param_i = request.getParameter("param_name_i"); processAction(ActionRequest request,… doView/doHelp/doEdit(RenderRequest request,… // To obtain the parameter just set … String param_i = request.getParameter("param_name_i"); 26

Parameters exchange (Java->JSP) From Java code: Inside the JSP load values with: OGF35, Delft (NL), 18 Jun 2012 <% // To load variables from Portlet Class … %> " scope="request"/> <% // To reference a param value %> <% // To load variables from Portlet Class … %> " scope="request"/> <% // To reference a param value %> doView/doHelp/doEdit(RenderRequest request,… // To send the parameter just call … request.setAttribute("param_name_j", param_name_j); doView/doHelp/doEdit(RenderRequest request,… // To send the parameter just call … request.setAttribute("param_name_j", param_name_j); 27

Portlet workflow OGF35, Delft (NL), 18 Jun 2012 JSP page doView ProcessAction ACTION/VIEW Architecture Hostname code provides two Enum types: Actions and Views For each Action a different View can be selected and easily managed by switch statements The two enumerated types used to manage the portlet behavior Between portlet methods: ‘doView and ProcessAction’. Jsp/Java Variable: ‘ PortletStatus ’ doView JSP page ProcessAction JSP page default init 28

Portlet Session data OGF35, Delft (NL), 18 Jun 2012 JSP page doView ProcessAction Two objecst: actionRequest processAction() renderRequest doView() doHelp() doEdit() init ProcessAction sends parameters to RenderRequest through: ActionResponse (2 nd parameter of ProcessAction ) response.setRenderParameter("param_name_y", param_value_y); doView/doHelp/doEdit (1 st parameter) RenderResponse param_name_w = request.getParameter (" param_name_w "); Parameters to jsp and select it: request.setAttribute("JobId", inputJobId); getPortletContext().getRequestDispatcher(”/.jsp"); 29

hostname-portlet example Portlet to send a test job into a distributed infrastructure o Input: Input file or a text A human readable job identifier o Ouput Hostname of running machine File containing information about the running machine Portlet statuses o INPUT (Shows the input fields) o SUBMIT (Shows the job submission feedback) o Jsp/Java variable ‘ PortletStatus ’ OGF35, Delft (NL), 18 Jun 2012 input.jsp submit.jsp 30

Hostname Action/View enums Actions o ACTION_INPUT o ACTION_SUBMIT Views (Possible portlet views) o VIEW_INPUT o VIEW_SUBMIT OGF35, Delft (NL), 18 Jun 2012 Default Action Simply forward portlet status to the doView() in: RenderParameter Default Action Simply forward portlet status to the doView() in: RenderParameter 31

Hostname Action/View Actions o ACTION_INPUT o ACTION_SUBMIT Views o VIEW_INPUT o VIEW_SUBMIT OGF35, Delft (NL), 18 Jun 2012 View Select the INPUT form view input.jsp (default view mode) View Select the INPUT form view input.jsp (default view mode) 32

Hostname Action/View Actions o ACTION_INPUT o ACTION_SUBMIT Views o VIEW_INPUT o VIEW_SUBMIT OGF35, Delft (NL), 18 Jun 2012 This action will submit the hostname job into the distributed infrastructure This action will submit the hostname job into the distributed infrastructure " method="post”> 33

Hostname Action/View Actions o ACTION_INPUT o ACTION_SUBMIT Views o VIEW_INPUT o VIEW_SUBMIT OGF35, Delft (NL), 18 Jun 2012 Select the proper view showing the Information page that shows the job has been submitted Select the proper view showing the Information page that shows the job has been submitted 34

Portlet Initialization portlet.xlm contains portlet initialization parameters init parameter name init parameter value The init() method uses s to load Portlet’ default parameter values Some of the default parameters values will be used to load portlet default preferences OGF35, Delft (NL), 18 Jun 2012 public void init() throws PortletException { // Load default values from portlet.xml init_PortletVersion = getInitParameter(“init parameter name"); … public void init() throws PortletException { // Load default values from portlet.xml init_PortletVersion = getInitParameter(“init parameter name"); … 35

hostname’ hostname’ To make parameter management easier all values are threated as String type init_PortletVersion - Portlet version number init_bdiiHost - Information system service (ldap://...:2170) init_wmsHost - Infrastructure resource manager (wms://...:7443/glite_wms_wmproxy_server) init_pxServerHost - Hostname of Robot proxy server (myproxy.ct.infn.it not used yet) init_pxRobotId - Id of the robot certificate init_pxRobotVO - Infrastructure virtual organization init_pxRobotRole - Role of generated proxy init_pxUserProxy - Standard proxy file (instead of Robots) init_pxRobotRenewalFlag - Allows automatic job renewal mechanism init_SciGwyAppId - Science Gateway application identifier '9' for GILDA VM tester application GridEngine' User tracking database (UTDB) access credentials o init_SciGwyUserTrackingDB_Hostname o init_SciGwyUserTrackingDB_Username o init_SciGwyUserTrackingDB_Password o init_SciGwyUserTrackingDB_Database – Users should not use Direct DB access to UTDB init_JobRequirements - Application job requirements init_pilotScript - Application pilot script OGF35, Delft (NL), 18 Jun

Init parameters init_PortletVersion init_bdiiHost init_wmsHost init_pxServerHost init_pxRobotId init_pxRobotVO init_pxRobotRole init_pxUserProxy init_pxRobotRenewalFlag init_SciGwyAppId init_SciGwyUserTrackingDB_Hostname init_SciGwyUserTrackingDB_Username init_SciGwyUserTrackingDB_Password init_SciGwyUserTrackingDB_Database init_JobRequirements init_pilotScript Your portlet version Infrastructure settings … Robot proxy settings Robot Id, Virtual Organization, Proxy Role, Renewal Flag pxUserProxy (mutually exclusive to Robots) Allows the use of a user proxy Just provide a valid path to a proxy file Robot proxy settings Robot Id, Virtual Organization, Proxy Role, Renewal Flag pxUserProxy (mutually exclusive to Robots) Allows the use of a user proxy Just provide a valid path to a proxy file Each application has a unique Id (GridEngine) GridEngine connection credentials (Unused) Grid Job requirements Job’ pilot script (bash) OGF35, Delft (NL), 18 Jun

Hostname Portlet preferences getPreferences method called by both: o doView and processAction private void getPreferences( ActionRequest actionRequest,RenderRequest renderRequest) { PortletPreferences prefs=null; if(null!=actionRequest) prefs = actionRequest.getPreferences(); else if(null != renderRequest) prefs = renderRequest.getPreferences(); if (null != prefs) { // The second parameter assigns a default value pref_value=prefs.getValue(" ", " "); private void getPreferences( ActionRequest actionRequest,RenderRequest renderRequest) { PortletPreferences prefs=null; if(null!=actionRequest) prefs = actionRequest.getPreferences(); else if(null != renderRequest) prefs = renderRequest.getPreferences(); if (null != prefs) { // The second parameter assigns a default value pref_value=prefs.getValue(" ", " "); OGF35, Delft (NL), 18 Jun

Multi Infrastructure Available only from Grid Engine Same behavior of standard hostname-portlet New preference pane showing more infrastructures and allowing to add/remove/edit infrastructure settings Allows job submissions over different infrastructure. The selection mechanism actually done via a random choice 39OGF35, Delft (NL), 18 Jun 2012

Job Submission (1/4) // Job details String executable="/bin/sh"; // Application executable String arguments="pilot_script.sh"; // executable' arguments String outputPath="/tmp/"; // Output Path String outputFile="application.out"; // Application stdout String errorFile="application.err"; // Application stderr // InputSandbox (string with comma separated list of file names) String inputSandbox= appServerPath +"/WEB-INF/job/pilot_script.sh” +","+ ; // OutputSandbox (string with comma separated list of file names) String outputSandbox="generated_output_file"; OGF35, Delft (NL), 18 Jun

Job Submission (2/4) // Software tags String jdlRequirements[] = pref_JobRequirements.split(";"); int numRequirements=0; for(int i=0; i<jdlRequirements.length; i++) if(!jdlRequirements[i].equals("")) { jdlRequirements[numRequirements] = "JDLRequirements=("+jdlRequirements[i]+")"; numRequirements++; } // Instanciate the JSAGA object JSagaJobSubmission tmpJSaga = new JSagaJobSubmission(); tmpJSaga.setBDII(bdiiHost);// Infrastructure top BDII OGF35, Delft (NL), 18 Jun

Job Submission (3/4) // Proxy renewal flag if((pref_pxRobotRenewalFlag.toLowerCase()).equals("true")) pxRobotRenewalFlag=true; else pxRobotRenewalFlag=false; // Associate a valid proxy (Robot/UserProxy) // Specifying a path for local proxy it will be used instead if(pxUserProxy==null || pxUserProxy.equals("")) { tmpJSaga.useRobotProxy( pxRobotId,pxRobotVO,pxRobotRole,pxRobotRenewalFlag ); else tmpJSaga.setUserProxy(pxUserProxy); OGF35, Delft (NL), 18 Jun

Job Submission (4/4) tmpJSaga.setExecutable(executable); // Specify the executeable tmpJSaga.setArguments(arguments); // Specify the application' arguments tmpJSaga.setOutputPath(outputPath); // Specify the output directory tmpJSaga.setInputFiles(inputSandbox); // Setup input files (InputSandbox) tmpJSaga.setOutputFiles(outputSandbox); // Setup output files (OutputSandbox) tmpJSaga.setJobOutput(outputFile); // Specify the std-outputr file tmpJSaga.setJobError(errorFile); // Specify the std-error file if(numRequirements>0) tmpJSaga.setJDLRequirements(jdlRequirements); // Assign requirements // Submit the job // If a WMS is specified the call to Job submission changes if(wmsHost!=null && !wmsHost.equals("")) tmpJSaga.submitJobAsync(username,hostUTDB,applicationId,wmsHost,jobIdentifier ); else tmpJSaga.submitJobAsync( username,hostUTDB,applicationId,jobIdentifier ); OGF35, Delft (NL), 18 Jun

From hostname to your portlet Extract hostname portlet form svn svn checkout ‘ cd ’ inside the hostname-portlet directory and try to compile it with the command: ant deploy If the compilation is successful create a physical copy of the portlet directory source code o cp –r hostname-portlet -portlet Customize your portlet settings configuring several XML files: o docroot/WEB-INF/portlet.xml -portlet _portlet … init_PortletVersion Specify your Portlet' version number … other init parameters OGF35, Delft (NL), 18 Jun

From hostname to your portlet o docroot/WEB-INF/portlet.xml (still) Title of yor porltlet Short title Portlet' keywords o docroot/WEB-INF/liferay-display.xml " (your institution ‘GILDA’) " (name displayed into the mangment interface) o docroot/WEB-INF/lib Overrides or extends the portlet java code global library repository ( /opt/glassfish3/glassfish/domains/liferay/lib/ ) o docroot/WEB-INF/liferay-portlet.xml o docroot/WEB-INF/glassfish-web.xml (Information managed by App server Glassfish) o docroot/images/ Each image used by the jsp pages should be located here o docroot/icon.png Place here a 16x16 image file with your portlet icon OGF35, Delft (NL), 18 Jun

From hostname to your portlet o /docroot/WEB-INF/src/it/infn/ct/hostname_portlet.java Make a copy of this source file into directory: o /.java> WARNING: Do not forget to assign to the class the same name of the java file o Start to develop the interface modifying jsp files and change java code Enums with correct Actions and Views modes using human readable identifiers. For simple Appliocation user interfaces there will be no need to add other JSPs or action/view modes to the existing Enum o Edit the java code in order to manage your application specific interface and prepare the correct sandboxes to submit properly the job o Prepare the pilot script and use it as parameter. OGF35, Delft (NL), 18 Jun

Development Environment (IDE) High level development tools can be integrated with the VM exploiting the shared directory; ( NetBeans or Eclipse) 47 See point 7 th of hostname-portlet Wiki content See point 7 th of hostname-portlet Wiki content OGF35, Delft (NL), 18 Jun 2012

Create NetBeans project Open new project and select: Categories: Java web Projects: Web Appliacations with existing sources. Press 'Next' Location: Point Netbeans to locate the portlet directory through the VM shared folder. Press 'Next' Accept any suggestion and proceed Press 'Next' Add other directory places: WEB-INF Content: Select the docroot/WEB-INF directory inside the portlet directory Library folder: point it to docroot/WEB-INF/lib Then press the 'Finish' button and the project will be created Right click on the project name and click on Peferences, then Libraries Select all jars pointed by :liferay_libs/ directory OGF35, Delft (NL), 18 Jun

Development Environment (Working Directory) Liferay Inc offers an SDK to develop plugins like portlet: o /opt/liferay-plugins-sdk-6.0.6/ There is an sh script able to create a liferay plugin ready to be edited and deployed. 49OGF35, Delft (NL), 18 Jun 2012

Compile your portlets As liferayadmin user connect to the VM and ‘ cd ’ to the liferay-plugin-sdk/portlets/ ; then type: o ant deploy During the compilation, have a look on the produced server log with: o tail -f /opt/glassfish3/glassfish/domains/liferay/logs/server.log WARNING: Compilation may fail even the ant deploy command tells that the deployment is successful; pay attention to the produced log 50OGF35, Delft (NL), 18 Jun 2012

Testing VM Connect the VM as liferayadmin user Extract from GILDA software repository: o svn checkout ! WARNING if there exists already a hostname portlet; remove the directory and extract again the code. This ensures you will have the latest version. Change directory to ‘(mi-)hostname-portlet’ and compile: o ant deploy Open the Server Log and wait until appears the message: Connect to the Liferay Page, install and test the portlet Do little changes on the code first and do several tests 51 [#| T12:26: |INFO|glassfish3.1|javax.enterprise.system.tools.deployment.org.glassfish.deployment.co mmon|_ThreadID=138;_ThreadName=Thread-1;|[AutoDeploy] Successfully autodeployed : /opt/glassfish3/glassfish/domains/liferay/autodeploy/hostname-portlet.war.|#] OGF35, Delft (NL), 18 Jun 2012

Advanced topics: GILDA VM patches From the GILDA VM 1.4 on it is possible to keep updated the VM. As soon as new patches are available the user will be prompted: o By a message into the VM main screen message o By the welcome message at each remote connection (SSH) New changes checked via: o A scheduled cron job starting each day at 9:00 am o At each machine boot o User may force the check with the script: /etc/GILDA/vmpatcher.sh View content/Apply patches: ovmupdate -i / vmupdate (!Please do a backup before to apply any patch) 52OGF35, Delft (NL), 18 Jun 2012

Advanced topics: Liferay domain How to start/stop/restart Liferay o Liferay and Glassish automatically start upon boot. These services (together) can be managed through: /etc/init.d/liferay o Glassfish may be managed by root user though the utility (*) /opt/glassfish3/glassfish/bin/asadmin Start/Stop Glassfish+Liferay: o asadmin start-domain (liferay) o Asadmin stop-domain (liferay) Show installed web applications o asadmin --port -u list-applications Undeploy portlets o sh asadmin --port --user undeploy Create domain o asadmin create-domain --portbase Getting help o asadmin –help (*) Disable the liferay service if you like to manage Glassfish directly 53OGF35, Delft (NL), 18 Jun 2012

References GILDA VM o GILDA VM Instructions o /wiki/Main/GILDA+Liferay+Virtual+Machine /wiki/Main/GILDA+Liferay+Virtual+Machine Wiki hostname portlet template o OGF35, Delft (NL), 18 Jun 2012

Thank you for your kind attention OGF35, Delft (NL), 18 Jun