Metadata-rich Service Oriented Grid Portals Mehmet Nacar mnacar@cs.indiana.edu

Outline Research Objectives Background Grid portals Research issues Architecture Contributions December 2, 2018 Mehmet Nacar

Research Objectives I Improve ease of Grid portal development We will research problems in federating multiple portal instances Integrating portals based on the same code (e.g. all GridSphere) Integrating portals with different code bases Integrating portals with non-portal services identity management authorization December 2, 2018 Mehmet Nacar

Research Objectives II We will investigate problems turning portals into Grid Web services It is the right architecture especially for Grid services WSRP is the Web Service standard WSRP must be coupled to other Web service standards for addressing and reliability This will require an examination and implementation of SOAP handlers Since user sessions expire, but grid jobs may still running at the backend. December 2, 2018 Mehmet Nacar

What is a portal? Web based application provides Personalization, single sign on and content aggregation Web, science and educational portal Yahoo, Google CIMA, VLAB and LEAD OneStart and Oncourse Portal vs. portal framework Jetspeed, GridSphere and Sakai frameworks Modern portal framework vs. traditional portals Jetspeed, GridSphere and uPortal CGI-based portals, ReciprocalNet December 2, 2018 Mehmet Nacar

Yahoo portal December 2, 2018 Mehmet Nacar Google portal snapshot replaces with portal skeleton page December 2, 2018 Mehmet Nacar

Portal page December 2, 2018 Mehmet Nacar

Portlets What is a portlet? Portlet examples Portlet container Building blocks of portals Portlet examples Calendar, chat, file transfer, weather and news. We have developed Grid portlets for OGCE Myproxy, job submission and GridFtp. Portlet container Provides lifecycle management of portlets Defines window states and portlet modes Equips packaging and deployment JSR 168 specification defines Portlet API Brings standard to deploy portlets within various portals December 2, 2018 Mehmet Nacar

Portal framework Portal framework aggregates portlet contents Portal services are portal capabilities Portal services are evaluated in different aspects: Administrative services Account management, portlet management User management services User profile, layout management There are also related portlets Administrative portlets are account creation, portlet registry User portlets are file transfer and calendar Portal front page displays anonymous content with login portlet What services involves when a user logs into the portal Login service  Authorization Service  Layout Service Picture of portal containers and portal frameworks run on tomcat servlet engine. Show portal services interaction in between portal container and portal framework December 2, 2018 Mehmet Nacar

Problem statement Portals and portlets are standardized but this opens new problem areas, particularly for Grid portals Federating multiple portal instances built from JSR 168 compatible frameworks. We must decouple portal “capabilities” from the container and make them into interoperable Web services. Examples include identity management, authorization and logging. Portlets and portals themselves can become Web services that can be federated. But we must show these can be combined with other Web service registration, discovery, reliability, and others. December 2, 2018 Mehmet Nacar

Grid portals What makes a portal Grid-enabled? Portlets provide access to Grid services Portal server leverage Grid credentials to access Grid services Provide capabilities/portlets GridFTP, job submission and job monitoring What tools and services are available to build Grid portals? Java COG: Grid client programming API MyProxy: credential management service OGCE: collection of portlets and tools for building portlets GAMA and PURSe: integrated Grid account management We will examine two motivating examples VLAB: traditional job submission portal for computational chemistry CIMA: scientific instrument and data portal December 2, 2018 Mehmet Nacar

VLAB Portal December 2, 2018 Mehmet Nacar

VLAB: The Virtual Laboratory for Earth and Planetary Materials Primarily a traditional job submission, monitoring, and management portal. Collaborative Grid services and portals support computational material science. Component based Grid portlet development makes application development easier. VLAB Challenges: Grid Portlets must be easy to develop using component libraries. HTML <form> actions in Grid portals typically have several steps: Stage data files in and out of the desired remote host. Run one or more executables. Keep track of job progress Store all of the information as “job pedigree” for reproducibility. December 2, 2018 Mehmet Nacar

Research issue: Grid components Grid portlet applications require dynamic interaction and fine-grained components. Portlets themselves need to be built out of components Grid services mostly use request/response paradigm Grid portlets use web forms heavily Compared to wikis, blogs, RSS-driven news portals, which have a different problem of content management. Grid widgets can provide components for: Proxy credential GridFtp operations Job submissions, multi-staged jobs Using widgets as tag libraries help to encapsulate reusable Grid components December 2, 2018 Mehmet Nacar

Grid Tag Libraries Grid tag libraries use the Java COG Kit abstractions Grid tags simplify association of composite Grid actions And increase reuse of code There are associated custom JSF tag extensions we’ve developed: <o:proxy/>, <o:task/>, <o:taskGraph/>, <o:taskAdd/> and <o:contextStore/> Taskgraph supports multi-staged jobs <f:verbatim> <o:taskGraph id="myGraph" method="#{taskgraph.test}" binding="#{taskgraph.taskgr}" > <o:proxy id=“myproxy" method=“#{proxy.create}" /> <o:task id="task1" method="#{task.create}" type="FileTransferInput" /> <o:task id="task2" method="#{task.create}" type="JobSubmit" /> <o:task id="task3" method="#{task.create}" type="FileTransferOutput" /> <o:taskAdd id="taskadd1" name="task1" depends="task2" method="taskgraph.add" /> <o:taskAdd id="taskadd1" name="task2" depends="task3" method="taskgraph.add" /> </o:taskGraph> </f:verbatim> December 2, 2018 Mehmet Nacar

CIMA Crystallography portal CIMA picture snapshot December 2, 2018 Mehmet Nacar

CIMA (Common Instrument Middleware Architecture) Primarily a data portal to online instruments Crystallographers collect data in participating laboratories and collaborate on samples. Portlets have to access data with group privileges. Explain cima use case, with problems? December 2, 2018 Mehmet Nacar

Research issue: federating portal instances CIMA and LSST Dark Energy Survey portal presents this problem: Multiple CIMA portal deployment instances are associated with different labs (IUMSC, Purdue, University of Sydney etc) Need to solve problem of portal identity federation. Recall identity management is key portal capability. Other problems (such as federated authorization) flow from this. Portal frameworks like GridSphere, uPortal and Jetspeed only support local “capabilities”. This approach has drawbacks: Portal services tightly coupled to portal container and cannot being reused. Portal frameworks have proprietary services. Authorization capabilities of the portals define access rights for portlets not for contents These “capabilities” should be full-fledged Web services This architecture greatly simplifies the federation problem Some portals have useful services, but the others doesn’t support these features. December 2, 2018 Mehmet Nacar

Federating multiple portals I Requirements Browser-based login is required for portals Each site maintains its own portal server Single sign on across the portal servers Federation can be done providing: Several portals have the same code base but run different instances Integrating portals with different code bases Integrating portals with non-portal services, which have identity and authorization that is external to the portal framework. December 2, 2018 Mehmet Nacar

Federating multiple portals II December 2, 2018 Mehmet Nacar

Identity service Identity service issues security tokes for users Identity service support interfaces including: Issuing identity/security tokens Authenticating security tokens for redirection Interfaces for administrative tasks including: Adding, removing, editing users December 2, 2018 Mehmet Nacar

Federating multiple portals III This the case of federation autonomous portals. Current portal frameworks has own services. Identity service can provide browser based federation or SSO December 2, 2018 Mehmet Nacar

Integrating portals with services Mediator is a Web service Administration of Web services through portal December 2, 2018 Mehmet Nacar

How WSRP fits into this architecture? Handles Grid job management problems WSRP provides Grid Web services Federated portals are autonomous Identity service provides federation Check identity token whether it is logged in or not. December 2, 2018 Mehmet Nacar

Implications Provides standard way for portals to use external services Coordination of portal services is handled outside of portal framework More services can be added dynamically Service interfaces are extensible New features can be added to services Enables lightweight portal frameworks (minimum local capabilities) Migration of portlets and services between portal frameworks will be easier Administration of Web services through portal December 2, 2018 Mehmet Nacar

Does this architecture scale well? There will be some overhead Local capabilities vs. services In long term portal servers will be overloaded The architecture turns out to be advantageous Adaptable to for new features Simplifies migration across portal frameworks Services can be replicated Identity service could run on several locations December 2, 2018 Mehmet Nacar

Milestones Implementation of identity, logging and authorization services using WS technologies. Design of mediator service Mediator needs to coordinate services This will require the investigation of SOAP handlers and their proper pipelining. Managing SOAP header processors rather than services. Integration of reliable and secure WSRP requires two handlers WS-RM reliability handler. WS-Security handler External portal services must provide distributed identity. Getting sample data for providers in CIMA case requires: Distributed identity service Authorization service Reliability service Logging service December 2, 2018 Mehmet Nacar

Contributions of this thesis Simplification of portlet development for Grid applications through reusable libraries. Portlets are “coarse-grained” components. Tag libraries for portlets are “fine-grained” components. Design, development and validation of service-oriented portal architecture Extends the current field Limited by close coupling of portal core services to specific frameworks Our approach provides a natural way to federate multiple portal instances and to integrate portal services with other Grid services. We will examine the extremes of portal scalability in this approach. This framework will be a demanding test of Web Service standards. Requires integration of several Web services standards Integration of several standards: registration, security, reliability and others. SOAP header processing is done by intermediaries Intermediary coordination is similar to but distinct from Web Service workflow standards such as BPEL. Show telescoping portal picture. This is the logical extreme. All of these make extensive use of SOAP headers December 2, 2018 Mehmet Nacar