1. uPortal-Sakai integration
JA-SIG Winter Austin
uPortal-Sakai integration presentation given by Andrew Petro at JA-SIG Winter Austin.

2. What is “integrated”? SSO between uPortal and Sakai?
Shared provisioning?
Sakai rendered using uPortal?
Sakai provides markup?
Portal provides markup, Sakai provides content?
Shared codebase?
What do we mean by integration between uPortal and Sakai? Do we mean mere single sign on between uPortal and Sakai? Do we mean shared provisioning such that the Sakai users are the uPortal users, the Sakai groups are the uPortal groups? Sakai’s understanding of what content to display to a user informs uPortal’s selection of content?
Do we mean a user experience much like the current Sakai rendered via uPortal? What produces the markup? Does Sakai own the markup generation or does Sakai just provide more abstract content that uPortal provides appropriate views upon?
Do we mean a shared codebase, using Sakai code to implement widgets (JSR-168 portlets, channels) that are not formally Sakai tools, do not run “in Sakai”, but rather run in a portal?
Well, of course we mean all these things. But before we set out to integrate Sakai and uPortal in general, before you set out to integrate your Sakai and uPortal in particular, we need to know what the goal is, what integration will mean.

3. Why integrate? Portal as aggregator?
Portal as one content delivery mechanism to rule them all…
(Layout management)
Provisioning efficiencies?
Why do we want to integrate uPortal and Sakai? Is it because we want our portal to aggregate other services? A digital dashboard, a web menu, one place to go from which you can find all other places and get a hint about whether you actually want to go to those other places right now? That’s one use of a portal, to ease navigation of your campus web of sites and services.
Or do you want one content delivery system to rule them all, one container for your services and widgets? You might want this to re-use your uPortal look and feel. To use its powerful layout management and customization abilities. Use uPortal to deliver Sakai-based widgets because of the value-add of being in the uPortal container.
Or do you want to realize some provisioning efficiencies? How many systems do we have to provision with student membership in course groups, with what widgets support particular courses? Would integration by bridges between uPortal and Sakai APIs allow provisioning efficiencies of provisioning one and the other also being provisioned?

4. Portal as aggregator uPortal Summarization, Syndication, Navigation
Sakai
Moodle
Legacy Homegrown LMS
One reason to integrate uPortal and Sakai is to take advantage of uPortal as an aggregator. This is what a portal does best. You have many sources of data and services. Sakai is one source. A goal of integrating Sakai content into uPortal would be for summarization and syndication of Sakai information. Portal as a dashboard view on Sakai. In implementing this you’d take advantage of the navigation uPortal has to offer in terms of being able to expand / minimize individual channels, focus in upon them, organize them into tabs, etc.
Webmail
Announcements
Summarization, Syndication, Navigation

5. One portal to rule them all…
Single look and feel
Layout management and navigation
Consider the Hypercontent counterexample
Or the goal can be more ambitious. You might desire that your uPortal not just provide a dashboard summary view over your Sakai. You might want it to be the actual framework wherein your Sakai is presented, a full face to Sakai. In going this route you would be trying to take advantage of portal efficiencies in terms of having a single look and feel, portal layout management and navigation to organize and present the pieces of Sakai alongside other pieces of your institutional information and online service presence.
In thinking about this route, we should bear in mind that Hypercontent went the other direction. Hypercontent is a content management system that in its 1.x versions was delivered in the formfactor of a uPortal channel. For 2.x it has been re-architected to be a standalone web application. The pattern seems to be that as applications get “big” enough, they graduate out from being portal widgets to be full-fledged web applications. Sakai is considerably bigger than most applications that are typically hosted entirely within a portal.

6. Provisioning efficiencies
Custom plugin re-use across systems
Group membership
Driving uPortal AuthZ, channel availability, fragment pushing off of Sakai groups
Relatedly, you might look to integrate to realize provisioning efficiencies. How many times do you really need to write the plugin, the process, that provides or populates with your institutional data? An efficiency of being in a portal is that components running in the portal can share an authentication, authorization, provisioning infrastructure. Or, in the other direction, your Sakai instance knows a lot of interesting information about which students are in which courses, what courses and more ad-hoc site groups exist in your university, and potentially all of this information can usefully be exposed. uPortal authorization decisions, such as who can subscribe to what channel, are driven by group membership. Also, what content by default goes to which uses is also driven off of group membership. So to predicate uPortal authorization or default content upon Sakai group membership, we need a Sakai-backed implementation of the uPortal groups APIs.

7. Kinds of integration Sakai instance as service provider Provisioning
Sakai as service library
The previous discussion was looking at this in terms of what integration can accomplish. Now I’d like to consider integration in terms of implementation – kinds of integration. There are three main kinds of integration. There’s one kind of integration that involves continuing to run a standalone Sakai instance and considering that web application as a provider of services consumed within the portal. Sakai as service provider. Secondly, there’s integration of backing provisioning that I just mentioned in terms of what it can accomplish and will mention again in terms of implementation. Thirdly, there’s Sakai as a service libarary – integration by re-use of Sakai service implementations and Sakai code.

8. Sakai as Service Provider
WSRP: provide services and markup
Web Services: provide just services, portlet provides markup.
Sakai as a service provider comes in two flavors, depending upon what is producing the markup. Under WSRP, Sakai provides both the services and the markup and the abstraction Sakai provides is that of remote portlets. Under a more general Web Services approach, Sakai provides services (an assignments service, a group membership service, an announcements service, etc.) and custom code in a portal widget (a channel or portlet) consumes these services and renders them, generating the markup in the portal.

9. Sakai as WSRP producer Vishal Goenka’s excellent work
Sakai produces WSRP for
Tools in the abstract
Specific placements (tool-in-context)
Vishal Goenka has contributed almost all of the progress to date on Sakai as a WSRP producer. Sakai produces WSRP for tools in the abstract, which has been an important proof of concept but isn’t usually useful in practice. Sakai also produces WSRP for specific placements of tools (tool-in-context). This is tremendously interesting and compelling WSRP to consume, since it results in the remote portlet rendering content relevant to some particular placement of the tool (that is, for some particular course or other Sakai site). Announcements for a specific course, rather than just the announcements tool in the abstract.

10. How this works
Select a tool and identify its id key (“sakai-announcements”, e.g.)
Use the tool id as the portlet handle
Unblock Sakai WSRP for your portal
Point uPortal WSRP Sakai WSRP producer
How this really works well is if you read Vishal’s PDF guide to WSRP production in Sakai. Here’s the short version. Pick out a tool you want to consume and identify its key, which is included an an XML registration file in the tools directory of its webapp. This tool id will be the “portlet handle”. Unblock Sakai WSRP for your portal – out of the box Sakai restricts what consumer IP addresses it will service. Point the uPortal WSRP consumer at the Sakai WSRP producer by configuring your WSRP consumer with the appropriate URLs.

11. Tool identifiers in webapp\tools\*.xml
<registration>
<tool
id="sakai.announcements"
title="Announcements" …
</tool>
</registration>
Here’s the typical example of a Sakai tool identifier.

12. Configuring WSRP consumer
And here’s an example of configuring the WSRP consumer in uPortal to consume that announcements tool in the abstract.

13. This is what it looks like when it renders
This is what it looks like when it renders. Since the tool isn’t in the context of any site, there are no announcements. So it demonstrates getting markup from Sakai via WSRP, but probably isn’t a portlet you’d want to run in production.

14. Authenticating consumer to provider
By remote address of the consumer
By HTTP_BASIC authentication (over a secure channel) – no built in support for doing this in uP WSRP consumer
WSRP asserts the authenticated username from portal to Sakai, so it’s important that only trusted consumers be allowed to access Sakai via WSRP. Sakai authenticates its WSRP consumers by means of the remote address they appear to be using (with a pretty restricted set of allowed IP addresses configured out of the box) and optional HTTP_BASIC authentication turned off out of the box. Over a secure channel, the basic authentication approach allows a shared secret to authenticate consumer to Sakai. However, there’s no built in support for doing this type of authentication for WSRP in uPortal (or in the underlying wsrp4j libraries uPortal uses to consume WSRP).
Once basic authentication were in use, more advanced approaches such as using a proxy CAS ticket (and using that to convey the authenticated username rather than relying upon portal assertion) should be investigated.

15. How this works in context
Obtain a tool’s placement ID
Tool in context of a site
Use the placement id as the portlet handle
How this works for a tool in the context of a particular site is also best explained by Vishal’s PDF. However, here’s the short version: you need a tool’s placement ID, which identifies the tool in the context of a site. You use this placement id as the portlet handle. Everything else remains the same.

16. Placement identifiers
<iframe name="Maincdf9fd58x398bx45d7x808fx76381f9c736c" id="Maincdf9fd58x398bx45d7x808fx76381f9c736c" title="Schedule Content" … src=" </iframe>
One way to get a placement ID is to examine the IFrames that Sakai produces. Highlighted here is the placement ID for a particular instance of the schedule tool.

17. Wrinkle Tools-in-context (with placement ids) are very numerous
So not reported as available portlets via WSRP – consumer has to know they are there and request them despite their not being advertised
Current WSRP consumers balk at attempting to consume unadvertised
However, there’s a wrinkle. Tools-in-context (these placement IDs) are very numerous. So they are not reported as available remote portlets via WSRP – consumer has to know they are there and request them despite their not being advertised. Current WSRP consumers in uPortal balk at attempting to consume unadvertised portlets. So Sakai is now producing this compelling WSRP, but uPortal is not yet able to consume it.

18. Issues here
No existing uP releases cope with Sakai’s requirement of out of band provisioning of portlet
uPortal 2.5.x WSRP consumer doesn’t work at all
(Consuming the WSRP nicely demonstrated in uP3)
wsrp4j as incubated work in progress makes fixing this difficult
No existing uPortal releases cope with Sakai’s requirement of supporting portlets not advertised by the WSRP producer. There has been issues getting the uPortal 2.5.x WSRP consumer to work at all (though it has also been demostrated to work since these slides were produced, so the statement in the slides above is probably way to harsh. There’s been ambiguity about whether it works. WSRP4j as an incubated work in progress without formal releases makes building and maintaining WSRP consumption support more difficult than it otherwise would be.

19. It’s time to fix and enhance uP WSRP consumption
Sakai produces compelling WSRP
So let’s consume it
Reasons to be positive and expect progress here
WSRP4j opportunities
As of Sakai 2.1.0, Sakai is exposing some compelling WSRP that one might actually want to consume. So let’s consume it. This requires enhancements to uPortal WSRP consumption support. There are reasons to be positive and expect progress here – now that the feature will buy something, people will step forward to work on it. Cf. fixing JSR-168 support bugs. There are further opportunities to contribute to the WSRP4j project itself and to innovate in, for example, how the consumer authenticates to the producer.

20. Co-developing WSRP consumption with Sakai WSRP production
Sakai’s HTTP Basic authentication
Eventually other authentication mechanisms (proxy CAS)
uPortal needs to “catch up” with Sakai here.
The uPortal WSRP consumer needs to co-develop with the Sakai WSRP producer in order to validate and begin taking advantage of features as they are introduced. Back in uPortal 2.4.1, uPortal WSRP consumption was, arguably, ahead of Sakai WSRP production. Now Sakai, with Vishal’s efforts, has caught up to and surpassed the uPortal WSRP consumer capabilities. Sakai now optionally supports HTTP_BASIC authentication of the consumer to the producer. So this feature needs to be added to the uPortal WSRP consumer, ideally in a way that shares an implementation with other portals using wsrp4j to build WSRP consumption support. Once that is working, we should add other, stronger authentication mechanisms, including proxy CAS.

21. Sakai instance as service provider
WSRP gives Sakai control over the service and the markup
But a custom, portal-appropriate view may be more effective
A custom JSR-168 (or IChannel) can provide this view on Sakai
Enabled by Sakai’s web services
And ease of exposing more such services
That was WSRP, the Sakai instance being responsible for both the backing services and the portlet abstraction of views on those services, all the way down to producing markup.
But a custom, portal-specific, portal-appropriate view on a service, rather than a generic Sakai-produced portlet abstraction, might be more effective for some uses. A custom JSR-168 (or IChanneL) can provide this view on Sakai. This is enabled by Sakai’s web services and the ease of exposing more such services.

22. Sakai Portlet
Dr. Chuck Severance’s Sakai JSR-168 portlet demonstates this “portlet consumes Sakai web services” approach
Dr. Chuck has produced an excellent example of a JSR-168 consuming Sakai web services to render Sakai content, and since this slide was produced has released an improved second version of this portlet.

23. Sakai syndicated content
Sakai tools exposing RSS feeds and the like
Allow users to roll their own aggregation
Sakai as a service-providing web application instance could also expose RSS feeds. Many tools could have appropriate RSS views, say latest announcements, latest updated assignments, upcoming scheduled events, and the like. Authentication could be accomplished by large revocable globally unique within a Sakai instance random number keys. Exposing such RSS feeds would allow users to roll their own aggregation in their RSS consumer of choice.
This is particularly interesting in light of the wave of RSS / Atom integration into web browsers, into clients, and into the “portals” that our constituencies are already using for other things – the Google dashboard site or the Yahoo portal. A student might very well want to see her upcoming assignments on her Yahoo portal page, and Sakai exposing bookmarkable RSS feeds is a way to deliver that feature and put the end users in charge of where they consume their Sakai data.

24. XML feeds out of Sakai Poor man’s web services
A lot of mileage out of this for specific use cases
The differences here are really about what abstraction of web services Sakai provides. Under WSRP, Sakai exposes a portlet abstraction. Under soap web services, Sakai exposes services, abstractions on the Sakai learning and collaboration domain functionality, using soap. Under RSS, Sakai exposes a summary view on some content in a very generic, and hence re-usable but weakly typed, way.
A final option I’d like to mention is Sakai producing XML feeds. uPortal is really great at rendering XML. We do this all the time. Not just RSS feeds, but other XML as well. There are excellent CWebProxy and CGenericXSLT channels, and many channel skins are implemented as XSL transforms.
For specific use cases involving low-interactivity dashboard widget views on Sakai (upcoming events?), a developer looking to accomplish a specific integration could get a lot of mileage out of this simple approach.

25. Provisioning Groups and permissions Layout Available channels
Okay, so we’ve covered Sakai as a web application instance provided services. Now I’d like to re-visit this shared provisioning concept and drill down into what implementations could look like. Provisioning is mostly about Groups and Permissions, Layout, and the available channels for any particular user.

26. Sakai groups as uP GAPs GAPs becomes abstract API with its own jars
Sakai groups store implementation
Sakai group information then available in uPortal
Not just groups of users
Groups of channels / portlets
Dan Ellentuck has been working on abstracting uPortal Groups and Permissions out into its own seperable API, its own .jars. This is helpful because it defines what the APIs are and provides an API against which to build Sakai components without dragging in all of uPortal. So what is needed is a Sakai-backed uPortal Groups store implementation. This might be backed by web services calls out to Sakai, or it might be backed by instances of Sakai services running inside the uPortal JVM against locally configured Sakai service instances that ultimately use the same backing data as your Sakai instance. Either way, the goal is to produce a group store implementation that makes Sakai group information available in uPortal.
Groups of users: Sakai is in possession of interesting data about which users regard themselves and/or are regarded by their professors as participants, in particular roles, in particular classes. Or in ad-hoc worksites that aren’t classes and may be unknown to your SIS or other data that could otherwise be backing uPortal groups configuration. Sakai is the authority on certain group memberships.
And groups of channels / portlets.

27. Externally defined channels
Available channels are defined in terms of Groups and Permissions
Implement a source of channels backed by Sakai containing channels that present Sakai content
Remember a few slides back when we talked about WSRP and the more-interesting WSRP of a tool in context? Well, those placement ID keys create a problem, namely, how to provision your uPortal instance to be aware of and take advantage of those keys. It turns out that uPortal models the available channels as a Group of channels, for a lot of good reasons, particularly taking advantage of the powerful and general permissions functionality. So the available channels you see are those Channels in the nested groups (categories) of channels that you have Permission to subscribe. A Sakai-backed Groups store implementation would represent the available Sakai WSRP portlets as instances of the uPortal WSRP consumer portlet provisioned with appropriate URLs and keys, and with permissions on those channels controlling who is allowed to see and subscribe to what.
This applies more generally that just configuring WSRP portlet instances. A Sakai-backed channel Group implementation could also represent RSS readers configured to consume Sakai RSS, and custom portlets configured to consume Sakai web services. The Sakai-backed part of the story comes in in using Sakai to determine what content is available to which users.

28. Externally defined layout fragments
Use GAPS and DLM, ALM, etc. to select content based on Sakai groups
So using Groups and Permissions, we can make Sakai content appropriately subscribable by end users of uPortal, and they can more or less roll their own aggregation, selecting content and placing it into custom tabs. Pretty cool.
But often it will be appropriate to provide plausible default layout components (fragments, which almost always look like tabs) rendering Sakai content. So we’re not done yet.
Note that we’re building a stack here. First we talked about how to get the content to render at all in uPortal (WSRP, RSS, custom portlets). Then we talked about how to make that content available for appropriate subscription without end users typing magic keys. Now I’d like to talk about how to automatically subscribe some of this content.
Advanced uPortal layout provisioning is currently driven by group membership. Group membership drives selection of template layout (under simple layouts) or selection of individual fragments to include by default (under ALM or DLM). Making Sakai groups of users available in uPortal would allow selecting among fragments based on Sakai group membership, which would work well if you produced a dashboard of weather widgets, say, on a fragment and wanted to make that available to members of a Meteorology course.
It seems to me that more powerful features are needed to drive really compelling uPortal layout provisioning of channels pointed at Sakai content.
uPortal layout management needs a powerful new feature of externally defined fragments. This would involve not just selecting among predefined fragments, but consulting Java code at runtime to spin new layout fragments. A Sakai-backed layout management component would call out to Sakai to find out which sites a user ought to get, and what tools make up those sites, and from this compose a description of fragments – tabs and columns and configured channels in the form of WSRP consumers, RSS readers and/or custom portlets.
And incidentally, this is a powerful and general feature. It advances “portal as application container” offering aggregation and navigation services. With externally defined fragments in place as a core feature of uPortal, applications can be developed as clusters of related portlets intended to be rendered together and an external system can drive selection and configuration of those portlets appropriate to a user. A potentially compelling, powerful, and general uPortal feature.
But the upshot for Sakai would be an ability for users to log into uPortal and get a portal-appropriate rendition of what they currently get when they log into Sakai, rendered alongside traditional portal content, with a single look and feel and navigation paradigm. Maximal integration, with the Sakai components really a first-class part of uPortal, with Sakai channels really provisioned, subscribable, Permissible channels, with Sakai layout fragments real layout fragments integrated into the layout through layout management.

29. Why provisioning integration is important
With a working WSRP consumer, there’s still the matter of obtaining and using the right tool placement ids for the right users.
To recap: there’s an integration stack here. Rendering the content at all. Making that renderable content subscrible in uPortal. And actually pushing the right components in the right arrangements to the right users. Provisioning is important for using uPortal in a way that reuses and leverages its portal features to best effect.

30. Sakai as service library
Finally, I’d like to talk about a basically totally different idea, that of Sakai as a service library.

31. MVC: re-using the model
Sakai models and provides service APIs and implementations for learning and collaboration domain
Chat service, discussion service, scheduler service…
Sakai is pretty modular in the way it is architected and implemented, and aspires to be even more modular as it continues to develop. Specifically, Sakai uses a Maven build process which supports its aggressively separating out service APIs and implementations for the learning and collaboration domain from code for particular tools and web-gui components. Sakai is about services with well-defined APIs and backing implementations of those APIs.

32. JSR-168 on these APIs
Example: a personal scheduler portlet built around the Sakai scheduler implementation
Minimize code duplication
So it should be feasible to write JSR-168 portlets built around these backing services. For example, you might decide you need a personal scheduler JSR-168 portlet. A way to implement such a portlet might be to build it around the Sakai scheduling service APIs. This would minimize code duplication and take advanatage of Sakai’s existing work in this domain.

33. Pieces of Sakai, running in JSR-168 portlets under uPortal
Mix and match tools
Include alongside portlets, channels developed in other ways
Before we were talking about consuming services exposed by a running Sakai instance. Now we’re talking about taking pieces of the Sakai implementation and running them inside JSR-168 portlets inside uPortal to provide pointed bits of functionality. Would it make sense to re-implement all of Sakai as JSR-168s so you can run them inside uPortal? That sounds like a huge amount of work for little benefit. But it might very well make sense in the case where you want to mix and match just a bit of what Sakai has to offer and offer it through a portal.

34. JA-SIG: climb the value stack?
We’ve had excellent collaboration on building a portal framework
And excellent collaboration on channel projects
As JA-SIG members look to do this more, and look to do this in Learning and Collaboration domain, be aware of Sakai
More generally, I think the thoughts recently expressed on the list that JA-SIG might fruitfully “climb the value stack” are worth strong consideration. We’ve had excellent collaboration on building a portal framework. And excellent collaboration on channel projects. But most of our channel projects have been general tools – web proxies, RSS readers, XML transformers, and the like. With exceptions, to be sure.
But as JA-SIG members look to collaborate more on portlet development, as we look to develop not just a portal but shared portlets for higher education to plug into our portal, some of these portlets are going to be in the domain of learning and collaboration. Because that’s what higher ed does, after all. And whenever any of us seeks to write a portlet that touches the learning and collaboration domain, it’s probably worth taking a look at what Sakai has already done in that domain and see what can be re-used.

35. How to get involved / contribute
I’ve talked in this presentation about what advantages integration can provide, what kinds of integration might be done, in general terms how that integration can be implemented. I’ve tried to cover the spectrum of what is meant by uPortal Sakai integration. I hope you came to this presentation already interested in integration, and now are more interested, that you’ve seen something and said “I want that.” Great. What was it?

36. There are too many options
There are too many options here, it’s hard to know what to focus on, how far to take what when
Input please: what is needed most urgently, who plans to do what
There are too many options here. We’ve got this really very flexible portal, and this flexible treasure-trove of content. It’s hard to know what to focus on, how far to take what when.
So, input please: what is needed most urgently? Who plans to do what? Which of these integrations, if actually available, would you put into production?

37. Sakai Portal DG Portal discussion group
There are places to discuss and work on this. Sakai offers a Portal discussion group.

38. Sakai Integration DG Integration discussion group
Collects integration use cases
And Sakai offers a more general Integration discussion group. I’d like to mention especially that this group collects integration use cases, and if you’ve a use case or requirement for uP-Sakai integration you can share, even if you’re not prepared to work on implementing it, contributing these use cases really helps to make integration more concrete.

39. Birds of a Feather
Further collect integration scenarios, requirements, desires, plans