Better SharePoint through Service Oriented Architecture and Development DEV346, PM346

Eugene Rosenfeld, MVP, MCAD CTO, Black Blade Associates Programmer, better programmer, architect, CTO.

About this Session Intended audience Application architects Some SharePoint dev experience is helpful Some distributed application design experience is helpful Very little code, mostly Visios

Agenda EAI and SOA Traditional SharePoint Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager application to docBlock service

EAI and SOA EAI – Enterprise Application Integration What does EAI have to do with SOA? The goal of both EAI and SOA is to let systems communicate

Prime Tenet of EAI Minimal intrusion into a system The fewer changes made to a system, the fewer things break The fewer changes made to a system, the fewer things to maintain The fewer changes made to a system, the fewer things will break when the system is upgraded

Intrusiveness (low to high) Client-side artifacts Images CSS JavaScript Server-side artifacts CAML (List / site def, content types, etc) Web Parts / application pages Event Receivers / workflows Any other compiled code

Agenda EAI and SOA Traditional SharePoint Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager application to docBlock service

Sample SharePoint Application

Aspects of the Architecture Entire codebase must reside on each web front end server Web front end servers bear the bulk of the load for executing the code Most code runs in the w3wp.exe process Most code runs in the context of an HTTP request

Problems with the Architecture Entire codebase must reside on each web front end server Deployment complexity Configuration complexity Potential application compatibility issues One more thing to worry about when upgrading to next version of SharePoint

Problems with the Architecture Web front end servers bear the bulk of the load for executing the code Difficult to determine load application places on farm May require scaling out the web front ends

Problems with the Architecture Most code runs in the w3wp.exe process Recycling the w3wp.exe process causes all applications to restart

Problems with the Architecture Most code runs in the context of an HTTP request HTTP has a 60 second timeout Bad code can break the HTTP request pipeline

Agenda EAI and SOA Traditional SharePoint Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager application to docBlock service

SOA Intro SOA: Service Oriented Architecture SOA is architecture, not technology

SOA Myths Myth: SOA is new Myth: SOA needs an Enterprise Service Bus Myth: SOA requires a particular technology Myth: SOA requires Web Services Myth: SOA requires XML Myth: SOA requires text-based interface Myth: SOA requires HTTP Proof: Look at WCF in.Net 3.0

Web Service is not a SOA Service

Whats Left in SOA Definition? SOA is a design principle that abstracts a business capability from the interface used to access the capability. SOA services should: Scale up Scale out Be distributable –Have remotable interfaces –Be topology-aware

Who are the Clients? HTML Client: IE, Firefox, Safari, etc RSS Client: Outlook, NewsGator, etc SMTP Client: Outlook, Notes, Evolution, etc Terminal Client: HyperTerminal, x3270, etc Web Service Client: ? – Custom Code XML Client: ? – Custom Code

Agenda EAI and SOA Traditional SharePoint Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager application to docBlock service

Design Goals Get as much code out of the w3wp.exe process as possible Get as much code out of SharePoint web front ends as possible

Why? To Mitigate Risk Writing the application is the easy part Operational cost of the application 70% of app cost after completion, during operations and maintenance Platform updates and configuration changes Platform variations –Windows Server 2003 / 2008 –x86 / x64 Permissions

Achieving the Design Goals Separate processing into 2 categories: Requires SharePoint infrastructure Does not require SharePoint infrastructure Separate processing duration into 3 categories Fast (< 15 sec) Medium ( > 15 sec) long running ( > 60 sec)

Design Implementation

My code fits more than one box Of course it does. It should. Most non-trivial applications will use at least 2-3 boxes.

For Example… Box 1: A click in a web part causes a document property in SharePoint to change. Box 2: The property change triggers an asynchronous event receiver to call a remote web service. Box 4: The remote web service starts a long- running operation. Box 6: The long-running operation completes and publishes the results back to SharePoint. Box 1: The web part displays the new state of the data when a user revisits the page.

Re-architecting an Application Its time when… foreach(SPWeb web in currentWeb.Webs) foreach(SPList list in web.Lists) //do lots of work! Process flows are the key to re- architecture Reuse processes, not code

Agenda EAI and SOA Traditional SharePoint Application Architecture SOA Intro Rethink the Application as Services Case Study: Document Section Manager application to docBlock service

Document Section Manager App What it did Software that allowed multiple people to edit the same document at the same time How it did it Converted a physical document into a virtual document with multiple physical sections Merged section changes back into main document so main document was always current

Document Section Manager App Implementation Document section changes trigger an asynchronous event receiver Event receiver invokes a custom Windows service to process document sections Windows service invokes MS Word through PIA / DCOM to perform the section merges Another Windows service uploads the processed document back to SharePoint

DSM Process Flow

DSMs Place in SharePoint Farm

Issues with the Architecture All processing ran on each web front end Deployment was a nightmare Deploy and configure MS Word to each WFE Deploy and configure custom Windows service to each WFE System reliability was sub-optimal, mostly due to constant DCOM issues

Issues with the Architecture Application did not work when SharePoint was installed on Windows Server 2008, due to DCOM differences Application did not work when SharePoint was installed on Windows Server 2003 x64 Document processing put substantial load on WFEs

Issues with the Architecture Main issue: The Document Section Manager was too intrusive in the SharePoint system. This caused the deployment, configuration, stability, and maintenance issues. This is an architectural issue that required a fundamental redesign of the application.

Re-architecting the DSM 1 3 4, 6

Re-architecting the DSM Document Assembler Service remains mostly unchanged but is moved from SharePoint farm to external application server on external hardware. Uploader Service remains mostly unchanged but is moved from SharePoint farm to external application server on external hardware.

Re-architecting the DSM New SOAP Service interface created for Document Assembler and Uploader services on external application server. Event receiver remains mostly unchanged but now invokes the Document Assembler Service remotely rather than locally. MS Word is removed from SharePoint WFEs.

docBlocks Place in SharePoint 10%90%

docBlock Client Process Flow

docBlock Service Process Flow

Tech Benefits of Re-architecture Deployment is 80% easier 90% less code running on SharePoint WFEs MS Word no longer required on SharePoint WFEs 95% less load on SharePoint WFEs 75% increase in processing reliability

Non-Tech Benefits Easier to find developer resources: not everyone working on the app needs to be a SharePoint expert. Easier to sell the application: very little push-back from IT concerned about what gets installed on SharePoint farm. Companies requesting services work to help them similarly re-architect their applications.

Non-Tech Benefits Application can now be distributed as a virtual or physical appliance. Application can now be hosted in the cloud.

Summary If processing doesnt require SharePoint, dont run the processing in SharePoint. Minimal intrusion into SharePoint yields a more stable, reliable, performant platform. Execute slow code asynchronously with respect to the web UI. Utilizing SOA service interfaces yields more manageable, upgradable, and supportable code.

Additional Resources Document Section Manager code: Things that Should be Easy Black Blade Web Site

Thank you for attending! Please be sure to fill out your session evaluation! Eugene Rosenfeld DEV346, PM346: Better SharePoint through Service Oriented Architecture and Development