2. OBA303 Excel Services: Overview and Deployment Scenarios Adrian Rupp Business Intelligence Practice Manager iSolve Business Solutions

3. Agenda Intro: Why Excel? –What is Excel Services? –Common Use Scenarios and Business Case Excel Web Service: Technical Overview –Architecture –Performance Scalability –Security, Versioning, Auditing –Talking to SSIS –Unsupported Features and Extensibility (UDF) Finally, Using for DW purposes

4. Why Excel? Excel is prevalent in the capital markets –It provides ease of use and familiarity to traders –It provides calculation and analysis capabilities Standard tool for many tasks: –Simple data manipulation –Huge calculations involving multiple spreadsheets New in Excel 2007 –Extensive UI improvements –Support more than16,000 columns and 1m rows

5. Companies suffer from Excelitis Unfortunately, analysts push the envelope on excel to such an extent that they develop highly complicated workbooks, and end up distributing multiple copies of the same spreadsheets and the changes via email many times over. This poses obvious problems for data consolidation, audit and legal compliance.

6. Excel Services: What is it? A new service in the SharePoint platform: –Provides Server-side Excel calculation engine –Provides Browser access to live, interactive server spreadsheets –Provides Web service access to server spreadsheets and server calculations

7. Web Access Options On Demand actions available in the web access UI through the browser These actions are also available through the web service API

8. Excel Services: The Big Picture Excel 2007: Design, Author & Publish to Excel Services Excel 2007: Manual Snapshot Programmatic Access: Set value, refresh, recalculate & retrieve snapshot Internet Explorer: Zero-foot print Interactive. Set parameter, sort, filter etc.

9. Excel Services: Common Scenarios Reduce “multiple versions of the truth” – or distribution of spreadsheets that become out of sync with each other Resolve regulatory and audit concerns Incorporate spreadsheets in portals and dashboards Protect intellectual property in spreadsheets Reuse logic & business models built in Excel without re- coding in BI Applications Leverage servers to speedup calculations

10. Business Requirement #1 Field sales need to quote prices –volume pricing models and discount sizes are dependent on a number of factors: Quantity to be sold Profit margin associated with the individual product financial analysts Frequent changes: –Adding price promotion

11. Business Requirement #2 Financial analysts need to perform very large risk analysis calculations on a nightly basis based on Monte Carlo simulations: The statistical models in these calculations are very large, and use a large number of spreadsheets, with parameters –These calculations generally take several hours –Important to have current results for compliance reviews

12. Business Requirement #3 Analysts need to distribute sales and support data, using a spreadsheet, to a number of people within a company. Make a few changes to a previous model –Update from live data in a SQL Server database –approval by manager prior to distribution –Incorporate manager recommended changes –Compare the previous version / audit changes

13. Excel Web Services Summary Open a spreadsheet file –Start a session with Excel Services Set values to cells and ranges Refresh data from external sources Calculate spreadsheet or specific ranges Get results values from cells and ranges Entire calculated spreadsheet –Live or Snapshot

14. What Excel Service is not A spreadsheet authoring tool –Cannot create / edit spreadsheet in a browser. Use Excel 12. Real-time joint authoring tool –Many users can interact with a spreadsheet simultaneously, but in their own sessions. Changes by users are not saved back to the original file. Distributed calculation processing across multiple server boxes –Load balances spreadsheets across multiple boxes and calculates more than one spreadsheet on a box (parallel processing for each request). –Does not distribute a single spreadsheet across multiple boxes.

15. Application Server Excel Services: Architecture Web Front End Excel Web Access (EWA) Excel Web Services (EWS) Excel Calculation Services (ECS) User Defined Functions ECS Proxy SharePoint

16. Performance and Scalability Excel Services are optimized for multiple users accessing the same spreadsheets. –Caches at multiple levels to improve collective performance for a group of users Excel Services can scale up by adding additional CPUs or memory to the server –Multiple CPUs to handle concurrent requests –Supports 64bit CPUs. Can be architected for –High number of users –High calculation load The multi-tiered approach allows for better security and isolation of services, such as extranet scenarios.

17. Architecting Scalability Proxy is used internally to handle –Communication between the components on the web front end and the application server in multiple- server configurations –Load balancing for multiple application servers Scale up: All these components could run on the same machine (“single box”) Scale out: Components on the web front end and the application server could be on different machines Scale either or both tiers up or out independently of the other –Bottleneck in rendering spreadsheets with Excel Web Access - Add more web front ends –Bottleneck is in calculations - Add more application servers to the farm

18. Clustering / Load Balancing Clustering –A cluster enables automatic failover for calculations. Load Balancing –Network Load Balancing: Windows Server 2003, or using separate load-balancing device. –Load balancing settings can specify several methods, including round-robin, local, or hash-based. –The performance increase for adding a new server is nearly linear.

19. Security Excel Services leverages the security infrastructure provided by SharePoint –Authentication –Authorization Role based access to spreadsheet Type of access; read, write, view only etc.

20. View Permissions Excel Services can use “View Only permission” set in SharePoint –Author can mark parts of the workbook as "Viewable on the Server” –Users with View Only permissions can load, render, interact, refresh, recalculate, and take snapshots of workbooks on Excel Services –But cannot access source or contents directly If user can download the workbook, they still do not have direct access to the data source –The data connection can use SSO or server’s unattended account to refresh on the server –The user cannot refresh that connection from Excel client. Excel client does not use the unattended account or SSO to refresh connections directly.

21. Marked Not Viewable When user only has the View Item right, s/he can only see the portions (range) of the workbook that have been marked as viewable on the server during the publish process View Restrictions Source: The team blog for Microsoft Excel and Excel Services

22. Security in Snapshot When a user with only view rights opens a workbook – it opens as a snapshot A snapshot is like a copy/paste of values and formatting User can see the numbers, but none of the proprietary information behind those numbers (formulas, connections, etc.) Source: The team blog for Microsoft Excel and Excel Services Original Workbook Snapshot

23. Users and Groups Simple user management –Create Viewer Groups –Add users to Viewers group Manage Users, Groups and Permissions Source: The team blog for Microsoft Excel and Excel Services

24. Publishing Control - Who First - Control WHERE spreadsheets is stored –Maintain a list of directory paths, or “Trusted Locations.” Excel Service requires a “Trusted Location” to load a spreadsheet. –Administrator can control WHO has access to these trusted locations Designate a trusted location where selective users can save spreadsheets for Excel Services. –Guarantees authoritative content

25. Publishing Control - Process Versioning – Group Policy – Workflow Change Control Functionality –Cell-based notification –Workflow process can be based on a complex Excel calculation Guarantee that spreadsheets have gone through proper review and approval cycles. Track record: who accessed which spreadsheet and when.

26. Publishing Process: Versioning SharePoint has robust check-in/check-out and versioning mechanism. –Major and minor version numbering –Security specifically for old versions of spreadsheets SharePoint has built-in functionality for retention and expiration of documents –Old versions are automatically retained –Or destroyed according to compliance requirements. Version History Options Source: The team blog for Microsoft Excel and Excel Services

27. Publishing Process: Set Policy, Auditing SharePoint allows administrators to specify policy –Events like Open, Create, Modify, and Delete, of spreadsheets are stored in a centralized audit log Audit Entry Granularity Source: The team blog for Microsoft Excel and Excel Services Specify Publishing Policy

28. Publishing Process: Workflow, Alerts Other things to define the publishing process: –May include a range of workflows, as shown below. –Alerts may also be useful in staying on top of the publishing process

29. Publishing Process: Approval SharePoint allows an administrator to setup a document library with document approval –Spreadsheet can be required for a review before user access –A custom workflow can send emails to a group of approvers in order Requiring Approval Source: The team blog for Microsoft Excel and Excel Services

30. Facts about Snapshot Snapshot generated by Excel Calculation Service represents the current state of the workbook in the Excel Services session Published item snapshots contain only the portions of the Excel file that was marked viewable. Snapshots do not contain Excel formulas or external data connections. Web service GetWorkbook method is used to get either the entire workbook or a snapshot.

31. Easy button= GetSnapshot.EXE Pre-compiled Script Task to download the snapshot: GetSnapshot.exe [workbook_path] > [snapshot_filename] For example: C:\>GetSnapshot.exe http://MyServer/reports/Original.xlsx > D:\xlreports\SnapshotCopy.xlsx Specify the Excel Server and the Snapshot destination location

32. Connect to Excel 2007 from SSIS Data Source= D:\Test.xlsx; Provider=Microsoft.ACE.OLEDB.12.0; Extended Properties=Excel 8.0; Note: Microsoft Excel 2007 files Provider is no longer JET. Microsoft Office 2007 shipped a new OLE DB provider, Microsoft Access Engine OLE DB Provider (ACE). ACE is supported in SQL Server 2005 SP2 and later. Note on 64-bit: Both JET and ACE are 32-bit only, so WOW64 mode on IA- 64 and x86-64 is needed. (through 32-bit version of dtexec.exe). http://ssis.wik.is/Data_Sources/Microsoft_Office_(2007)

33. Unsupported: Access Data Access datasources are not supported for workbooks in Excel Services. Because this may have problems scaling and performing in an enterprise server environment. But it’s possible to add the provider to the trusted providers list and create Data Source Name (DSN) files on the server.

34. Unsupported: Linked Workbook Excel workbooks are not supported as sources for workbooks in Excel Services. Not supported in this version of Excel Services. Those workbooks will fail to load on the server. Move the data from the linked workbooks into a single workbook (Or see UDF discussed later)

35. UDF: External Workbook Range Use the Excel Web Services API to get the value of a particular cell of an external workbook instead of referencing that workbook directly. Refer the Excel Web Services API, and write a UDF to call to it to retrieve the cell values from an external workbook.

36. UDF: Consume SharePoint List Implement a UDF that can be called from an Excel 2007 workbook hosted on Excel Services to consume SharePoint Lists.

37. UDF: Query Tables Query Table on Excel Services Query tables are very useful for displaying refreshable relational data.In this version, Excel Services does not support query tables. Workaround: Data as an array formula in Excel 2007. It is simple to turn this array formula into a pseudo query table. Array of data is returned that is structurally similar to a query table, except without the sorting and filtering functionality.

38. UDF: Read/Write to SQL Ability to read/write to SQL databases enables shared manipulation of the data. Excel services doesn’t support this but it’s possible to read/write to it through UDFs. Possibility of SQL injection attacks. –For more information, see SQL Injection http://msdn2.microsoft.com/en- us/library/ms161953.aspx

39. Benefits of Excel Services for DW Encourage approved / authoritative content from business analysts Outsource business logic / calculation model maintenance to business users Scalable and Secured architecture Avoids sending spreadsheets in emails Web Service access to excel data from SSIS Can work in extranet scenarios

40. Idea: Direct Data to OLAP Easy to avoid the database engine and ETL legwork Recommended to archive and version the spreadsheets Excel Services Approved, Authorized content from analysts Data Warehouse Reference Attributes Small Partitions Large Partitions and Dimensions SSIS Dim / Prt Destination Task

41. Resources and References How to: Get an Entire Workbook or a Snapshot http://msdn2.microsoft.com/en-us/library/ms519570.aspx The team blog for Microsoft Excel and Excel Services http://blogs.msdn.com/excel/ LuisBE on Services http://blogs.msdn.com/luisbeonservices Extending the Excel Services Programmability http://msdn2.microsoft.com/en-us/library/bb267252.aspx

42. Thank you http://www.microsoft.com/southafrica/ucs/2007