Redmond Protocols Plugfest 2016 Jinghui Zhang Office Interoperability Test Tools (Test Suites and Open Source Projects) Software Engineer Microsoft Corporation

Protocol Implementation Scenarios Client Extending or replacing a Microsoft Client (i.e. Office) Man-in-Middle Manipulation of Protocol traffic Server Extending or replacing a Microsoft Server Test Suites focus on the man-in-middle and server replacement scenarios. Client Extending or replacing a Microsoft Client (i.e. Office)

Open Specification is authored by the Product & User Assistance teams Document is tested in a clean room environment and a synthetic client is developed to simulate protocol traffic and test against a Microsoft protocol server Test Suites are deployed internally to test protocol server and supporting documentation is authored Closely working with 3 rd parties we test implementations and increase the quality of the test suites

Test Driven Quality Assurance  Ambiguity Review  Simulation of the implementation Process 1 Open Specification Review  Normative language  Requirement Extraction 2 Requirement Analysis  Requirements become Test Cases  Groups of Test Cases become Scenarios 3 Test Suite Development …, this method MUST return a SOAP fault with the specified error code.

SUT Control Adapter Test Cases TC System Under Test Test Suite Design Adapters Protocol Adapter Pluggable Different transports Scenario Based Derived from Open Specification Protocol BehaviorConfiguration Generate/Receive protocol messages Less likely to be replaced C# SUT configure/query Likely to be replaced (product specific) PowerShell, C#

Test Suite Example: Versioning Generates Protocol Traffic “synthetic client” Source code provided Customizable against 3 rd party implementations Automatable

Test Suite Components Test Cases Drives test steps and validates requirements. Traditional test cases MBT (model-based testing) test cases Protocol Adapter To create/manage connection between client and server, and generate/consume protocol messages. SUT Control Adapter To control SUT at runtime PowerShell Script Adapter Managed Source Code Adapter (C#) Interactive Adapter

SharePoint Server Protocol Test Suites ProtocolProtocol name MS-LISTSWSLists Web Service Protocol Specification MS-VERSSVersions Web Service Protocol Specification MS-SITESSSites Web Service Protocol Specification MS-DWSSDocument Workspace Web Service Protocol Specification MS-ADMINSAdministration Web Service Protocol Specification MS-MEETSMeetings Web Services Protocol Specification MS-OUTSPSLists Client Sync Protocol Specification MS-WDVMODUUOffice Document Update Utility Extensions MS-WEBSSWebs Web Service Protocol Specification MS-WWSPWorkflow Web Service Protocol Specification MS-AUTHWSAuthentication Web Service Protocol Specification MS-SHDACCWSShared Access Web Service Protocol MS-CPSWSSharePoint Claim Provider Web Service Protocol MS-WSSRESTListData Data Service Protocol MS-OFFICIALFILEOfficial File Web Service Protocol MS-COPYSCopy Web Service Protocol MS-VIEWSSViews Web Service Protocol

SharePoint File Sync and WOPI Protocol Test Suites ProtocolProtocol name MS-FSSHTTP MS-FSSHTTPB MS-FSSHTTPD File Synchronization via SOAP over HTTP Protocol Binary Requests for File Synchronization via SOAP Protocol Binary Data Format for File Synchronization via SOAP MS-WOPI Web Application Open Platform Interface Protocol

Exchange Web Service (EWS) Test Suite ProtocolProtocol Name MS-OXWSATT Attachment Handling Web Service Protocol MS-OXWSBTRF Bulk Transfer Web Service Protocol MS-OXWSCONT Contacts Web Service Protocol MS-OXWSCORE Core Items Web Service Protocol MS-OXWSFOLD Folders and Folder Permissions Web Service Protocol MS-OXWSMSG Message Types Web Service Protocol MS-OXWSMTGS Calendaring Web Service Protocol MS-OXWSSYNC Mailbox Contents Synchronization Web Service Protocol MS-OXWSTASK Tasks Web Service Protocol

Exchange ActiveSync (EAS) Test Suites ProtocolProtocol Name MS-ASAIRS Exchange ActiveSync: AirSyncBase Namespace Protocol MS-ASCAL Exchange ActiveSync: Calendar Class Protocol MS-ASWBXML Exchange ActiveSync: WAP Binary XML (WBXML) Algorithm MS-ASCNTC Exchange ActiveSync: Contact Class Protocol MS-ASCON Exchange ActiveSync: Conversations Protocol MS-ASDOC Exchange ActiveSync: Document Class Protocol MS-AS Exchange ActiveSync: Class Protocol MS-ASHTTP Exchange ActiveSync: HTTP Protocol MS-ASNOTE Exchange ActiveSync: Notes Class Protocol MS-ASPROV Exchange ActiveSync: Provisioning Protocol MS-ASRM Exchange ActiveSync: Rights Management Protocol MS-ASTASK Exchange ActiveSync: Tasks Class Protocol MS-ASCMD Exchange ActiveSync: Command Reference Protocol

Exchange MAPIHTTP Protocol Test Suites ProtocolProtocol Name MS-OXCFOLD Folder Object Protocol Specification MS-OXCFXICS Bulk Data Transfer Protocol Specification MS-OXCMAPIHTTP Messaging Application Programming Interface (MAPI) Extensions for HTTP MS-OXCMSG Message and Attachment Object Protocol Specification MS-OXCNOTIF Core Notification Protocol MS-OXCPERM Exchange Access and Operation Permissions Protocol Specification MS-OXCPRPT Property and Stream Object Protocol Specification MS-OXCROPS Remote Operations (ROP) List and Encoding Protocol Specification MS-OXCRPC Wire Format Protocol Specification MS-OXCSTOR Store Object Protocol Specification MS-OXCTABL Table Object Protocol Specification MS-OXNSPI Exchange Server Name Service Provider Interface (NSPI) Protocol MS-OXORULE Rules Protocol Specification

Test Environment Topology Test Suite Network Topology Machine name SharePoint Test Suite Environment Exchange Test Suite Environment SUT * Microsoft SharePoint Server 2013/2010/2007 Microsoft Exchange Server 2015/2013/2010/2007 CLIENT Microsoft Visual Studio 2013 Ultimate Protocol Test Framework Microsoft Visual Studio 2013 Ultimate Protocol Test Framework Optional Microsoft Network Monitor and Parsers OR Message Analyzer Spec Explorer 2010 Microsoft Network Monitor and Parsers OR Message Analyzer *3 rd party implementation

Using Visual Studio Open Visual Studio Test Suite Solution and rebuild the solution Open the Test Explorer Select test cases, right click and select the option to Run/Debug them Using the included batch files All tests in all protocols, all tests within one protocol, or a single test case Using command line Use mstest.exe Running the Test Suite