UtiliZing VSTS for dynamics AX Brian Hof, Marshalltown Company | Bernd Loske, Edgewater Fullscope

Agenda Development Challenges Source Control Build and Deployment Application Lifecycle Management

Development Challenges

Development Challenges Artifact Id Conflicts Development Conflicts Development Best Practices Compile Errors Code Delivery Revisioning Housekeeping Task Management Inadequate Information

Development Challenges \Classes\FSCustom1 method1() method2() \Classes\FSCustom2 method1() method2() \Classes\SalesFormLetter standard1() \Classes\FSCustom1 method1() method2() \Classes\FSCustom3 method1() method2() \Classes\SalesFormLetter standard1() \Classes\SalesFormLetter standard1() \Classes\FSCustom3 method1() method2() \Classes\SalesFormLetter standard1() \Classes\FSCustom2 method1() method2()

Source Control with VSTS

Source Control General Configure DAX for VSTS Using VSTS Multiple Developer Machines

Source Control - General VSTS is file based Each artifact is exported to .xpo Synchronizations and reverts imports .xpo VSTS Code Repository Archive & Tracing

Source Control - Configuration Developer Configures Developer Machine Create workspace / repository Configure Version Control Parameters Configure TFS Parameters Synchronize Check In

Source Control – Using VSTS Developer Synchronizes with VSTS Synchronize Add to version control Check out Validates against other developers Once we are done – Check in will update the repository. History Rollback

Source Control – Multiple Developers Multiple developer boxes Developers can not modify artifacts that are not checked out. Developers can check out artifacts that are already checked out. Last developer to Check-in has to do the merge.

Source Control USR VSTS worries only about the layer you are in. CUS VAR ISV SYS /SYP

Source Control DEMO

Build & Deployment

Build & Deployment Why automate? Consistency Reliability Collaboration Limit build environment Same steps, same hardware, same software every time Reliability Limit time between builds Frequent build enables frequent verification Collaboration Limit changes between builds Why did it break? Who can fix it? Result: Agility I build (and verify), therefore I release

Build & Deployment How? PowerShell scripts Team Foundation Build Change Management and TFS Integration for Multi-Developer Projects Team Foundation Build Pioneered by Joris de Gruyter Open-source libraries and templates for AX builds CodeCrib.AX.TFS Caveat: TFS 2015 has been released Requires AX 2012 R2/R3 CU9 TFBuild has been overhauled (legacy XAML builds still supported)

Build & Deployment – TFBuild Windows Workflow Foundation (WF) Workflow resembles a flowchart Accepts input parameters Declares (scoped) variables Steps are Activity objects System.Activities.CodeActivity Accepts input arguments System.Activities.InArgument<T> Performs logic Execute() override Returns output arguments System.Activities.OutArgument<T> Visual Studio WYSIWYG editor Stored as .XAML

BUILD & DEPLOYMENT – TFBUILD Concepts: Build process template WF workflow implementing a build process Build definition Configures a specific build in a TFS team project Triggers, workspace, process template, parameters, retention policy, etc Build controller Initiates and monitors builds Build agent Executes builds (selected by controller using tags)

Build & Deployment AX + VSTS ecosystem: Isolated developer systems AOS instance AX client Visual Studio SQL Server (may be shared) Shared version control (TFS) Model code (XPO/ALD files) Team Foundation Build Controller(s) Agent(s) Staging environment(s) Test, Sandbox, Production…

Build & Deployment Goals: Manage complexity (K.I.S.S.) Automate! Single layer customizations Single model customizations TFS issues with AOT elements customized across multiple models Test and develop against actual business data Refreshed often from production My goals, as defined by my work requirements/environment, which will inform the remaining slides. Our production DB ~150 GB

Deep Dive Build & Deployment Suit up, and let’s descend into madness! This will set the stage for how to achieve a solution that meets the goals.

Build & Deployment – Deep Dive AX 2012 Database: Two SQL databases Business data Model data Element IDs Every AOT element has an ID Class, method, table, field, etc Generated when first added Manual creation XPO import Model file import Tie model & business data together Data cannot be haphazardly interchanged! Security objects are one type of element likely to cause noticeable issues if databases become mismatched!

Build & Deployment – Deep Dive Scenario: Alice and Xander begin with same data… Both modify the model (AOT)… Both synchronize the data dictionary… They can no longer swap business databases! OK…but how can they exchange model data? Adds: A1 [ID:1] A2 [ID:2] Deletes: D1 X1 [ID:1] X2 [ID:2] X3 [ID:3]

Build & Deployment – Deep Dive Exchange XPOs? Va′ = A1 [1], A2 [2], X1 [4], X2 [5], X3 [3] Vx′ = A1 [4], A2 [5], X1 [1], X2 [2], X3 [3], D1 Exchange model files? Va′ = X1 [?], X2 [?], X3 [?], D1 Vx′ = A1 [?], A2 [?] Added: A1 [ID:1] A2 [ID:2] Deleted: D1 X1 [ID:1] X2 [ID:2] X3 [ID:3] Synchronize with TFS? Va′ = A1 [1], A2 [2], X1 [4], X2 [5], X3 [3] Vx′ = A1 [4], A2 [5], X1 [1], X2 [2], X3 [3] TFS, then exchange model store file? Alice creates model store file, Xander imports  Same model & IDs  Xander’s model & business data are mismatched XPO – Xander retains D1 (not deleted by XPO import); mismatched IDs Model file – Alice and Xander effectively swap models (element IDs unknown) TFS – Same models; mismatched IDs

Build & Deployment – Deep Dive Solution: Developers synchronize to TFS Element IDs won’t match Doesn’t matter, if they don’t need to share business data Ensure builds remain compatible with deployment targets: 1) Maintain common business data origin between systems 3) Deploy model data exclusively from build system 2) Maintain element ID integrity between builds Build against Temporary DB and Primary DB (next slide)

Build & Deployment – Deep Dive Temporary DB: AOT is reset for each build Uninstall all models before importing XPOs Ensures deleted elements are removed Export temporary model file Element IDs not important Primary DB: AOT must never be reset Code changes must be accumulative Install temporary model file over existing AOT Deleted elements are removed Pre-existing elements & their IDs are preserved New elements are added with new auto-gen IDs Compile updated AOT Export model store file Deploy to any matched target (same data origin) Referring back to my goals: When refreshing a staging environment from production, the build environment’s Primary DB must also be refreshed (and vice versa). When refreshing a developer system from production, simply sync from TFS afterward.

Build & Deployment – Deep Dive Compile via AX client: Slower (single-threaded) Easy to detect success Reliable Compile via AXBuild.exe: Faster (multi-threaded) Difficult to detect success Log: AxCompileAll.html Part actual errors, part false alarms Complicated retry loop Errors in log? Import log into AX client Recompile problem objects Repeat N times Refer to: Blog Post #1 Blog Post #2 Provides code to automate retry process Unreliable Users have observed successful builds which do not actually work* Comparison of the two methods for compiling the AOT. My choice: AX client. *The number of worker threads used by AXBuild.exe may be important to ensure success. General practice: let it decide how many to use.

Build & Deployment Resurface (Example) Now we must learn to swim!

Build & Deployment – Example Build phase: Generate model file Uncompiled AOT .axmodel Generate model store file Compiled AOT .axmodelstore Deployment phase: Restore database (optional) Import model store file Post-deployment tasks Model file only used internally.

Build Phase – Example Model: Model store: Combine XPOs from TFS Using Temporary DB Scrub files* Uninstall all models Synchronize data dictionary Import labels (ALDs from TFS) Import combined XPO Export temporary model file Model store: Using Primary DB Install temporary model file Scrub files* Deploy .NET references Import labels (ALDs from TFS) Import VS projects (from TFS) Compile Full CIL generation Synchronize data dictionary Export model store file *Scrub files – Delete XppIL, cache, VSAssemblies, etc Repeatedly scrub files to avoid various problems! When exporting the model store, you may also export the primary model as an additional build artifact.

Deployment Phase – Example Target system(s): Restore database (optional) Restore SQL backup Business database Model database Post-restore SQL script Scrub files Import model store file Temporary schema Initialize Import model store Apply* Drop Synchronize data dictionary Post-deployment tasks e.g. class MT_PostDeployTasks Notes: *AOS down while applying temporary schema Multiple targets may be specified Supports independent execution (without build) Deploys last successful build Enables progressive deployment pending QA

DEMO Build & Deployment Show & tell: build definition, process template, activity library, build results (VS here; web later during ALM?)

Build & Deployment - References Environment considerations and Microsoft-recommended practices: Models, Layers, and the Model Store [AX 2012] Change Management and TFS Integration for Multi-Developer Projects (white paper) Build and deploy scripts for Microsoft Dynamics AX 2012 (PowerShell) Deploying customizations across Microsoft Dynamics AX 2012 environments (white paper) Building with TFBuild (materials by Joris de Gruyter): How We Manage Development - Automated Builds Easy Automated Builds: Part 1 Part 2 Part 3 Sample build templates CodeCrib.AX.TFS (custom activity library) Class documentation CodePlex open source repository

Application Lifecycle Management

Application Lifecycle Management Requirements Management Task Management Planning Metrics Reporting According to repeatable proven practices

Application Lifecycle Management Centered around Work Items Requirement Risk Issue Task Bug Any entity defined by process template. Attachments, Links

Application Lifecycle Management Accessing VSTS ALM Visual Studio (Team Explorer) TFS Web Access MS Excel MS Project Power BI

Application Lifecycle Management Alerts Dashboards Backlogs Queries API

Source Control DEMO

Final reminders Complete Surveys USE VSTS !

Speaker contact info Brian Hof MARSHALLTOWN brianh@marshalltown.com

Speaker contact info Bernd Loske Edgewater Fullscope FOCUS2016@fullscope.com