Platform Upgrades As A Service Raj Nagarajan, Robert Enyedi

Why are we here?

Software Upgrades - TODAY Platform Upgrades As a Service 3 1.Read release notes 2.Make changes 3.Fingers crossed 4.Test 5.Rinse and repeat [StackOverflow] Practical approach to keeping jQuery up to date?Practical approach to keeping jQuery up to date? [jQuery] jQuery UI 1.9 Upgrade Guide%jQuery UI 1.9 Upgrade Guide

Software upgrades – different perspectives Painful – what were they thinking? Boring/vague release notes Trial-n-Error When will this bug be fixed? Developers Costly – I have no time for this Why this often? When will my team’s feature- request rollout? Managers Upgrades breaks test cases Long regression cycles Collateral damage Testers Cool new XYZ feature Sorry about that bug  Who designed this? Let us rewrite. How long should we maintain backwards compatibility? Platform Developers Fix this security issue NOW! Security This is a production P1 issue – what is the ETA for rollout? Ops Platform Upgrades As a Service 4 End Developers Platform Developers

ebay scale ~3000 deployments per month to nodes ~900 deployments per week to nodes Deployment Planned releases: 1 major, 3 minor, 20+ micro releases PER YEAR Out-of-cycle releases for production issues, including security patches. Releases developers spread across 4 continents (North America, Europe, Asia, Australia) and multiple time-zones Stack Overflow like forum based support model Adoption Thousands of production VM pools Multi-billion traffic hits / day Metrics Platform Upgrades As a Service 5

A better way: Platform Upgrade ebay

Streamlined User Interaction Platform Upgrades As a Service 7 git Telemetry Pull request Maven

Personalized and Actionable Upgrade Report Platform Upgrades As a Service 8 Early code analysis and fix Delivered as.md on Git Detect and document manual changes Outdated JDK or system overrides Misused APIs, non-optimal settings Application Health Check

Under the Hood Architecture, data, community

Repositories MavenGitHub Jenkins Application Bundles REST API Orchestration Service Architecture Platform Upgrades As a Service 10 Service Tasks Service Commands GitHubMavenJenkins Cloud External Tool PAAS App Config and Health Web UI Code 100% Java RESTEasy Platform Upgrade Emergency Patch Upgrade Module

Example use case Upgrade project from platform version 1.1 to 2.2 Upgrade Module Architecture Platform Upgrades As a Service Upgrade10to11 Upgrade11to12 Upgrade12to20 Logger upgrade Package refactoring API refactoring Upgrade20to21 Upgrade21to22 Platform versions Upgrade paths Add-ons Code 90% Scala 10% Java Eclipse JDT Chain of commands

Telemetry Driven Product Evolution Platform Upgrades As a Service 12 The journey so far: 8 months, 550+ projects, 99% adoption Full data about the submitted upgrade requests Data collection from Maven repository to facilitate move and cleanup Data Service usage, success rate, artifact usage Effective troubleshooting Traceability for post-mortems and platform improvements Statistics CI build detects usage of old platform version Runs the upgrade User receives a pull request with instructions Proactive upgrades Use the Git log to identify upgrade pain points Rely on hard data vs customer complaint to fine tune the upgrade service Full traceability

Open Source: Ostara What does it do? –The core functionality of the ebay platform upgrade solution –Suitable for but not exclusive to Maven and Spring based Java projects Development strategy –Keep generic non-ebay features in the Ostara code base –Incubation modules in both ebay and Ostara repositories –Build community & extend with more use cases Platform Upgrades As a Service 13 ebay platform upgrades Ostara ebay extensions Codehttps://github.com/eBay/ostarahttps://github.com/eBay/ostara LicenseApache Communityhttps://groups.google.com/d/forum/ostaraosshttps://groups.google.com/d/forum/ostaraoss

Future Directions Platform Upgrades As a Service 14 Create a simple upgrade path with the patch On all affected applications, apply the patch in their Git repositories, build/test/deploy the patched applications Emergency patches Use semantic analysis of code to detect types Apply all applicable refactoring patterns Allows a more fast paced API cleanup, reduces lifetime of deprecated APIs Deep API refactoring Extend the project to non-Java platforms like node.js, Scala/Akka Polyglot use cases

Q&A

Global Platforms and Infrastructure Raj Nagarajan Robert Enyedi Community