Embed Size (px)
Citation preview
Platform Upgrades As A ServiceRaj Nagarajan, Robert Enyedi
Why are we here?
Platform Upgrades As a Service 3
Software Upgrades - TODAY
1. Read release notes
2. Make changes
3. Fingers crossed
4. Test
5. Rinse and repeat
[StackOverflow] Practical approach to keeping jQuery up to date?[jQuery] jQuery UI 1.9 Upgrade Guide%
Platform Upgrades As a Service 4
Software upgrades – different perspectives
• Painful – what were they thinking?• Boring/vague release notes• Trial-n-Error• When will this bug be fixed?
Developers
• Costly – I have no time for this• Why this often?• When will my team’s feature-
request rollout?
Managers
• Upgrades breaks test cases• Long regression cycles• Collateral damage
Testers
•Cool new XYZ feature•Sorry about that bug •Who designed this? Let us rewrite.•How long should we maintain backwards compatibility?
Platform Developers
•Fix this security issue NOW!Security
•This is a production P1 issue – what is the ETA for rollout?Ops
End Developers Platform Developers
Platform Upgrades As a Service 5
Agility @ ebay scale
• ~3000 deployments per month to 87000 nodes• ~900 deployments per week to 26000 nodesDeployment
• Planned releases: 1 major, 3 minor, 20+ micro releases PER YEAR• Out-of-cycle releases for production issues, including security patches.Releases
• 1500+ developers spread across 4 continents (North America, Europe, Asia, Australia) and multiple time-zones
• Stack Overflow like forum based support modelAdoption
• Thousands of production VM pools• Multi-billion traffic hits / dayMetrics
A better way: Platform Upgrade Experience @ ebay
Platform Upgrades As a Service 7
Streamlined User Interaction
git
TelemetryPull request
Maven
Platform Upgrades As a Service 8
Personalized and Actionable Upgrade Report
Early code analysis and fixDelivered as .md on Git
Detect and document manual changes
Outdated JDK or system overrides
Misused APIs, non-optimal settings
Application Health Check
Under the Hood
Architecture, data, community
Platform Upgrades As a Service 10
Repositories
Maven GitHub
Jenkins
Application Bundles
REST API
Orchestration Service Architecture
Service Tasks
Service Commands
GitHub Maven Jenkins
Cloud Email External Tool
PAASApp Config and Health
We
b U
I
Code
100% JavaRESTEasy
Platform Upgrade
Emergency Patch
Upgrade Module
Platform Upgrades As a Service 11
Example use caseUpgrade project from platform version 1.1 to 2.2
Upgrade Module Architecture
1.0 1.1 1.22.0
2.1 2.2
Upgrade10to11
Upgrade11to12
Upgrade12to20Logger
upgrade
Package refactoring
API refactoring
Upgrade20to21
Upgrade21to22
Platform versions
Upgrade paths
Add-ons
Code90% Scala10% JavaEclipse JDT
Chain of commands
Platform Upgrades As a Service 12
Telemetry Driven Product Evolution
The journey so far: 8 months, 550+ projects, 99% adoption
•Full data about the submitted upgrade requests•Data collection from Maven repository to facilitate move and cleanup
Data
•Service usage, success rate, artifact usage•Effective troubleshooting•Traceability for post-mortems and platform improvements
Statistics
•CI build detects usage of old platform version•Runs the upgrade•User receives a pull request with instructions
Proactive upgrades
•Use the Git log to identify upgrade pain points•Rely on hard data vs customer complaint to fine tune the upgrade service
Full traceability
Platform Upgrades As a Service 13
Open Source: Ostara
• What does it do?– The core functionality of the ebay platform upgrade solution– Suitable for but not exclusive to Maven and Spring based Java projects
• Development strategy– Keep generic non-ebay features in the Ostara code base– Incubation modules in both ebay and Ostara repositories– Build community & extend with more use cases
ebay platform upgrades
Ostara
ebay extensions
Code https://github.com/eBay/ostaraLicense ApacheCommunity https://groups.google.com/d/forum/ostaraoss
Platform Upgrades As a Service 14
Future Directions
• Create a simple upgrade path with the patch• On all affected applications, apply the patch in their Git
repositories, build/test/deploy the patched applications
Emergency patches
• Use semantic analysis of code to detect types• Apply all applicable refactoring patterns• Allows a more fast paced API cleanup, reduces lifetime of
deprecated APIs
Deep API refactoring
• Extend the project to non-Java platforms like node.js, Scala/AkkaPolyglot use
cases
Q&A
Global Platforms and Infrastructure
Robert [email protected]
Communityhttps://github.com/eBay/ostarahttps://groups.google.com/d/forum/ostaraoss
