#atlassian

JON MORT • CONSULTANT • ADAPTAVIST • @ADAPTAVIST

Scaling Plugins in Critical Systems

Getting It Right

About Me

• Level One• Level Two• Level Two• Level Two

• Level One

Who Am IAbout Me

• Atlassian Consultant/Developer at Adaptavist • 5 years of experience developing Atlassian plugins • Plugin License Manager was my fault • Spoken at various Conferences, including Summit about:

• Atlassian development • Performance tuning

• Level One• Level Two• Level Two• Level Two

• Level One

Adaptavist - Who Are WeAbout Me

• 60 people in 4 offices in the UK, USA and Germany

• Customers in 60+ countries including over half of the Fortune 500

• Longest established Atlassian partners - since 2005

• Professional Services, Managed Services, Products and Training

• Focused on helping Enterprises plan, deliver and manage software and applications using the Atlassian toolset

• Level One• Level Two• Level Two• Level Two

• Level One

Why Should You Listen To Me?About Me

• Because I’ve messed up • A lot

• … but also made it better

Who, What, Why?

• Level One• Level Two• Level Two• Level Two

• Level One

Add-On DevelopersWho are you?

• Level One• Level Two• Level Two• Level Two

• Level One

Application AdministratorsWho are you?

• Level One• Level Two• Level Two• Level Two

• Level One

Product Owners & UsersWho are you?

Data Center

• Level One• Level Two• Level Two• Level Two

• Level One

Before Data CenterUsers

WAN

Index

Database

JIRA

• Level One• Level Two• Level Two• Level Two

• Level One

The ArchitectureWith Data Center

WAN

Users

. . .

Replicated Indexes

Multiple JIRA Nodes

Database Shared Home

Load Balancer

”

“ <plugin-info> <param name=“atlassian-data-center-compatible">true</param> </plugin-info>

NO!

NO!

C A C H E S & P O O L S

S TAT E

Scaling Plugins

C O O R D I N AT I O N

U S E R I N T E R FA C E

T E S T I N G

I N T H E C O D E

The Problem with State

Consistency Availability

Partition Tolerance

Use Provided Storage

Mechanisms

State - Do’s & Don’ts

DO Read up distributed systems theory

DO Consider and test failure modes

DO Keep as little state as possible

DO Use the appropriate supported APIs

DON’T Just assume everything will be OK

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

S TAT E

C A C H E S & P O O L S

Scaling Plugins

C O O R D I N AT I O N

U S E R I N T E R FA C E

T E S T I N G

I N T H E C O D E

Caching

Map<String, ?> cache =

Local, Cluster, Hybrid

Keys Values

Local Local Local

Cluster Replicated Replicated

Hybrid Replicated Local

Pool Party!

Be flexible to changes of Scale

• Level One• Level Two• Level Two• Level Two

• Level One

Pool Monitoring

Caches and Pools - Do’s & Don’ts

DO Cache values that are complex to create and cheap to store

DO Invalidate caches appropriately using time and freshness checks

DO Pool resources where appropriate

DO Use and know the new caching APIs

DON’T Use hash maps or statics or sessions for caching!

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

C A C H E S & P O O L S

C O O R D I N AT I O N

Scaling PluginsS TAT E

U S E R I N T E R FA C E

T E S T I N G

I N T H E C O D E

Become Immortal Become

Immutable

CONF-15313 Concurrent Modification

Exception

Synchronization Makes Me Cry

Synchronization Makes Me Cry

Cross Node Concurrency

Scheduled Jobs

Testing Concurrency

Sucks

Coordination - Do’s & Don’ts

DO Think about concurrency - read Java Concurrency in Practice

DO Use immutable data structures with reentrant logic

DON’T Be scared and ignore potential problems

DON’T Use locking (like synchronised blocks)

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

C A C H E S & P O O L S

T E S T I N G

Scaling PluginsS TAT E

U S E R I N T E R FA C E

C O O R D I N AT I O N

I N T H E C O D E

How to Lose Friends and

Alienate People

Everyone’s a bit unstable

Finding the Bottleneck

Don’t be THAT Plugin

Be Vocal

Testing - Do’s & Don’ts

DO Make it easy to detect application API changes - Integration test

DO Performance testing

DO Actively try to increase the test coverage - these are important!

DO Test at scale and announce results

DO Declare how you tested for Data Center

DON’T Ignore Security

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

C A C H E S & P O O L S

U S E R I N T E R FA C E

Scaling PluginsS TAT E

T E S T I N G

C O O R D I N AT I O N

I N T H E C O D E

One True

Plugin

Blend In

Use the ADG

Enterprise Systems do not have only

5 Projects

User Interface - Do’s & Don’ts

DO use and know the ADG

DO Think about scaled use

DO Think about the impact you might make

DON’T Think you’re the only plugin that matters

DON’T Lock the UI up

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

C A C H E S & P O O L S

I N T H E C O D E

Scaling PluginsS TAT E

T E S T I N G

C O O R D I N AT I O N

U S E R I N T E R FA C E

Watch your Weight

Put your eggs in one batch

webjars.org

Your Black Box isn’t Opaque

Atlassian Spring Scanner

Atlassian Pocketknife

Useful Libs - Do’s & Don’ts

DO Watch A P2 Plugin and a SaaS Platform Walk into a Bar… from AtlasCamp

DO Know your OSGi

DO Use Atlassian Spring Scanner

DO Look at Pockeknife

DO Use Wired Tests

DO Take advantage of batching

C O L U M N T I T L E C O L U M N T I T L E C O L U M N T I T L E

In Summary

#atlassianKey takeaways:

• Understand your state• Play nice - respect the application you execute within.• Be mindful of the user experience - does it scale?• Understand the bottlenecks within your plugin - are you lazy enough?• Be immutable, else be remembered for the wrong reasons!• Test at scale or announce that you haven’t.

Thank you!

JON MORT • CONSULTANT • ADAPTAVIST • @ADAPTAVIST