High Performance Tabular Databinding

High Performance Tabular Databinding

Tony JuckelCargill Risk Management

Tuesday, April 19, 2011

What Are We Going to Cover

• Quick overview of the databinding API

• Some performance numbers

• Tips on getting the best performance with the current iteration of the databinding framework


Who Am I And Why Do I Care?

• Building trading applications on top of Eclipse for Cargill Risk Management

• Want to display a high volume of very volatile data

• Appreciate the Databinding APIs for ancillary benefits (table editors, viewer comparators, etc)


ID Price Delta

Obj 1 ID Obj 1 Price Obj 1 Delta



Table or Tree Viewer

Pieces of the Puzzle



ObservableListContentProvider

Domain Object 1

Domain Object 2

Domain Object 3

ID Price Delta








ID ObservableMap

Delta ObservableMap

Price ObservableMap

ID Price Delta





Domain Object 1

Domain Object 2

Domain Object 3




ID ObservableMap

Delta ObservableMap

Price ObservableMap

ID ObservableMapCellLabelProvider

Delta ObservableMapCellLabelProvider

Price ObservableMapCellLabelProvider

ID Price Delta





Domain Object 1

Domain Object 2

Domain Object 3




ID ObservableMap

Delta ObservableMap

Price ObservableMap




1) Domain Object 1 update notifies listening ObservableMaps.

ID Price Delta





Domain Object 1

Domain Object 2

Domain Object 3




ID ObservableMap

Delta ObservableMap

Price ObservableMap




2) ObservableMaps notify their label providers.

ID Price Delta





Domain Object 1

Domain Object 2

Domain Object 3




ID ObservableMap

Delta ObservableMap

Price ObservableMap




3) Label providers notify the table that their label has changed for a given content item.

ID Price Delta





Domain Object 1

Domain Object 2

Domain Object 3


Easy Performance Boosts

• SWT.VIRTUAL to speed initial table creation

• Improves initial bind even without an ILazyContentProvider

• Don’t overmutate bound state


Easy Performance Boosts

• StructuredViewer#setUseHashlookup

• Helps table efficiently find Widgets associated with an updated Content Item

• If false, we notice performance degredation at just 200 Content Items


Bug #303847

• Only affects Properties API

• MapSimpleValueObservableMap

• AbstractMap#get, #containsKey, etc

• Fixed in 3.6

• Feature patched in our app (still on 3.5)


Digging Deeper

• Sample Application

• https://github.com/ajuckel/org.example.emfdb


https://github.com/ajuckel/org.example.emfdb




Choices in Notification

• Firing Changes

• N1: Notify about each property change individually

• N2: Notify about each property change in batch (via NotificationChain)

• N3: Notify that INSTRUMENT__GREEKS has changed


Choices in ObservableMaps

• M1: Each map traverses subproperty

• M2: Share an intermediate ObservableMap and use observeDetail for property changes


Metrics

• Initial Bind Performance

• How long does it take to initially populate the table?

• Update a subset of data

• How long to update greeks for ~10% of content items?


Numbers!

1000 Content Elements1000 Content ElementsInitial Bind (ms) Update (ms)

510 9270

10,000 Content Elements10,000 Content ElementsInitial Bind (ms) Update (ms)

4325 10+ minutes

Case N(1|2)/M1:Notifying on greeks property updates, and each map doing property traversal


Numbers!


510 13134


4325 10+ minutes

Case N(3)/M1:Notifying that greeks object changed, and each map doing property traversal.


Where Has the Time Gone?

• Used VisualVM to collect profiling data

• VisualVM profiling has a very high probe effect, so profiled much smaller bind (only 100 content elements)

• Holy ObservableTracker.getterCalled(), Batman!


Profiling Results

• Number of times getterCalled() was called when updating 6 properties on each of 14 content elements from a set of 100

• Calls scale with size of ObservableSet, not number of changed elements

• (+ (* 8400 8) (* 8484 3) 166 84 2 84 83)

getterCalled()N1/M1N3/M1

9307193467


With Intermediate Map


468 8454


3852 10+ minutes

Case N(1|2)/M2:Notifying on greeks property updates, and using intermediate greeks map. Better, but still far from usable.


What’s Next?

• We’ve found that traversing properties via IObservables is VERY expensive, due to the usage of ObservableTracker.getterCalled()

• Can we create an IObservableMap that only listens to part of the property chain, yet still allows getting/setting of leaf values?


Yes We Can

• EMFPartialListeningMapDecorator

• Uses an intermediate ObservableMap as a delegate for all property listening.

• Uses IValueProperty#getValue(Object) and IValueProperty#setValue(Object,Object) to get/set leaf values without ObservableTracker.getterCalled()


Without Leaf Listening


262 10


2263 89

Case N3/M3:Only listens for changes to Instrument.greeks


Profiling Results

• Dramatic reduction in calls to ObservableTracker#getterCalled()

• Horrible class name, but finally gives acceptable performance

getterCalled()N1/M1N1/M2N3/M3

9307193467

269


In Conclusion

• Use SWT.VIRTUAL

• Use StructuredViewer#setUseHashlookup()

• Don’t use observables directly on properties of properties.

• Either project values of interest into your content item, or use something like EMFPartialListeningMapDecorator


Questions?

• [email protected]

• https://github.com/ajuckel/org.example.emfdb

• @ajuckel

• http://sputteringdigitized.blogspot.com


mailto:[email protected]

mailto:[email protected]





http://sputteringdigitized.blogspot.com

http://sputteringdigitized.blogspot.com

Technology

High Performance Tabular Databinding