HyperBatch

HyperBatchDaniel PETERLead Applications Engineer, KenandySalesforce MVPBay Area Salesforce Developer User Group Organizer20x [email protected]@danieljpeter

A Hyper-Fast Batchable Interface for Salesforce

Apex Database.Batchable

Case study: Account / Contact Batches

• Prerequisite: 121K Accounts already in the system

• CreateContactsBatch: Creates 3 Contacts for each Account with a random “probability” field for each. 363k Contacts total.

• UpdateAccountsBatch: For each Account, update the highest and lowest probability on the Account by querying the child Contacts. Get the overall highest and lowest probability across all the Accounts.

• DeleteContactsBatch: Delete all of the Contacts in the system. Keep a running total of how many get deleted.

Speed!Why HyperBatch?

CreateContactsBatch

Using traditional Apex Database.Batchable: 45 mins


UpdateContactsBatch

Using traditional Apex Database.Batchable: 10 mins


DeleteContactsBatch

Using traditional Apex Database.Batchable: 33 mins total (got row lock errors, had to run twice)


CreateContactsHyperBatch

Using HyperBatch: 2 mins 12 seconds


UpdateContactsHyperBatch

Using HyperBatch: 38 seconds


DeleteContactsHyperBatch

Using HyperBatch: 1 min, 11 seconds



Operation Database.Batchable HyperBatch Difference Percentage

CreateContacts 45 2.2 42.8 4.9%UpdateContacts 10 0.6 9.4 6.3%

DeleteContacts 33 1.2 28 3.6%Total 88 4 84 4.6%

SummaryRunning all 3 example batch jobs takes only 4 mins instead of 88 mins.

You save 84 mins.

It only takes 4.6% of the time!


User ExperienceWhy HyperBatch?

Traditional Apex batch HyperBatch

ConcurrencyWhy HyperBatch?

Max concurrent jobs processing

Apex Batch: 5 per org

HyperBatch: ? 20 / user?

ConcurrencyWhy HyperBatch?

Row lock behavior

Apex Batch: default is a failed batch execution. Retry logic can be built, but it will likely exceed the transaction limits.

HyperBatch: row locks retry automatically until the transaction succeed. Each re-attempt gets a new context!

SummaryHow it works

• HyperBatch interface that mimics the Database.Batchable interface.

• Browser orchestration for selecting jobs and running them on-demand.

• Lightning Design System, Visualforce (Lightning Components would be a data bottleneck).

• AJAX toolkit for PK chunking the query locator.

• Parallel remote actions fire the qeueables for the batch executions methods. (Not serial!)

• Wrapping requests in unique identifiers for closed loop execution – JavaScript function binding.

• JavaScript polls for the status of the qeueables, waiting for them to complete.

• Each execute can return some state of type Object, it can be anything. These are stored in a custom object, and a list of them is returned to the finish() method, then they are deleted.

InterfaceHow it works

HyperBatch Architecture

Roadmap

• Enhance the user interface

• Test methods

• Support custom iterators instead of just query locator

• Support simple data operations like update a field or delete records without having to write Apex

• Chunk in 2 dimensions: (Parent Id, then Record Id) to avoid row lock errors

• Salesforce Developer Blogs: “Data Chunking Techniques for Massive Orgs“ by Daniel Peter(https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking-techniques-massive-orgs.html)

• Presentation from Forcelandia 2016: “PK Chunking – Divide and conquer massive objects in Salesforce” (http://www.slideshare.net/danieljpeter/forcelandia-2016-pk-chunking)

• GitHub repo: HyperBatch (https://github.com/danieljpeter/HyperBatch)

Resources

Q & ADaniel PETERLead Applications Engineer, KenandySalesforce MVPBay Area Salesforce Developer User Group Organizer20x [email protected]@danieljpeter

MerciDaniel PETERLead Applications Engineer, KenandySalesforce MVPBay Area Salesforce Developer User Group Organizer20x [email protected]@danieljpeter

Technology

HyperBatch