28
Durable Streaming and Enterprise Messaging Jay Hurst Director, Product Management [email protected] @extraidea John Brock SMTS – Enterprise Messaging [email protected] @_ johnbrock

Durable Streaming and Enterprise Messaging

Embed Size (px)

Citation preview

Page 1: Durable Streaming and Enterprise Messaging

Durable Streaming and Enterprise Messaging

 Jay Hurst  Director, Product Management  [email protected]  @extraidea

 John Brock  SMTS – Enterprise Messaging  [email protected]  @_ johnbrock

Page 2: Durable Streaming and Enterprise Messaging

Safe Harbor

 Safe harbor statement under the Private Securities Litigation Reform Act of 1995:

 This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.

 The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.

 Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

Page 3: Durable Streaming and Enterprise Messaging

Jay Hurst Director - Product Management, Salesforce [email protected] @extraidea

Page 4: Durable Streaming and Enterprise Messaging

John Brock SMTS – Enterprise Messaging [email protected] @_ johnbrock

Page 5: Durable Streaming and Enterprise Messaging

Streaming Data from Salesforce

 Many customers are facing a set of challenges around pushing changes out of Salesforce.

 Raise your hand if you:

•  Have used the Classic or Generic Streaming API in Salesforce

•  Have tried to build synchronization or change detection based on streaming, but are having issues

•  Have a need for a reliable, durable flow of data in and out of Salesforce

Page 6: Durable Streaming and Enterprise Messaging

Enterprise Messaging

Page 7: Durable Streaming and Enterprise Messaging

Enterprise Messaging

 The goal is to support the concept of an Event as a first class object

•  An event is an immutable, time-stamped set of values.

•  Can represent multiple values

•  Actions - "A report was generated", "An order has been placed"

•  Measurements – "API limits are at 50%", "1MM records have been created"

•  State Changes - "Account 0010000023234 field Name was updated to 'Acme, Inc.'"

•  Allow a large number of events to flow through the platform

•  Scale to millions of events per minute per pod

•  API first design to support:

•  Publishing events to a message channel

•  Consuming events from a message channel

•  Provide ability to replay events from a defined time-window

Page 8: Durable Streaming and Enterprise Messaging

Where are we today? Classic Streaming API

Based on the Bayeux/CometD protocol

•  The client will open up a long polling connection to the Salesforce server and subscribe to a Push Topic

•  The client must regularly reconnect to keep the session alive

•  When a record meets the query, we send an event down the connection to the client

•  The client then reconnects to the long polling connection and waits for the next message

 Allows you to define a query and have Salesforce send an event to your listener whenever a record is modified and meets that query

•  The messages are sent in near real time

•  The Streaming API is not loss-less

•  If the client is not connected, the message will not be received, and cannot be replayed

Page 9: Durable Streaming and Enterprise Messaging

Where are we going tomorrow (or shortly thereafter)? Durable Streaming API

 The first implementation of Enterprise Messaging will be to enhance the Streaming API

•  Provide the ability for replay of Generic Streaming API events (Durable Streaming)

•  Existing Streaming Channel pushes will generate the Conduit events

•  Additional Event ID will be provided in message

•  Event ID will be atomic and ever-increasing

•  A new replay method will be available that will replay all events from a provided ID

•  The replay will re-deliver all messages that are after the provided ID

•  Event replay will be available for up to 24 hours

•  Protocol (Bayeux/CometD) will remain and require changes only to take advantage of replay

•  Clients will need to store and provide the Event ID

•  If no event ID provided in subscribe, then the events will be delivered from the tip of the queue

•  This will result in the existing behavior available today

Page 10: Durable Streaming and Enterprise Messaging

Durable Streaming Demo

Page 11: Durable Streaming and Enterprise Messaging

Durable Streaming Code Samples

Page 12: Durable Streaming and Enterprise Messaging

Code Sample Javascript Client Replay Extension

Page 13: Durable Streaming and Enterprise Messaging

Code Sample Javascript CometD Example

Page 14: Durable Streaming and Enterprise Messaging

Code Sample Java Client Replay Extension

Page 15: Durable Streaming and Enterprise Messaging

Code Sample Java CometD Example

Page 16: Durable Streaming and Enterprise Messaging

Event Architecture

Page 17: Durable Streaming and Enterprise Messaging

Enterprise Messaging – Event Time Window

 The Event Time Window allows for events to be added in time order to the channel

•  Events are added through a producer API and retrieved from the channel via the consumer API

•  The "window" will slide with time, and any events within the window can be replayed

•  Events are ordered

•  Replay events from anywhere in event log window

Page 18: Durable Streaming and Enterprise Messaging

Event Architecture – Conduit API

 Conduit API Allows for Producing and Consuming Events

•  Simple publisher and consumer API

Page 19: Durable Streaming and Enterprise Messaging

Event Architecture – Conduit SPIs

 Conduit SPI Allows a common API for multiple Time Ordered Event Log implementations

•  Conduit SPI interface allows multiple concrete event log implementations

•  Kafka, SQL, HBase, In Memory, etc...

•  Provides a common API across all providers

•  The SPIs will also allow for composing event logs into different SLAs

Page 20: Durable Streaming and Enterprise Messaging

Streaming API v2 How do Conduit and Streaming v2 Interact

 Runs as an embedded or stand-alone service

 Client interface via CometD

 Event processing pipeline

 Conduit - SPI

•  Use different data stores

•  Plug and play

•  Holds client information

•  Optimizes event replay

•  Plus much more!

Page 21: Durable Streaming and Enterprise Messaging

App Server

Oracle

Clients

Current Streaming API

Streaming Events Table

Conduit Events Table

Winter ‘16

Page 22: Durable Streaming and Enterprise Messaging

App Server

Current Streaming API

Clients

Internal Conduit Push API

Kafka Cluster

Future

Page 23: Durable Streaming and Enterprise Messaging

App Server

Kafka Cluster

Clients

Internal Conduit Push API

HBase

Future++

Page 24: Durable Streaming and Enterprise Messaging

Enterprise Messaging – Roadmap (Safe Harbor J)

 Winter '16

•  Durable Generic Streaming (Limited Pilot)

 Spring '16

•  Durable Generic Streaming GA

•  Durable Classic Streaming Pilot

 Summer '16

•  Durable Classic Streaming GA

•  Kafka-Backed Queues

•  Conduit Producer API public implementations for Apex

•  Conduit Consumer API public implementations for Apex

•  Conduit API implementations for Workflow/Process Builder

Page 25: Durable Streaming and Enterprise Messaging

What did we Learn?

 Existing Features that begin to support messaging

•  Classic Streaming API

•  Generic Streaming API

 Information on Durable Streaming

 Demo of Durable Streaming

•  Showed an example of Generic Durability

•  Showed an example of Classic Durability

 Roadmap for Event Messaging

Page 26: Durable Streaming and Enterprise Messaging

Q&A

Page 27: Durable Streaming and Enterprise Messaging

Share Your Feedback, and Win a GoPro!

3 Earn a GoPro prize entry for each completed survey

Tap the bell to take a survey 2Enroll in a session 1

Page 28: Durable Streaming and Enterprise Messaging

Thank you