Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline Deploy to Production

ADVANCED SPARKAND TENSORFLOW MEETUP

Updates and AnnouncementsAugust 4, 2016

advancedspark.com

MEETUP METRICS

• 4,000 Members in 1 Year!?

• Github Repo: 600 Stars, 230 Forks

• DockerHub Repo: 5,400 Pulls!!

MEETUP AGENDA

• Deploying and Scaling ML Models with PipelineIO Open Source (Chris Fregly)

• Fundamental Algorithms of Neural Networks (Sam Abrahams)

PIPELINE.IO

Extending Your ML Pipelines into Production

100% Open Source!

http://github.com/fluxcapacitor/pipeline

BRAINSTORMING AND VALIDATING

• Major Gaming Company

• Large Ride Sharing Service

• Popular Q & A Site

• Online Clothing Retailer

• Dominant Video Streaming

PIPELINE.IO FOCUS

• Model Deploying and Testing

• Model Scaling and Serving

• Online Model Training

• Dynamic Model Optimizing

MODEL DEPLOYING AND TESTING

Continuously Test and Deploy Models

in Production!

DEMO!

Deploy Spark ML DecisionTree to Production

Deploy to Cloudor On-Premise!

MODEL SCALING AND SERVING

DEMO!

Circuit Breakers and Request Batching

ONLINE MODEL TRAINING

• Continuous, Incremental, and Partial Training

• Kafka + Spark Streaming + Spark ML

• Real-time, Dynamic Recommendations

DEMO!

Real-time, Dynamic Recommendations

DYNAMIC MODEL OPTIMIZING

Generate OptimizedCode from Spark ML!

DEMO!

Dynamic Code Generation of DecisionTree

BECOME A CONTRIBUTOR!

WE’RE HIRING!!

• Kafka, Spark ML, and TensorFlow Contributors

• Systems Engineers

• GPU/CUDA Engineers

• C++, Java, Scala, Python

ONLYNICE PEOPLE!!

SANTA CLARA WORKSHOP, AUG 6, 2016