Berlin Apache Flink Meetup #9Community Update

July 2015

Robert MetzgerCommitter and PMC

Memberrmetzger@apache.org

@rmetzger_

2

Apache Flink is an open source platform for scalable batch and stream data processing.

Apache Flink is …

flink.apache.org

• The core of Flink is a distributed streaming dataflow engine.• Executing dataflows in

parallel on clusters• Providing a reliable

foundation for various workloads

• DataSet and DataStream programming abstractions are the foundation for user programs and higher layers

3

One engine for many use cases

flink.apache.org

Real time streaming topologies

Machine Learning at scale

Graph Analysis

Long batchpipelines

4

What happened?• Flink on Wikipedia: https://

en.wikipedia.org/wiki/Apache_Flink • New JobManager Dashboard• Apache SAMOA 0.3.0-incubating with

Flink integration• New “Features” page• Contributors list (can you spot your

name?)https://cwiki.apache.org/confluence/display/FLINK/List+of+contributorsflink.apache.org

5

New JobManager Dashboard

flink.apache.org

6

New JobManager Dashboard

flink.apache.org

7

New Features page

flink.apache.org

8

New Architecture diagram in 0.10 documentation

flink.apache.org

9

Now in master (0.10-SNAPSHOT)

flink.apache.org

• Low watermarks / Event time• New JM Dashboard• Akka messages are now aware of

leader IDs (for HA)• Zookeeper integration (for HA)• Live accumulators (runtime only)• Stability improvements

10

Articles and Meetups

flink.apache.org

1. Meetup in Chicago(Slides [2])

Meetup in Hamburg [3]

[1] http://www.diva-portal.org/smash/get/diva2:843219/FULLTEXT01.pdf[2] http://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi[3] https://www.smaato.com/big-data-nosql-meetup-hamburg-with-apache-flink-at-smaato/

• Master Thesis: Streaming Predictive Analytics on Apache Flink [1]

11

Happy users on Twitter

flink.apache.org

12

GitHub stats

flink.apache.org

520+ Stars

13

Upcoming

• Chicago: Flink Training @ Capital One

• Bay Area: Stream & Graph Processing @ MapR

flink.apache.org

14

Flink Forward registration & call for abstracts is open now

flink.apache.org

• 12/13 October 2015• Kulturbrauerei Berlin• With Flink Workshops / Trainings!