Upload
others
View
14
Download
0
Embed Size (px)
Citation preview
Introduction to Java 8
Parallelism Frameworks
Douglas C. [email protected]
www.dre.vanderbilt.edu/~schmidt
Professor of Computer Science
Institute for Software
Integrated Systems
Vanderbilt University
Nashville, Tennessee, USA
2
• Recognize how Java 8 applies functional programming features to its parallelism frameworks
Learning Objectives in this Lesson
3
Learning Objectives in this Lesson• Recognize how Java 8 applies functional programming
features to its parallelism frameworks, e.g.
• Fork-join pools
4
• Recognize how Java 8 applies functional programming features to its parallelism frameworks, e.g.
• Fork-join pools
• Parallel streamsfilter(not(this::urlCached))
collect(toList())
map(this::downloadImage)
flatMap(this::applyFilters)
Learning Objectives in this Lesson
…
5
• Recognize how Java 8 applies functional programming features to its parallelism frameworks, e.g.
• Fork-join pools
• Parallel streams
• Completable futures
Learning Objectives in this Lesson
/page\ =
supplyAsync
(getStartPage())
/imgNum\ = /page\
.thenComposeAsync
(crawlHyperLinks
(page))
/imgNum\ = /page\
.thenApplyAsync
(countImages(page))
.thenApply(List::size)
/imgNum\.thenCombine(/imgNum\,
(imgNum, imgNum) ->
Integer::sum)
Completable futures also provide a reactive asynchrony programming model
6
• Recognize how Java 8 applies functional programming features to its parallelism frameworks, e.g.
• Know how these features are applied in several example case study apps
New &Improved
Learning Objectives in this Lesson
Socket
Socket
List of URLs to Download
…
List of Filters to Apply
Persistent
Data Store
7
Overview of Java 8 Parallelism Frameworks
8
• Java 7 introduced the fork-join pool
See www.infoq.com/interviews/doug-lea-fork-join
Overview of Java 8 Parallelism Frameworks
9
• Java 7 introduced the fork-join pool
• This framework provides high performance, fine-grained task execution for data parallelism
join joinjoin
Processsequentially
Processsequentially
Processsequentially
Processsequentially
DataSource1.1 DataSource1.2 DataSource2.1 DataSource2.2
DataSource1 DataSource2
DataSource
fork()
fork() fork()
See www.dre.vanderbilt.edu/~schmidt/PDF/DataParallelismInJava.pdf
Overview of Java 8 Parallelism Frameworks
10See en.wikipedia.org/wiki/Divide_and_conquer_algorithm
Result solve(Problem problem) {
if (problem is small)
directly solve problem
else {
a. split problem into
independent parts
b. fork new sub-tasks
to solve each part
c. join all sub-tasks
d. compose result
from sub-results
}
}
• Java 7 introduced the fork-join pool
• This framework provides high performance, fine-grained task execution for data parallelism
• It supports parallel programming by solving problems via “divide & conquer”
Overview of Java 8 Parallelism Frameworks
11
• Java 7 introduced the fork-join pool
• This framework provides high performance, fine-grained task execution for data parallelism
• It supports parallel programming by solving problems via “divide & conquer”
• Employs work-stealing to optimize multi-core processor performance
See gee.cs.oswego.edu/dl/papers/fj.pdf
Overview of Java 8 Parallelism Frameworks
12
• Java 8 adds two new parallelism frameworks related to functional programming
Additional Frameworks & Languages
Applications
Operating System Kernel
System Libraries
Java Execution Environment (e.g., JVM)
Threading & Synchronization Packages
Java
/JN
IC+
+/C
CSee www.ibm.com/developerworks/library/j-jvmc2
Overview of Java 8 Parallelism Frameworks
13
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams filter(not(this::urlCached))
collect(toList())
…
map(this::downloadImage)
flatMap(this::applyFilters)
See docs.oracle.com/javase/tutorial/collections/streams/parallelism.html
Overview of Java 8 Parallelism Frameworks
14
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
• Partitions a stream into multiple substreams that run independently & combine into a “reduced” result
filter(not(this::urlCached))
collect(toList())
…
map(this::downloadImage)
flatMap(this::applyFilters)
Overview of Java 8 Parallelism Frameworks
15
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
• Partitions a stream into multiple substreams that run independently & combine into a “reduced” result
• Chunks of data in the substreamscan be mapped to multiple threads(& cores)
filter(not(this::urlCached))
collect(toList())
…
map(this::downloadImage)
flatMap(this::applyFilters)
Overview of Java 8 Parallelism Frameworks
16
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
• Partitions a stream into multiple substreams that run independently & combine into a “reduced” result
• Chunks of data in the substreamscan be mapped to multiple threads(& cores)
• Leverage the common fork-join pool
filter(not(this::urlCached))
collect(toList())
…
map(this::downloadImage)
flatMap(this::applyFilters)
Overview of Java 8 Parallelism Frameworks
See dzone.com/articles/common-fork-join-pool-and-streams
17
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
• Partitions a stream into multiple substreams that run independently & combine into a “reduced” result
• Chunks of data in the substreamscan be mapped to multiple threads(& cores)
• Leverage the common fork-join pool
filter(not(this::urlCached))
collect(toList())
…
map(this::downloadImage)
flatMap(this::applyFilters)
Parallel streams provides fine-grained data parallelism functional programming
Overview of Java 8 Parallelism Frameworks
18
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
2. Completable futures
See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html
/page\ =
supplyAsync
(getStartPage())
/imgNum2\ = /page\
.thenComposeAsync
(crawlHyperLinks
(page))
/imgNum1\ = /page\
.thenApplyAsync
(countImages(page))
.thenApply(List::size)
/imgNum1\.thenCombine(/imgNum2\,
(imgNum, imgNum) ->
Integer::sum)
Overview of Java 8 Parallelism Frameworks
19
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
2. Completable futures
• Supports dependent actionsthat trigger upon completion of async operations
See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletionStage.html
/page\ =
supplyAsync
(getStartPage())
/imgNum2\ = /page\
.thenComposeAsync
(crawlHyperLinks
(page))
/imgNum1\ = /page\
.thenApplyAsync
(countImages(page))
.thenApply(List::size)
/imgNum1\.thenCombine(/imgNum2\,
(imgNum, imgNum) ->
Integer::sum)
Overview of Java 8 Parallelism Frameworks
20
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
2. Completable futures
• Supports dependent actionsthat trigger upon completion of async operations
• Async operations can runin parallel in thread pools
See www.nurkiewicz.com/2013/05/java-8-definitive-guide-to.html
/page\ =
supplyAsync
(getStartPage())
/imgNum2\ = /page\
.thenComposeAsync
(crawlHyperLinks
(page))
/imgNum1\ = /page\
.thenApplyAsync
(countImages(page))
.thenApply(List::size)
/imgNum1\.thenCombine(/imgNum2\,
(imgNum, imgNum) ->
Integer::sum)
Overview of Java 8 Parallelism Frameworks
21
• Java 8 adds two new parallelism frameworks related to functional programming
1. Parallel streams
2. Completable futures
• Supports dependent actionsthat trigger upon completion of async operations
• Async operations can runin parallel in thread pools
Java 8 completable futures & streams can be combined to good effects!!
List of URLs to Download
…
filter(this::nonNull)
collect(toFuture())
map(this::downloadImageAsync)
flatMap(this::applyFiltersAsync)
map(this::checkUrlCachedAsync)
Overview of Java 8 Parallelism Frameworks
22
• These frameworks often eliminate the use of synchronization or explicit threading when developing parallel apps!
Alleviates many accidental & inherent complexities of parallel programming
Overview of Java 8 Parallelism Frameworks
23
• Both Java 8 frameworks use the fork-join pool framework by default
See www.oracle.com/technetwork/articles/java/fork-join-422606.html
Overview of Java 8 Parallelism Frameworks
24
Summary of Example Case Study Apps
25
• SearchStreamGang case study uses regular expressions to find phrases in the complete works of William Shakespeare in parallel
See github.com/douglascraigschmidt/LiveLessons/tree/master/SearchStreamGang
Summary of Example Case Study Apps
Starting SearchStreamGangTest
PARALLEL_SPLITERATOR executed in 409 msecs
COMPLETABLE_FUTURES_INPUTS executed in 426 msecs
COMPLETABLE_FUTURES_PHASES executed in 427 msecs
PARALLEL_STREAMS executed in 437 msecs
PARALLEL_STREAM_PHASES executed in 440 msecs
RXJAVA_PHASES executed in 485 msecs
PARALLEL_STREAM_INPUTS executed in 802 msecs
RXJAVA_INPUTS executed in 866 msecs
SEQUENTIAL_LOOPS executed in 1638 msecs
SEQUENTIAL_STREAM executed in 1958 msecs
Ending SearchStreamGangTest
26
• ImageCounter recursively crawls web pages counting # of images in parallel
Summary of Example Case Study Apps
See github.com/douglascraigschmidt/LiveLessons/tree/master/Java8/ex19
imgs2
imgs3
imgs1
uci.png
robot.png
...ka.png
...
wm.jpg
...
oz.jpg ...vette.
jpg
27See github.com/douglascraigschmidt/LiveLessons/tree/master/ImageStreamGang
Summary of Example Case Study Apps• ImageStreamGang shows how the StreamGang framework can be combined
with Java 8 streams & completable futures to download,filter, store, & display images in parallel
collect(toList())
Parallel Streams
…
filter(not(this::urlCached))
map(this::downloadImage)
flatMap(this::applyFilters)
ImageStreamGang
28
End of Introduction to Java 8 Parallelism Frameworks