Upload
dongwon-kim
View
341
Download
1
Embed Size (px)
Citation preview
Hive, Presto, and Spark on
TPC-DS benchmark
Dongwon Kim, PhDSK Telecom
Contents• Experimental setup• Experimental results
[Experimental setup] TPC-DS dataset and query • Hive
• Entirely depend on github.com/hortonworks/hive-testbench• Distributed data generator
• A small dataset (100GB)• A large dataset (1TB)
• DDLs• External table declaration• Partitioned table declaration (ORC)
• 66 queries provided (out of 99 TPC-DS queries)
• Presto• Use hive-hadoop2 connector to read the same partitioned table• Use the same query
• Spark• Connected to Hive MetaStore to read the same partitioned table• Use the same query
[Experimental setup] Cluster setup• A single master node with 5 slave nodes
• Two 12-core processes with total 48 hyper threads• 128GB main memory• 10 HDDs
• Hadoop 2.7.3• Hive 2.1.1 + Tez 0.8.4
• A LLAP worker on a node uses 192 cores and 80GB• Presto 0.162
• A Presto worker on a node uses 192 cores and 80GB• distributed-joins-enabled = false
• Spark 2.0.2• 4 Spark executors on a node uses 192 cores and 80GB
[Experimental setup] Performance monitoring tool
• github.com/eastcirclek/swimlane-graphs• Hive/Presto/Spark task swimlane graph + Ganglia resource utilization
graph• To observe the main cause of performance bottleneck
[Experimental results] Characteristics of each engine• Hive
• Improve significantly through LLAP• Good for both small and large workload• Especially good for IO-bound workloads
• Spark• Improve CPU performance through Whole Stage Code Generation• Especially good for CPU-bound workloads• Does not outperform Hive and Presto for IO-bound workloads
• Presto• Pipelined execution to reduce unnecessary disk IOs• Good for simple queries• Works okay only when data is fit into memory
[Experimental results] Query execution time (100GB)
with query72 without query72Pairwise comparison reduction in
sum of running times Pairwise comparison reduction in sum of running times
Spark > Hive 26.3 %(1668s 1229s) Hive > Spark 19.8 %
(1143s 916s)
Hive > Presto 55.6 %(2797s 1241s) Hive > Presto 50.2 %
(982s 489s)
Spark > Presto
62.0 %(2932s 1114s)
Spark > Presto
5.2%(1116s 1057s)
Spark > Hive >>> Presto Hive > Spark >= Presto
Reversed
Gap reducedsignificantly
Hive with LLAP is good even for small workload* When comparing each pair of engines, I count queries that are completed by both of two engines.
[Experimental results] Query execution time (1TB)
with query72 without query72Pairwise comparison reduction in
sum of running times Pairwise comparison reduction in sum of running times
Hive > Spark 28.2 %(6445s 4625s) Hive > Spark 41.3 %
(6165s 3629s)
Hive > Presto 56.4 %(5567s 2426s) Hive > Presto 25.5 %
(1460s 1087s)
Spark > Presto
29.2 %(5685s 4026s)
Presto > Spark
58.6%(3812s 1578s)
Hive > Spark >>> Presto Hive > Presto > Spark
Reversed
* When comparing each pair of engines, I count queries that are completed by both of two engines.
Hive with LLAP is good for large, I/O-bound workloadPresto works okay if data fit into memory
[Experimental results] Curse of Query72 on Hive and Presto
72 89 43 63 19 3 51 52 42 55 82 29 17 25 49 39 91 40 21 13 12 73 96 48 20 85 34 84 79 32 7 26 45 27 46 88 68 15 97 92 93 66 87 24 28 56 71 83 60 76 31 64 740
200400600800
100012001400160018002000
Presto Spark
100GB dataset
72 89 43 63 19 3 42 52 55 25 49 29 17 75 82 40 88 31 13 71 91 56 85 60 68 28 46 48 26 7 51 66 21 34 27 20 45 12 87 73 79 15 32 96 84 76 39 93 92 970
200400600800
100012001400160018002000
Presto Hive
72226751979295823993219496847312187932 3 9120439852428963451555344827 7 269046681319858740667628885060291756587154254931800
200400600800
100012001400160018002000
Spark Hive
72 75 91 49 13 26 88 71 40 66 56 60 68 7 31 27 48 79 87 46 45 73 15 34 84 20 21 12 39 32 96 76 28 51 92 85 930
5001000150020002500300035004000
Presto Hive
726782929795225121122015261996392784 3 131828434552 7 344842465532738987636890799176664071935094568554605825173129884974800
5001000150020002500300035004000
Spark Hive
72 26 13 91 21 15 20 12 27 7 84 39 45 96 48 46 34 40 71 68 66 73 87 92 97 79 32 51 28 76 85 56 93 83 60 24 49 88 31 740
5001000150020002500300035004000
Presto Spark
[Experimental results] Curse of Query72 on Hive and Presto 1TB dataset
[Experimental results] Curse of Query72 on Presto and Hive
Presto SparkHive
High CPU utilizationfor a long time(looks like “plateau”)
No plateau observed thanks to
WholeStageCodeGen
plateau
w/ WholeStageCodeGen w/o WholeStageCode-Gen
CPU
Network
Disk
[Experimental results] Whole Stage Code Generation
Presto Hive
Without WholeStageCodeGeneration,“plateau” is observed like Presto and Hive
10th stage takes much longer without WholeStageCodeGeneration
plateau
[Experimental results] Whole Stage Code Generation
75 71 64 83 73 40 56 55 43 85 89 63 84 27 92 82 28 93 91 12 96 32 7 66 20 45 98 42 68 52 87 90 94 79 3 48 26 60 46 34 67 51 15 29 65 76 50 31 49 80 19 18 21 58 17 13 97 22 95 54 25 39 88 24 721
10
100
1000
w/ WSCG w/o WSCG
100GB dataset
[Experimental results] Performance of Hive w/ and w/o LLAP
67 51 70 97 92 68 46 42 71 73 90 34 39 48 84 52 43 63 98 13 3 7 79 66 20 26 27 96 21 45 89 80 12 56 15 32 40 18 49 19 55 60 54 87 31 76 75 82 22 88 28 58 94 93 95 721
10
100
1000
10000
llap container
LLAP shows improvement over container-based Tez for most queries
100GBdataset
1TBdataset
80 84 96 79 94 51 91 31 56 67 27 32 95 46 72 39 19 48 90 52 21 45 15 3 34 55 42 20 12 97 43 60 63 22 26 76 75 83 88 68 70 89 65 28 13 87 40 58 18 85 25 17 29 93 50 92 66 73 711
10
100
1000
10000
llap container
All queries : 78.3% reduc-tionAll except 72 : 36.1% reduc-tion
All queries : 44.9% reduc-tionAll except 72 : 27.9% reduc-tion
[Experimental results][Query 75] without and with LLAPwithout LLAP without LLAP
with LLAP
with LLAP
[Experimental results][Query 93] without and with LLAPwithout LLAP without LLAP with LLAP
with LLAP
[Experimental results][Query 94] without and with LLAPwithout LLAP without LLAP with LLAP
with LLAP
[Experimental results][Query 93] Difference pattern of resource utilization
Network CPU
Presto Hive
CPU Network
Spark
The end