If you can't read please download the document
Upload
-
View
19.351
Download
2
Embed Size (px)
Citation preview
uDig Introduction
WMS Performance Tests!Mapserver & Geoserver
FOSS4G 2007
Presented by Brock Anderson and Justin Deoliveira
Shapefiles vs. PostGIS,Concurrency,and other exciting tests...
Brock:
Presentation Outline
Goals of testing.
Quick review of WMS.
Description of the test environment.
Discussion of performance tests and results.
Questions.
Brock:
WMS Performance
Goals
1. Compare performance of WMS GetMap requests in Mapserver and Geoserver.
2. Identify configuration settings that will improve performance.
3. Identify and fix inefficiencies in Geoserver.
We do not test stability, usability, etc.,
We do not test styling or labelling.
We focus on vector input.
*
Brock:
WMS Performance
Keeping the tests fair
Not an easy job!
We tried to understand what each server does under the hood to ensure we're not accidentally performing unnecessary processing on either server.
Justin: -explain the two out of the box behave quite differently - fast cgi vs database connection pooling - tried to make as equal possible ==> GeoSever doing < 1 bit png output (ask andrea about appropriate terminology)
Web Map Service (WMS)
WMS
User
http://server.org/wms? request=getmap& layers=states,lakes& bbox=-85,36,-60,49& format=png&...
A Map
Justin: - Breif overview of WMS for the layman
WMS Performance
Test Environment
ShapefilesMapserver 4.10.2
Server Computer
Client Computer
JMeter 2.2
WMS requests
Data
Additional Server Specs: Dual core (1.8Ghz per core). 2GB RAM. 7200RPM disk. Linux. PostgreSQL 8.2.4. PostGIS 1.2.
Apache 2.2.4 (with mod_fcgi)
Geoserver 1.6 beta 3
Tomcat 6.0.14
WMS requests
Vector Data
Brock:
WMS Performance
Test #1: PostGIS vs. Shapefiles
Two Data Sets:
3,000,000 Tiger roads in Texas
10,000 Tiger roads in Dallas, Texas
Both data sets are in PostGIS and shapefile format.
Spatial indexes on both data sets.
Mapserver and Geoserver layers point at the data.
Minimal styling.
JMeter issues WMS requests to fetch ~1,000 features, limited by the 'bbox' parameter.
And the results are...
Brock: - random bounding box generation - exact same requests for each server - 10,000 subset of 3 million
WMS Performance
Test #1: PostGIS vs. Shapefiles
Notes: This test uses two different data sets: one with 3 million features, the other with 10,000. Each bar is an average of 30 sample WMS requests, each using a different bounding box to fetch and draw appx. 1000 features (+/- 15%). The same 30 requests are executed for each scenario. One request at a time (no concurrency). Mapserver and Geoserver use the same data. Mapserver is using FastCGI via Apache/mod_fcgi. Spatial indexes on both data sets. Quadtree indexes generated by 'shptree'. No reprojection required. Minimal styling. Responses are 1-bit PNG images.
Brock:
Response time in milliseconds on the Y-axis.
Three interesting ways to look at the graph- PostGIS vs. Shapefile- Indexing (large data set vs. small data set)- Mapserver vs. Geoserver-
Geoserver had many code changes to improve performance.
WMS Performance
Test #2: Concurrent Requests
Using the same tiger roads data set with 10,000 records.
We issue multiple requests with pseudo-random BBOXes that fetch approximately 1,000 features.
The main difference is that now we're issuing multiple concurrent requests.
Let's see what happened...
Justin: - explain how the concurrency, we chose a worst case concurrency model, there are many others
WMS Performance
Test #2: Concurrent Requests
Notes: Data in PostGIS and shapefile formats. Mapserver and Geoserver use the same data. Mapserver is using FastCGI via Apache/mod_fcgi. 20 FastCGI mapserv processes. Geoserver uses connection pooling with 20 connections. Spatial indexes on both data sets. No reprojection required. Minimal styling. Responses are 2-color PNG images. More details in the appendix.
Justin: - suprising results - expected postgis to perform much better under a load - however, read-only access means no locking for shapefiles - Mapserver with fast cgi with 20 processes - GeoServer with connectino pooling at 20 connections
WMS Performance
... or Throughput, if you prefer
An alternative way to summarize the data collected for the concurrency test. (Higher lines are better here.)
Justin: - same data, different metric, througput - mapserver remains consistent for longer, geoserver starts to degrade around 15 concurrent requests (garbage collector?)
Wms Performance
Test #3: Reprojection
Currently Mapserver calls PROJ for every vertex, but it could improve by batching those into a single call.
PROJ optimizes by assuming these source and target datums are equivalent.
Geotools is slightly faster than PROJ for these cases.
Geoserver simplifies geometry before reprojecting.
0
6
22
9
12
0
4
19
21
31
Brock:
This test only scratches the surface.
At least it gives a sense of how much on-the-fly reprojection affects response time.
WMS Performance
CGI vs. FastCGI (Mapserver only)
Notes: Average of 30 samples. One request at a time (no concurrency). Each request fetches one layer with 1000 features from a data set of 10,000. Spatial indices on both data sets. No reprojection required. Minimal styling. Responses are 1-bit PNG images. The same binary file was used for both CGI and FastCGI. FastCGI through Apache and mod_fcgi.
Brock:
What is FastCGI?
Conclusion: use FastCGI unless you have compelling reasons no to.
WMS Performance
Breakdown of Mapserver Response Time
FastCGI eliminates Start mapserv process and Connect to DB costs.
The Write image step is dependant on output format.
Brock
WMS Performance
Breakdown of Geoserver Response Time
404: Document not found
WMS Performance
Servlet Container and Java (Geoserver only)
These results show average response times for the same WMS request when Geoserver is backed by different Servlet containers and Java versions.
Using shapefile backend.
Conclusion: Use Java 6!
Tomcat 6 doesn't support Java 1.4
Justin
WMS Performance
Outcome of the tests
Lots of performance optimizations to Geoserver which will be available in version 1.6.
Identified a few places where Mapserver can improve too. (These will be reported as bugs as time permits.)
Both servers can be FAST, but require some special configuration.
WMS Performance
The Road to Speed
All will be in Geoserver 1.6
Data sources with high connection overhead will benefit much more from FastCGI.
Brock does mapserverJustin does GeoServer - out of the box geoserver much slower ==> defautl 24 bit rendering, logging on, default styling has transparency , anti-aliasing on by default
Performance Tips (Mapserver)
Beware of
PROJECTION
'init=epsg:4326'
END
The init= syntax causes one lookup in the PROJ4 'epsg' file for every occurrence in the map file. (Move your most-used EPSG codes to the top of the 'epsg' file.)
Use FastCGI instead of ordinary CGI. Instruction here: http://mapserver.gis.umn.edu/docs/howto/fastcgi
Ensure you have enough FastCGI processes.
Brock
WMS Performance
Performance Tips (Geoserver)
Geoserver has many features enabled by default. Gain performance by disabling features you don't need.
Transparent styles double draw time. Use opacity=1 in your SLD to disable.
Antialiasing linework is costly. Try '&format_options=antialias:none' to disable.
Experiment with disabling PNG native acceleration
Favour Java 6 over Java 5 over Java 1.4.
JVM Settings: Increase heap size. Use -server switch.
Experiment with different shapefile index depths.
Turn off logging
WMS Performance
How can the servers improve?
Mapserver
More efficient scanning of shapefile quadtree indexes. [ Bug Reported ]
Batch PROJ calls when doing on-the-fly reprojection.
Reduce number of 'epsg' lookups on map files.
Geoserver
Various optimizations to the renderer.
[ Fixes Committed ]
More efficient scanning of shapefile quadtree index. [ Fixes Committed ]
WMS Performance
Questions? Contact Us.
Brock Anderson: [email protected]
Justin Deolivera: [email protected]
Introduction to uDig
General WMS Performance Tips
Only fetch from your data source the features that will be drawn, otherwise the servers have to spend time scanning and discarding the unused ones.
Output format affects response time. 256 color PNG is faster to create than PNG24 on both servers.
On-the-fly reprojection has a price. Store data in the same projection it's most commonly requested in.
WMS Performance
Appendix
Breakdown of Mapserver Response Time
The graph represents mapserv running in CGI mode to show all startup costs. Metrics for Load map file, Connect to DB, Fetch & store, Draw and Write image were collected by modifying source code to capture and log durarions of those operations. Query time measured with PostgreSQL's explain analyze command. Start mapserv process + Network delay = difference between response times recorded by JMeter and my custom mapserv logging which recorded the total time servicing a request.
PostGIS vs Shapefiles
This test uses two different data sets: one with 3,000,000 features, the other with 10,000. Each request fetches 1000 features by limiting with a 'bbox' WMS parameter. Each bar is an average of 30 samples. One request at a time (no concurrency). Mapserver and Geoserver use the same data. Mapserver is using FastCGI via Apache/mod_fcgi. Spatial indices on both data sets. The shapefile indices were generated with 'shptree'. No reprojection required. Minimal styling. Responses are 2-color PNG images (indexed color).
The unusual Mapserver result for the case of a 3 million record shapefile has been reported to the Mapserver bug tracker: http://trac.osgeo.org/mapserver/ticket/2282
WMS Performance
Appendix
Concurrency and Throughput
Notes: Data in PostGIS and shapefile formats. Mapserver and Geoserver use the same data. Mapserver is using FastCGI via Apache/mod_fcgi. 20 FastCGI mapserv processes. Geoserver uses connection pooling with 20 connections. Spatial indexes on both data sets. No reprojection required. Minimal styling. Responses are 2-color PNG images (indexed color). Concurrent requests were fired in bursts with zero ramp up (as near to simultaneously as possible). I.e. For the test of 10 concurrent requests, all ten requests were fired at the same time. Once all the responses came back then the next burst of requests went out. Requests use random bboxes which fetch ~1000 features. The same random bboxes are used against both servers.
Response times are measured in milliseconds. Throughput times represent responses per second.
The concurrency level is the left-most column in each table (1, 2, 5, 10, ...).
WMS Performance
Appendix
Summary of Geoserver code changes made to improve performance:
optimized access to the shapefile spatial index (it was reading tiny sections of the file instead of doing some buffered access)
figure out the optmimal palette out of the SLD style (when possible, that is, when antialiasing is off)
* don't access the dbf file when not necessary
* avoid unecessary operations, like duplicating over and over the same coordinate[] during rendering (loading it, generalize, reproject, copy back in the geometry and so on, now the array it's copied just once)
Raw list of changes here:
http://jira.codehaus.org/secure/ManageLinks.jspa?id=55176
WMS Performance
MapserverResponse time (milliseconds)PostGISShapefile
15039
25140
59175
10182147
15269229
20315283
40784612
601269905
Mapserver Throughput# Concurrent RequestsResponses per secondPostGISShapefile
119.624.9
228.233.4
535.451.6
1038.453.8
1542.555
2042.454.1
4043.254.9
6043.151.5
GeoserverPostGISShapefile
14227
24330
58147
10166103
15261162
20378252
40747514
601170773
Geoserver Throughput# Concurrent RequestsPostGISShapefile
124.635.6
232.341.8
547.168.6
1049.974.1
1549.273.3
2047.768
4048.368
6047.870.7
Mapserver (Response times)PostGISShapefile
15039
25140
59175
10182147
15269229
20315283
40784612
601269905
???Page ??? (???)09/21/2007, 08:42:06Page /
W W W . R E F R A C T I O N S . N E T
Introduction to uDig
2/13/05
W W W . R E F R A C T I O N S . N E T
Introduction to uDig
2/13/05
Geoserver (Response times)PostGISShapefile
14227
24330
58147
10166103
15261162
20378252
40747514
601170773
???Page ??? (???)09/21/2007, 08:42:06Page / Mapserver (Throughput times)PostGISShapefile
119.624.9
228.233.4
535.451.6
1038.453.8
1542.555
2042.454.1
4043.254.9
6043.151.5
???Page ??? (???)09/21/2007, 08:42:06Page / Geoserver (Throughput times)PostGISShapefile
124.635.6
232.341.8
547.168.6
1049.974.1
1549.273.3
2047.768
4048.368
6047.870.7
???Page ??? (???)09/21/2007, 08:42:06Page / Jetty 6.0.2Response time (milliseconds)Column D
Java 1.4179
Java 595
Java 664
Tomcat 6.0.14Column J
Java 1.41.#NAN
Java 595
Java 663
Mapserver (using PROJ to reproject) Response time (milliseconds)Mapserver
None44
Geog WGS84 UTM 14N WGS8450
Geog WGS84 UTM 14N NAD2766
Geog WGS84 SPS NAD8353
UTM 14N WGS84 - SPS NAD8356
Geoserver (using Geotools to reproject)Geoserver
None40
Geog WGS84 UTM 14N WGS8444
Geog WGS84 UTM 14N NAD2759
Geog WGS84 SPS NAD8361
UTM 14N WGS84 - SPS NAD8371
MapserverResponse time (milliseconds)PostGISShapefile
1,000 of 10,0005039
1,000 of 3,000,00047386
Response time (milliseconds)PostGISShapefile
CGI8157
FastCGI5242
Time (in milliseconds)PostGISShapefile
Start mapserv process1515
Load map file33
Connect to DB140
Query200
Fetch & store70
Draw1128
Write image88
Network delay33
GeoserverPostGISShapefile
1,000 of 10,0004227
1,000 of 3,000,0004233
PostGISShapefile
Start mapserv process15ms
Load map file3ms
Connect to DB14msn/a
Query20msn/a
Fetch7msn/a
Draw11ms28ms
Write image8ms
Network delay3ms
???Page ??? (???)08/31/2007, 08:20:50Page / Mapserver Response time (milliseconds)Column D
Start (CGI)254
Switch to FastCGI225
Re-order 'epsg' file106
Output format39
Geoserver Column H
Start800
Logging Off600
Transp. styles off500
Output format450
JVM settings300
Code change40