Sterling Quinn´s great presentation about map caching on Esri European User Conference 2011.
- 1. Advanced Map Caching TopicsTecnicas avanzadas de almacenamiento en cache de mapasEsri European User Conference 2011
2. Prerequisite knowledge What is map caching? How to author a map for caching How to build map cache tiles 3. Topics in this session Whats new in caching in ArcGIS 10 Approaching a large caching job- Building a test caches- Choosing the tiles to cache- Monitoring the tile creation- Case study: State of Indiana imagery basemap Cache update strategies 4. Whats new in caching in ArcGIS 10 5. Mixed mode image format Creates JPEG tiles unlesstransparentpixels detected If transparent pixels detected,creates PNG32 Enormous space savings forlarge caches Mixed format helps with thisJPEG on JPEG cache overlay problem 6. How the mixed format creates tiles 7. Compact cache storage format Stores tiles in compact, continuous file streams(bundles)- Maximum ~16,000 tiles per bundle Faster copying Smaller size on disk Generally faster to create Less fragmentation 8. Move spatial subsets of tiles between caches New export and import tools allow collaborative andbest available cachingExportImport 9. Cache export tool Export Map Server Cache - Based on extent or polygon features - Convert storage format - Use for cache import or as a disconnected cacheExport Exported using Nevada and Utah state boundary features. 10. Cache import tool Import Map Server Cache -Based on extent or polygon features -Must have same storage formatImport Import from a previously exported map cache. 11. Cache import for imagery Merge high resolution imagery into a base cache Seamless integration import feature boundary 12. Collaborative caching Use export and import tools- Import the best available cached tiles- Esri Community Base Maps programPasadena City College Contribution 13. Caches treated as a raster dataset in ArcGIS Add Data and browse to cache directory.- Looks like any other raster Export caches for disconnected field work- Export to compact format recommended 14. Approaching a large caching job 15. What should you cache? Base maps Operational layers that satisfy one of the following:- High volumes of traffic- Dont change often- Cover small scales only 16. Choosing scalesSample 10 level cache Build just the scales you need Determine closest scale (Rasterresolution) Divide scale by 2 for eachsubsequent scale Adjust smallest scale to fullextentFinal level is ~75% of the total Consider Web Mercator scales 17. Choosing an image format Image format affects- Tile storage space requirements- Web application performance (speed and supportedbrowsers)- Tile image quality- Tile transparency 18. Basemap: Image format guidelines Fewer colors (~256)- PNG 8 Many colors- JPEG - Small file size - Quality settings range from 55 (orthos) - 90 (vector maps) 19. Choosing JPEG quality Which one looks better?JPEG 90 21KB JPEG 55 15KBJPEG 55 10KB JPEG 90 33KB 20. Operational layers: image format guidelines Vector overlays (roads, boundaries)- PNG 32 because of good antialiasing Other overlays (weather, etc)- MIXED or PNG (bit depth depends on number of colors) 21. Example: Tiles are too large Aerial photo andvector blend using PNG 32 22. Antialiasing: Often worth the cost 23. Apply antialiasing in the MSD for best performance 1. 2.Gives you this Box already checked shows that antialiasing is happening through MSD 24. Reducing duplicate labels Caches drawn using supertiles Duplicates can occur across supertile SupertileNo antialiasingboundaries4096x4096Supertile Solution is annotationantialiasing2048x2048 512x512 tile size256x256 tile sizeArcGIS Online / Bing / Google 25. B u i l d i n g a t e s t c a c h e 26. How to build a test cache1. Select test area with varying geography2. Create a simple feature class covering the test area3. Create tiles at all scale levels based on this feature class 27. Note the following from the test cache Appearance of tiles Performance of tiles in client Cache creation time Cache size on disk 28. Choosing the tiles to cache 29. Not all tiles are created equalhttp://hotmap.msresearch.us 30. Use GIS modeling to predict frequently visited areasThis feature class covers 25% of Californias areabut includes 97% of its population 31. Tiles you dont want to create and store Cache on demand OR Insert a Data not available tileMake your own heatmap of tile usage: ArcGIS Server descriptive tile analysis 32. Caching based on a feature class Features should be:- Big- Generalized 33. Why do the features need to be big? How do I find out the bundlesize?- Map Server Cache TilingScheme to Polygons tool- See Esri Dev Summit 2011presentation online for fulldemo 34. Monitoring the tile creation 35. Setting the number of instances Keep your CPU below 100% Start with N+1 instances where N = the number of coreson server Adjust number of instances based on test cacheTask Manager Manage Map Server Cache Tiles2 CPUs3 Instances 36. Generate using multiple LAN hosts Configure instances for CPUs on all SOC hosts Enable Use local cache directory Temp cache SharedSOCTempCache cache hosts SOM Temp cache 37. Use status tracking with your featuresHelpful script:Create gridded feature class for tracking ArcGIS Server map caching jobs 38. Recovering from errors Generate rebuild features from error log- Convert map cache-build log failures to footprintpolygons tool Cache Validation tool- Looks for missing tiles- Looks for tiles below a certain size (white or gray tiles) - False positives with building roofs and water - Useful for validating source imagery 39. Case study: State of Indianaimagery basemap 40. Requirements Single Indiana state basemap with most recent data- Maintain 6 resolution where available ArcGIS Online/Bing/Google tiling scheme Integrate neighboring data for small scales Record metadata for imagery in basemap 41. Approach6" scale576.00 20 Individual 6" counties1,128.50 192,256.99 18Use gridded tile4,513.99 17Web Mercator Tiling Scheme9,027.98 16 18,055.96 15 36,111.91 14 Clip to state boundary 72,223.82 13144,447.64 12288,895.29 11577,790.58 101,155,581.159IN Full extent2,311,162.3184,622,324.6179,244,649.236 18,489,298.455 36,978,596.914 73,957,193.823Not Built147,914,387.602295,828,775.301591,657,550.500 42. Stats on the final cache Image format: JPEG 55 62,245,522 tiles Total cache size 452.72 GB Average tile size 7.63 KB 3 weeks to build 43. C a c h e u p d a te s tr a te g ie s 44. Can you update the cache? How current does the data need to be? How fast can the update take place? Can the update be scripted? Esri Spain elections example 45. Strategic cache updates1. Detect where changes have taken place2. Merge these features3. Use them to define the update areaHelpful scriptsShow edits since reconcileCompare two feature classes in a file geodatabase 46. Updating a cache using a staging server Staging ArcGISProduction ArcGIS Server Instance Server InstanceMap serviceLayers for TOCAll layers for and Query cartography ofmap serviceMap serviceCache folderCopy tilesCache folder(On-demand caching needs the fullmap to build the cache) 47. Updating a cache and data using a staging server Staging ArcGIS Production ArcGIS Server InstanceServer Instance All layers forcartography of map serviceLayers for TOC Copy tiles and QueryMap service Map serviceCache folder Cache folder Geodatabase ReplicationMulti-user versioned GeodatabaseGeodatabase (File or Multi-user) 48. Q u e s t i o n s