Predicting popular areas of a Predicting popular areas of a tiled Web map as a strategy tiled Web map as a strategy

for server-side cachingfor server-side caching

Sterling QuinnSterling Quinn

IntroductionIntroduction

This project presents a predictive model This project presents a predictive model for popular areas of a Web mapfor popular areas of a Web map

Model output indicates where server-side Model output indicates where server-side cache of map tiles should be createdcache of map tiles should be created

Selectively caching based on popular map Selectively caching based on popular map areas can save time and disk spaceareas can save time and disk space

Project objectivesProject objectives

Describe server-side caching of map tilesDescribe server-side caching of map tiles

Describe the need for selective cachingDescribe the need for selective caching

Present a predictive model for popular areas of Present a predictive model for popular areas of the mapthe map

Describe ways the model could be used and Describe ways the model could be used and evaluatedevaluated

Web map optimization and the Web map optimization and the advent of server-side cachingadvent of server-side caching

Organizing large maps in Organizing large maps in manageable “tiles” is not newmanageable “tiles” is not new

Large paper map Large paper map series are indexed in series are indexed in organized gridsorganized grids

CGIS, a pioneering CGIS, a pioneering GIS, used “frames” to GIS, used “frames” to organize data (right)organize data (right)

From Tomlinson, Calkins, & Marble, 1976, p. 56.

Other techniques for organizing Other techniques for organizing maps in tiles or grid systemsmaps in tiles or grid systems

Pyramid technique Pyramid technique successively successively generalizes rasters in generalizes rasters in groups of four cells groups of four cells (right)(right)

Quadtree structures Quadtree structures index datasets in a index datasets in a hierarchy of hierarchy of quadrantsquadrants

From De Cola & Montagne, 1993, p. 1394.

Server-side caching of map Server-side caching of map tiles is newtiles is new

Tiled maps allow users to retrieve just the Tiled maps allow users to retrieve just the needed pieces of the mapneeded pieces of the map

Cached map tiles give extremely fast Cached map tiles give extremely fast performanceperformance

Traditional map servers (ArcIMS, WMS) draw Traditional map servers (ArcIMS, WMS) draw the image on the flythe image on the fly

Early static map servers returned the entire map Early static map servers returned the entire map at onceat once

Advent of tiled maps and server-Advent of tiled maps and server-side cachingside caching

Microsoft Terra Server an early Microsoft Terra Server an early deployment of massive amounts of cached deployment of massive amounts of cached imagery tilesimagery tiles

Google Maps serves cached map tiles Google Maps serves cached map tiles with AJAX techniques to create a with AJAX techniques to create a “seamless” Web mapping experience“seamless” Web mapping experience

Many sites have followed Google’s patternMany sites have followed Google’s pattern

Caching optionsCaching options

Current caching optionsCurrent caching options

Current GIS software allows analysts to Current GIS software allows analysts to create tile caches for their own mapscreate tile caches for their own maps ESRI’s ArcGIS ServerESRI’s ArcGIS Server MapnikMapnik Microsoft MapCruncherMicrosoft MapCruncher

Caching can require enormous Caching can require enormous resources on the serverresources on the server

Caches covering big areas at large scales Caches covering big areas at large scales can include millions of tilescan include millions of tiles Many gigabytes, or even terabytes of storageMany gigabytes, or even terabytes of storage Days, weeks, or sometimes months to Days, weeks, or sometimes months to

generategenerate

Many GIS shops lack resources to Many GIS shops lack resources to maintain large cachesmaintain large caches

Selective caching as a strategy for Selective caching as a strategy for saving resourcessaving resources

Administrator can cache only the areas Administrator can cache only the areas anticipated to be most visitedanticipated to be most visited

Remaining areas can be:Remaining areas can be: Added to the cache “on-demand” when first Added to the cache “on-demand” when first

user navigates thereuser navigates there Covered with a “Data not available” tileCovered with a “Data not available” tile Left blankLeft blank

Implications of selective cachingImplications of selective caching

Wise because some tiles (ocean, desert) Wise because some tiles (ocean, desert) will rarely, if never, be accessedwill rarely, if never, be accessed

Requires an admission that some areas Requires an admission that some areas are more important than othersare more important than others

Poses challenge of predicting popular Poses challenge of predicting popular areas before the map is released.areas before the map is released.

The need for a predictive modelThe need for a predictive model

Project presents a predictive model Project presents a predictive model for where to pre-cache tilesfor where to pre-cache tiles

““Which places are most interesting?”Which places are most interesting?”

Inputs are datasets readily available to Inputs are datasets readily available to GIS analystGIS analyst

Output vector features are a template for Output vector features are a template for where to pre-cache tileswhere to pre-cache tiles

Purpose of the modelPurpose of the model

Help majority of users see a fast Web map Help majority of users see a fast Web map while minimizing cache creation time and while minimizing cache creation time and storage spacestorage space

Not a descriptive modelNot a descriptive model

Descriptive model would Descriptive model would show where existing show where existing users have already users have already viewedviewed

Microsoft Hotmap a good a good example of a descriptive example of a descriptive tool (right)tool (right)

Descriptive models are Descriptive models are useful in deriving useful in deriving predictive modelspredictive models

Source: Microsoft Hotmaphttp://hotmap.msresearch.us

Advantages of a predictive modelAdvantages of a predictive model

Doesn’t require the map to be deployed Doesn’t require the map to be deployed alreadyalready

Can include fixed and varying geographic Can include fixed and varying geographic phenomenaphenomena

Has applications far beyond map cachingHas applications far beyond map caching

Proposed methodsProposed methods

Study area and conditionsStudy area and conditions

Model will predict popular places for a Model will predict popular places for a general base mapgeneral base map

Study area of CaliforniaStudy area of California

May create models for thematic maps if May create models for thematic maps if time allowstime allows

Input datasetsInput datasets

Populated / developed areasPopulated / developed areas

Road networksRoad networks

CoastlinesCoastlines

Points of interestPoints of interest

Populated / developed areasPopulated / developed areas

Human Influence Human Influence Index grid created by Index grid created by the Socioeconomic the Socioeconomic Data and Applications Data and Applications Center (SEDAC) at Center (SEDAC) at Columbia UniversityColumbia University

Model selects all grid Model selects all grid cells over a certain cells over a certain valuevalue

Road networksRoad networks

Major roads buffered by a given distanceMajor roads buffered by a given distance

AllAll roads within national parks, roads within national parks, monuments, historical sites, and monuments, historical sites, and recreation areas, buffered by a given recreation areas, buffered by a given distancedistance

CoastlinesCoastlines

All coastlines buffered by a given distance All coastlines buffered by a given distance (wider buffer on inland side)(wider buffer on inland side)

Points of interestPoints of interest

Set of 50 interesting points chosen by model Set of 50 interesting points chosen by model authorauthor Mountain peaksMountain peaks Theme parksTheme parks Sports arenasSports arenas Etc.Etc.

Represents a flexible layer that could be tailored Represents a flexible layer that could be tailored to local needsto local needs

Deriving the outputDeriving the output

Merge all layers togetherMerge all layers together

Clip to California outline buffered by ½ Clip to California outline buffered by ½ milemile

Remove small holes and polygonsRemove small holes and polygons

Dissolve into one multipart featureDissolve into one multipart feature

Using the model outputUsing the model output

Output is a vector dataset that can be used as a Output is a vector dataset that can be used as a template for creating cached tilestemplate for creating cached tiles

Compare model output area with total area to Compare model output area with total area to understand percent coverageunderstand percent coverage

Compare model output with actual usage over Compare model output with actual usage over timetime

Refine if necessaryRefine if necessary

LimitationsLimitations

Models on a world or country level should Models on a world or country level should account for Internet connectivityaccount for Internet connectivity

Input datasets have varying collection datesInput datasets have varying collection dates

Input datasets vary in resolution and precisionInput datasets vary in resolution and precision

Maps with many scales might require multiple Maps with many scales might require multiple iterations and variations of the modeliterations and variations of the model

Questions?Questions?